SCR database: Results

Below the performance of a set of segmentation methods is given. All methods are described in [1]. Pixel error is used as evaluation measure. This error is defined as the proportion of pixels for which any of the five object labels (right lung, left lung ,heart, right clavicle, left clavicle) is not in agreement with the reference standard. Clearly, other evaluation measures could be used, and they could be applied to only certain objects. See [1] for a discussion of other measures, and use the result browser to examine the segmentation results for individual images and to get more information about the method behind these acronyms.

The table below lists the mean and standard deviation of the pixel error over all images for each method, sorted for increasing pixel error. Note that the mean shape method is a trivial segmentation method which uses the mean position of each object as result, independent of the input image.

methodmean (std)
Human observer0.029 (0.008)
MCP (Dieter Seghers)0.033 (0.017)
Voting0.033 (0.010)
PC post-processed0.040 (0.013)
ASM/PC hybrid0.042 (0.018)
Pixel classification0.043 (0.014)
AAM/PC hybrid0.044 (0.017)
AAM whiskers + BFGS    0.046 (0.018)
ASM tuned0.044 (0.014)
AAM whiskers0.051 (0.017)
ASM default0.055 (0.029)
AAM default0.080 (0.059)
Mean shape0.146 (0.043)

The figure below shows the results of all methods as a box plot (minimum, maximum, median and first and third quartile), sorted by increasing median error.

References

[1] B. van Ginneken, M.B. Stegmann, M. Loog, Segmentation of anatomical structures in chest radiographs using supervised methods: a comparative study on a public database, Medical Image Analysis, nr. 1, vol. 10, pp. 19-40, 2006.