Below the performance of a set of segmentation methods is given. All methods are described in [1]. Pixel error is used as evaluation measure. This error is defined as the proportion of pixels for which any of the five object labels (right lung, left lung ,heart, right clavicle, left clavicle) is not in agreement with the reference standard. Clearly, other evaluation measures could be used, and they could be applied to only certain objects. See [1] for a discussion of other measures, and use the result browser to examine the segmentation results for individual images and to get more information about the method behind these acronyms.
The table below lists the mean and standard deviation of the pixel error over all images for each method, sorted for increasing pixel error. Note that the mean shape method is a trivial segmentation method which uses the mean position of each object as result, independent of the input image.
| method | mean (std) |
| Human observer | 0.029 (0.008) |
| MCP (Dieter Seghers) | 0.033 (0.017) |
| Voting | 0.033 (0.010) |
| PC post-processed | 0.040 (0.013) |
| ASM/PC hybrid | 0.042 (0.018) |
| Pixel classification | 0.043 (0.014) |
| AAM/PC hybrid | 0.044 (0.017) |
| AAM whiskers + BFGS | 0.046 (0.018) |
| ASM tuned | 0.044 (0.014) |
| AAM whiskers | 0.051 (0.017) |
| ASM default | 0.055 (0.029) |
| AAM default | 0.080 (0.059) |
| Mean shape | 0.146 (0.043) |
The figure below shows the results of all methods as a box plot (minimum, maximum, median and first and third quartile), sorted by increasing median error.