PURPOSE: To determine observer performance in the detection of multiple sclerosis (MS) lesions on magnetic resonance (MR) images of the brain and to assess the dependence of observer performance on lesion size, parenchymal location, pulse sequence, and supratentorial versus infratentorial level. MATERIALS AND METHODS: This HIPAA-compliant protocol was approved by the institutional review board, and previously acquired MR data from a healthy volunteer and a patient with MS were used to derive parameter maps, with waiver of informed consent. Parameter maps and image simulator software were used to generate 320 phantom brain images with simulated supratentorial and infratentorial MS lesions. Images were displayed with T2-weighting or fluid-attenuated inversion recovery (FLAIR) contrast. Four readers independently evaluated the images, rating lesions on a five-point certainty scale. Observer performance was measured by using the area under the alternative free-response receiver operating characteristic curve (A(1)), and significance was determined with the z test. RESULTS: Pooled A(1) scores were significantly better for FLAIR imaging (0.96 +/- 0.01 [standard error]) than for T2-weighted MR imaging (0.89 +/- 0.04) supratentorially (P =.05) but were similar for FLAIR imaging (0.90 +/- 0.06) and T2-weighted MR imaging (0.88 +/- 0.05) infratentorially. A(1) scores for cortical, deep white matter, and periventricular lesions were 0.93 +/- 0.05, 0.97 +/- 0.02, and 0.89 +/- 0.04, respectively, for FLAIR imaging and 0.77 +/- 0.06, 0.99 +/- 0.01, and 0.89 +/- 0.05, respectively, for T2-weighted MR imaging. FLAIR scores were significantly higher than T2-weighted scores for cortical lesions. Linear correlation was found between A(1) and lesion size (r = 0.5). CONCLUSION: Supratentorially, performance was better with FLAIR imaging than with T2-weighted MR imaging. Infratentorially, performance was moderate with both modalities. Observers did better with FLAIR imaging in the detection of cortical lesions, and performance improved with increasing lesion size.