Severe developmental deficits in face recognition ability (developmental prosopagnosia, or DP) have been vigorously studied over the past decade, yet many questions remain unanswered about their origins, nature, and social consequences. A rate-limiting factor in answering such questions is the challenge of recruiting rare DP participants. Although self-reported experiences have long played a role in efforts to identify DPs, much remains unknown about how such self-reports can or should contribute to screening or diagnosis. Here, in a large, population-based web sample, we investigated the effectiveness of self-report, used on its own, as a screen to identify individuals who will ultimately fail, at a conventional cutoff, the two types of objective tests that are most commonly used to confirm DP diagnoses: the Cambridge Face Memory Test (CFMT) and the famous faces memory test (FFMT). We used a highly reliable questionnaire (alpha = .91), the Cambridge Face Memory Questionnaire (CFMQ), and revealed strong validity via high correlations of .44 with the CFMT and .52 with the FFMT. However, cutoff analyses revealed that no CFMQ score yielded a clinical-grade combination of sensitivity and positive predictive value in enough individuals to support using it alone as a DP diagnostic or screening tool. This result was replicated in an analysis of data from the widely used PI20 questionnaire, a 20-question self-assessment of facial recognition similar in form to the CFMQ. We therefore recommend that screens for DP should, wherever possible, include objective as well as subjective assessment tools.