Abstract Semantic dimensions such as context availability, imageability and valence, form core components of many theoretical accounts of lexical processing. Typically, normative data for such semantic dimensions are drawn from subjective ratings, however, questions have been raised regarding the reliability and validity of these ratings. In this paper, we discuss this issue with a focus on context availability norms. Using data collected for another study, we show that context availability ratings required significantly higher rates of data exclusions at the level of both participants and items compared to other variables. In addition, high standard deviations at the item level, indicated a substantial degree of disagreement between participants. This suggested that some participants may have had difficulty understanding the concept of context availability which hindered their ability to complete the ratings. We provide recommendations for future research focusing on context availability and for norm collection procedures more broadly in order that the validity of such norms can be improved. In particular, we suggest that clear guidelines are required for data cleaning in order that the reliability of such norms is maximised and to facilitate replication across studies.