Artiklis arutleme inimestelt semantilist leksikaalset infot koguva uurimuse peamiste probleemide üle. Kirjeldame katset, millega kogume konkreetsushinnanguid eestikeelsetele sõnadele. Artikli eesmärk on analüüsida semantiliste tunnuste hinnangutena kogumist kui meetodit tervikuna. Hinnangute kogumise metoodikat on vaja sisuliselt ja kriitiliselt hinnata, sest sellise info kogumine on nii keelepsühholoogia kui ka keeletehnoloogia valdkonnas aina olulisemal kohal. Esmalt käsitleme konkreetsust ja abstraktsust kui mõisteid ning seda, kuidas neid varem uuritud on. Seejärel anname ülevaate uuringutest, mis on kogunud konkreetsushinnanguid teiste keelte sõnade kohta ning toome välja selliste hinnangute peamised kasutusalad. Kolmandaks anname ülevaate eestikeelsete sõnade konkreetsushinnanguid koguvast katsest ning sellega kaasnevatest probleemidest nii sisu kui ka vormi osas. *** Collecting concreteness ratings for Estonian words This paper analyzes introspection as a method of collecting large amounts of semantic data from human participants. Semantic indexes, such as lexical concreteness ratings, are becoming increasingly predominant in current psycholinguistic and language technology research. However, human introspection and categorized semantic properties also constitute fields over which researchers traditionally quarrel. Hence, an analysis of the issues introduced by this methodology is needed, with the aim of sparking discussion and introducing doubt into overly confident frameworks, hopefully even leading to more uniform and solid research designs where fewer classic mistakes are made. In this paper, we first discuss the concepts of concreteness and abstractness as well as the usefulness of such lexical indexes, followed by the description of a vast experiment collecting concreteness ratings from at least 2000 Estonian speakers. In the main part of the paper, however, we discuss a number of stimulus-based and organisatory issues, which this method either inherently entails or might introduce, as well as make suggestions for overcoming them.