SUBTLEX-SR: A subtitle-based frequency norm for Serbian

SUBTLEX-SR is a subtitle-based frequency norm for Serbian, the Serbian member of the SUBTLEX family of psycholinguistic frequency resources (following Brysbaert & New, 2009). It provides word-form and lemma frequencies, contextual diversity, and dispersion measures derived from the Serbian portion of OpenSubtitles v2018. **Contents.** The resource consists of two lexical tables: - **Wordform table** (2,198,809 entries): one row per surface form, with frequency from a 50-million-token lemmatized subsample, frequency from the full 64,842-film cleaned corpus, contextual diversity, three dispersion measures (Gries DP, DPnorm, Juilland D over decade buckets), and POS distribution.- **Lemma table** (330,535 entries): one row per lemma, with subsample-derived frequency, contextual diversity, and POS distribution. Both tables are provided in two scripts (Latin and Cyrillic) and in two formats (CSV and Apache Parquet). Methodology JSON files documenting all construction decisions, and a deterministic per-film manifest of the lemmatized subsample, are included for reproducibility. Four figures from the accompanying paper (baseline correlations, register divergence, Zipf distribution, CD vs. frequency) are also included. **Headline numbers.** 64,842 distinct films (the contextual-diversity base); 287.6 million alphabetic tokens in the cleaned corpus; 50.5 million classla-tokens in the lemmatized subsample; year coverage 1902–2020 with frequency data restricted to films from 1950 onwards. **Construction summary.** Source: OpenSubtitles v2018 raw Serbian (178,596 XML files). Cleaning included repair of a CP-1250-as-CP-1252 mojibake encoding error affecting 90.6% of source documents, two-phase deduplication consolidating 178,494 cleaned uploads into 64,842 distinct films, language-script normalization, and contamination filtering. Lemmatization performed with classla 2.2.1 (standard model variant) on a stratified subsample of 8,933 films. **Validation.** SUBTLEX-SR correlates strongly with OPUS's pre-computed Serbian frequency table (Pearson *r* = 0.97 on 498,489 forms; internal-consistency check) and moderately with the web-derived srLex baseline (*r* = 0.68 on 216,260 forms; *r* = 0.69 on 35,968 lemmas). The reduced srLex correlation reflects a register difference between subtitle dialogue and web prose; the divergent vocabulary sorts coherently into dialogue-characteristic classes (negated future-tense auxiliaries, vocatives, interjections) on one side and news/government-characteristic classes (country and region names, politicians' surnames, formal connectives) on the other. **Limitations.** No behavioral validation against lexical decision RT data has been performed for this release; this is being pursued through collaboration with the Laboratory for Experimental Psychology, University of Novi Sad. classla's Serbian model lemmatizes a small set of Serbian forms to their Croatian variants (e.g., *šta → što*, *koga → tko*); this affects all classla-sr users and is documented in the README. The corpus contains translation residue from non-Serbian source films (predominantly English-language Hollywood and BBC content) and some Bosnian/Croatian-orthography forms typical of the BCMS continuum. See the README and methodology JSON files for full documentation. **Companion paper.** Popović, M. (in preparation). *SUBTLEX-SR: A subtitle-based frequency norm for Serbian, with attention to register coverage in existing Serbian frequency resources.* Submitted to Language Resources and Evaluation. **License.** CC BY-SA 4.0. Source subtitle text is not redistributed with this deposit; see the README for source-corpus access via OPUS. **References.** - Brysbaert, M., & New, B. (2009). Moving beyond Kučera and Francis: A critical evaluation of current word frequency norms and the introduction of a new and improved word frequency measure for American English. *Behavior Research Methods*, 41(4), 977–990.- Lison, P., & Tiedemann, J. (2016). OpenSubtitles2016: Extracting large parallel corpora from movie and TV subtitles. *Proceedings of LREC 2016*, 923–929.- Ljubešić, N., & Dobrovoljc, K. (2019). What does neural bring? Analysing improvements in morphosyntactic annotation and lemmatisation of Slovenian, Croatian and Serbian. *Proceedings of BSNLP 2019*, 29–34.