The wacky wide web a collection of very large linguistically processed web crawled corpora

Publication

AuthorsMarco Baroni OASilvia Bernardini OAAdriano Ferraresi OAEros ZanchettaOA

Year2009

JournalLanguage Resources and Evaluation

Abstract

This article introduces ukWaC, deWaC and itWaC, three very large corpora of English, German, and Italian built by web crawling, and describes the methodology and tools used in their construction. The corpora contain more than a billion words each, and are thus among the largest resources for the respective languages. The paper also provides an evaluation of their suitability for linguistic research, focusing on ukWaC and itWaC. A comparison in terms of lexical coverage with existing resources for the languages of interest produces encouraging results. Qualitative evaluation of ukWaC versus the British National Corpus was also conducted, so as to highlight differences in corpus composition (text types and subject matters). The article concludes with practical information about format and availability of corpora and tools.

LanguageEnglish, German, Italian

Building and evaluating web corpora representing national varieties of english2017 Crowdsourcing for web genre annotation2016 Corpus based vocabulary lists for language learners for nine languages2014 Twitter n gram corpus with demographic metadata2013 Psycholinguistic measures for german verb pairs semantic transparency semantic relatedness verb family size and age of reading acquisition2018 Association norms for german noun compounds and their constituents2015 Blind a set of semantic feature norms from the congenitally blind2013 Extracting semantic representations from word co occurrence statistics stop lists stemming and svd2012 Specificity ratings for English data2024

Language Learning with Technology2021

Computational linguistics · Cambridge University Press eBooks2015

Building and Exploring Web Corpora. Proceedings of the 3rd web as corpus workshop, incorporating cleaneval · Ghent University Academic Bibliography (Ghent University)2007

Corpus Linguistics and the Web2007

Building general- and special-purpose corpora by Web crawling · Archivio istituzionale della ricerca (Alma Mater Studiorum Università di Bologna)2006

A Figure of Merit for the Evaluation of Web-Corpus Randomness2006

CUCWeb2006

Web-based frequency dictionaries for medium density languages2006

Large linguistically-processed web corpora for multiple languages2006

Wacky! Working papers on the Web as Corpus · Institutional Research Information System (Università degli Studi di Trento)2006

Proceedings of the 2nd International Workshop on Web as Corpus - WAC '062006

Web Text Corpus for Natural Language Processing2006

Creating and using Web corpora · International Journal of Corpus Linguistics2005

Developing Linguistic Corpora: a Guide to Good Practice2005

The TREC terabyte retrieval track · ACM SIGIR Forum2005

Evaluation of Japanese Web-based Reference Corpora: Effects of Seed Selection and Time Interval2005

Making the Web More Useful as a Source for Linguistic Corpora · Applied Corpus Linguistics2004

Introduction to the Special Issue on the Web as Corpus · Computational Linguistics2003

The impact of corpus size on question answering performance2002

Genres, Registers, Text Types, Domains and Styles: Clarifying the Concepts and Navigating a Path through the BNC Jungle2002

Word Frequency Distributions · Text, speech and language technology2001

Corpus Linguistics in North America2001

Syntactic clustering of the Web · Computer Networks and ISDN Systems1997

The Search for Units of Meaning1996

Accurate methods for the statistics of surprise and coincidence1993

The wacky wide web a collection of very large linguistically processed web crawled corpora

Publication

Abstract

Extracted information

Edits / History

Cited by

References