Papers reviewed and determined not to be word norm studies. Use the flag icon to report errors or suggest re-inclusion.
18265 papers
Although some progress has been made on the quality of Machine Translation in recent years, there is still a significant potential for quality improvement. There has also been a shift in paradigm of machine translation, from “classical” rule-based systems like METAL or LMT1 towards example-based or statistical MT.2 It seems to be time now to evaluate the progress and compare the results of these efforts, and draw conclusions for further improvements of MT quality.
Abstract This paper deals with the more traditional sociolinguistic area of language attitudes. Based on data from the city of Århus, Denmark, I discuss the methods used in speaker evaluation experiments. The paper presents both a methodological discussion and the results of a study using the described techniques. The method is a mainly qualitative version of ‘the verbal guise technique’, and involves young people in Århus. Twelve speech samples (six female and six male voices) were used for this experiment, representing three different accents. The aim is to investigate the possibility of the Århus accent being a linguistic norm ideal among the young people in Århus. The results show that this does not seem to be the case, and that speakers are evaluated differently according to both accent and gender.
We present a method for automatic RMRS semantics construction from dependency structures, following the semantic algebra of Copestake et al. (2001). We have applied this method to a subset of the TIGER Dependency Bank for German (Forst et al., 2004) to obtain a semantic treebank for (HPSG) parser evaluation. We describe the semantics construction mechanism and give evaluation figures from manual validation of the treebank. These indicate high precision of the automatic RMRS construction process.
To ease the interpretation of higher order factor analysis, the direct relationships between variables and higher order factors may be calculated by the Schmid-Leiman solution (SLS; Schmid & Leiman, 1957). This simple transformation of higher order factor analysis orthogonalizes first-order and higher order factors and thereby allows the interpretation of the relative impact of factor levels on variables. The Schmid-Leiman solution may also be used to facilitate theorizing and scale development. The rationale for the procedure is presented, supplemented by syntax codes for SPSS and SAS, since the transformation is not part of most statistical programs. Syntax codes may also be downloaded from www.psychonomic.org/archive/.
Different words are usually assumed to be semantically independent in most existing similarity measures, which is not often true in practice. The semantic relatedness between words cannot be conveniently employed in the existing measures. We propose a novel similarity measure based on the earth mover's distance (EMD). In the proposed measure, the semantic distances between words are computed based on the electronic lexical database-WordNet and then the EMD is employed to calculate the document similarity with a many-to-many matching between words. Experiments and results demonstrate the effectiveness of the proposed similarity measure.
@conference{ai-giguet-2005-1, author = {Giguet, Emmanuel and Luquet, Pierre-Sylvain}, title = {Multilingual Lexical Database Generation from parallel texts with endogenous resources}, booktitle = {PAPILLON-2005 Workshop on Multilingual Lexical Databases}, year = {2005}, month = {December 12-14}, address = {Chiang Rai, Thaïland} }
Recent empirical experiments on surface realizers have shown that grammars for generation can be effectively evaluated using large corpora. Evaluation metrics are usually reported as single averages across all possible types of errors and syntactic forms. But the causes of these errors are diverse, and the extent to which the accuracy of generation over individual syntactic phenomena is unknown. This article explores the types of errors, both computational and linguistic, inherent in the evaluation of a surface realizer when using large corpora. We analyze data from an earlier wide coverage experiment on the FUF/SURGE surface realizer with the Penn TreeBank in order to empirically classify the sources of errors and describe their frequency and distribution. This both provides a baseline for future evaluations and allows designers of NLG applications needing off-the-shelf surface realizers to choose on a quantitative basis. 1
This paper presents a statistical approach to unknown word type prediction for a deep HPSG grammar. Our motivation is to enhance robustness in deep processing. With a predictor which predicts lexical types for unknown words according to the context, new lexical entries can be generated on the fly. The predictor is a maximum entropy based classifier trained on a HPSG treebank. By exploring various feature templates and the feedback from parse disambiguation results, the predictor achieves precision over 60%. The models are general enough to be applied to other constraint-based grammar formalisms. 1
Abstract. The present paper focuses on representation of morphological meanings on the underlying syntactic level. The concept of semantic counterparts of morphological meanings, the so-called grammatemes, was introduced in Functional Generative Description in the 1960’s. We suggest an elaborated system of these grammatemes, which have become a part of the tectogrammatical level of the Prague Dependency Treebank.
Three studies demonstrate the warm glow heuristic (Monin, 2003) without relying on aggregated ratings, and illustrate the important distinction between correlating average ratings versus averaging individual correlations. In Study 1, we re-analyze previous data correlating individual ratings with aggregates from another small sample of raters. In Study 2, we correlate individual familiarity ratings with normed attractiveness from a large sample of raters (n > 2,500). Study 3 bypasses the issue of aggregates altogether by having participants provide both attractiveness and familiarity ratings and computing correlations within participants. Despite this more conservative approach, the results of all three studies support the existence of the beautiful–is–familiar phenomenon.
A visual presentation procedure is introduced that presents target words followed by a dynamic mask until recognition. This form of stimulus degradation prolongs the word recognition process. Differences in word recognition latencies—which are usually quite small—are magnified, and thus can be more easily observed. The results of two experiments on the Internet with a total of 141 participants establish the task’s ability to magnify differences in word recognition latencies stemming from word familiarity (Experiment 1) and word prototypicality (Experiment 2). Both factors interact with stimulus degradation, but at different presentation intervals; these results are discussed as evidence for comparing models of word recognition. The new procedure can be used for assessing individual differences, such as implicit motives and self-focused attention. Further applications are discussed.
Agent technologies represent a promising approach for the integration of interorganizational capabilities across distributed, networked environments. However, knowledge sharing interoperability problems can arise when agents incorporating differing ontologies try to synchronize their internal information. Moreover, in practice, agents may not have a common or global consensus ontology that will facilitate knowledge sharing and integration of functional capabilities. We propose a method to enable agents to develop a local consensus ontology during operation time as needed. By identifying similarities in the ontologies of their peer agents, a set of agents can discover new concepts/relations and integrate them into a local consensus ontology on demand. We evaluate this method, both syntactically and semantically, when forming local consensus ontologies with and without the use of a lexical database. We also report on the effects when several factors, such as the similarity measure, the relation search level depth, and the merge order, are varied. Finally, experimenting in the domain of agent-supported Web service composition, we demonstrate how our method allows us to successfully autonomously form service-oriented local consensus ontologies.
The European Language Resources Association (ELRA) was founded in 1995 with the mission of providing language resources (LR) to European research institutions and companies. In this paper we describe the background, the mission and the major activities since then.
The paper presents details and comparison of two valuable language resources for Czech, two independent verb valency frames electronic dictionaries. The FIMU verb valency frames dictionary was designed during the EuroWordNet project and contains semantic roles and links to the Czech wordnet semantic network. The VALLEX 1.0 format is based on the formalism of the Functional Generative Description (FGD) and was developed during the Prague Dependency Treebank (PDT) project. We present the tools and approaches that were used within the process of adopting the FIMU Vallex format for the wordnet enriched valency frames. 1.
INTRODUCTORY REMARKS: HISTORICAL LINGUISTICS AND THE DATING OF HEBREW TEXTS CA. 1000–300 B.C.E.* Ziony Zevit University of Judaism In 1927, M. H. Segal’s A Grammar of Mishnaic Hebrew (Oxford University Press, 1927, reprinted in 1958 with corrections and addenda) helped launch a new sub-discipline in historical linguistics: The History of Hebrew. In order for him to establish that Mishnaic Hebrew was a well-defined linguistic stage in the history of Hebrew meriting a description on its own terms, it was necessary to demonstrate the ways in which it was unlike Biblical Hebrew. He produced impressive lists of data illustrating that the differences between Biblical Hebrew and Mishnaic Hebrew extended to style of expression, vocabulary, and grammar, that is, phonology, morphology, and syntax. His lists illustrated that of the 1350 verbs in the Biblical Hebrew lexicon, Mishnaic Hebrew lost 250 verbs while gaining about 300 new ones. Through analysis of its lexicon, Segal showed how Aramaic semantic calques on Hebrew changed the meanings of Biblical Hebrew words that continued into Mishnaic Hebrew or how Biblical Hebrew words were replaced by Aramaic words or how new Hebrew words replaced old Hebrew words. Segal’s research indicated beyond doubt that Mishnaic Hebrew was not a debased or slightly evolved form of Biblical Hebrew. The repertoire of its linguistic norms was not described in grammars of Biblical Hebrew while its lexical resources were larger and more diverse than those of Biblical Hebrew. From an historical perspective it had to be studied on its own because it was geographically discontinuous with most of Biblical Hebrew, because the linguistic environment in which it was spoken differed significantly from that of Biblical Hebrew, and because it was a few centuries younger than Biblical Hebrew but not necessarily its direct stemmatic continuation. Historical linguistics begins by noting that living languages change. Their phonology changes as do their morphology and syntax and vocabulary when new words are introduced and old ones drop out of use or when the semantic load of individual vocables shift. Linguists have observed, on the basis of two centuries of research into many languages, that change occurs more easily and hence rapidly—when and if it occurs—in phonology and lexicon than in *!These introductory remarks were delivered November 22, 2004 before presentations by a panel of scholars dealing with the question of whether or not biblical texts can be dated linguistically. Hebrew Studies 46 (2005) 322 Zevit: Introductory Remarks morphology and syntax. But change occurs, exactly the type of changes that Segal described in 1927. Since the 1920s, work on delimiting the characteristic features of Hebrew in many of its historical periods has continued unabated, primarily at institutions in Israel, but also in some located in Europe and North America. Nowadays, scholars talk about Modern Israeli Hebrew, Haskalah Hebrew, Medieval Hebrew, Mishnaic/Tannaitic Hebrew, and, of course, Biblical Hebrew. Researchers in Israel work on all periods of Hebrew, from Biblical Hebrew through the contemporary language, whereas those outside of Israel work primarily on Hebrew from both the First and Second Temple periods, including some Mishnaic Hebrew, but more often on the Hebrew of the Dead Sea Scrolls, an ill-defined type that fits chronologically somewhere between Biblical Hebrew and Mishnaic Hebrew. A bibliographically rich summary of the achievements and the state of research in Mishnaic Hebrew is available in Moshe Bar Asher, “Mishnaic Hebrew: An Introduction,” HS 40 (1999): 115– 151. Projects aimed at refining notions about Hebrew of the First Temple period were stimulated not only by the comparative data supplied by the Ugaritic after the 1930s, but also by research into Aramaic dialects from early antiquity through the modern period, and by work on Akkadian in general and the Amarna dialects in particular. In addition, such projects benefited directly from advances in semantics and dialect studies, by studies of the living linguistic and textual traditions in diasporic Jewish communities, and by the study and analysis of newly discovered Hebrew and Aramaic inscriptions. The inscriptions proved to be of major importance because they supplied archaeologically dated, uncurated texts for linguistic analysis. Many scholars contributed to the advance of knowledge in this area and I name a few whose...
Abstract In order to demonstrate elevated disgust sensitivity and facilitated disgust learning in patients suffering from blood injection injury phobia, 23 phobics and 20 controls underwent an evaluative conditioning experiment. They were presented with picture pairs consisting of affectively neutral pictures (CS), which were followed by either disgust-inducing, fear-inducing, pleasant, or neutral scenes (US). During the presentation we recorded the electromyogram (EMG) of the musculus levator labii as a specific disgust indicator. Affective ratings for the pictures were determined before and after conditioning. Also, CS-US contingency verbalisation (CV) was assessed. Phobics reported a greater overall disgust sensitivity, experienced stronger feelings of disgust, and showed greater EMG responses while viewing disgust-eliciting scenes than control subjects. Evaluative conditioning occurred equally in both groups and depended on CV.
The gradient descent optimization method has been a de facto standard learning algorithm in computational models of category learning. However, it can be considered as a normative (vs. descriptive) model of human learning processes. In particular, there are three concerns associated with the learning algorithm& #x2014;namely, complexity, regularity, and context independency. In response to these limitations, the present study introduces an alternative, hypothesis-testing& #x2014;like learning algorithm on the basis of a stochastic optimization method. The new learning model, termed SCODEL, provides qualitatively simple interpretations for its implied category-learning processes. Moreover, SCODEL is the first modeling attempt to depict individually unique and context-dependent learning processes. Four simulation studies were conducted and showed that the present model has the competence to operate as several different types of learners in various plausibly real-life situations.
We describe briefly the redevelopment of Space Fortress (SF), a research tool widely used to study training of complex tasks involving both cognitive and motor skills, to be executed on currentgeneration systems with significantly extended capabilities, and then compare the performance of human participants on an original PC version of Space Fortress (SF) with the revised Space Fortress (RSF). Participants trained on SF or RSF for 10 sets of eight 3-min practice trials and two 3-min test trials. They then took tests involving retention, resistance to secondary task interference, and transfer to a different control system. They then switched from SF to RSF or from RSF to SF for 2 sets of final tests and completed rating scales comparing RSF and SF. Slight differences were predicted on the basis of a scoring error in the original version of SF used and on slightly more precise joystick control in RSF. The predictions were supported. The SF group started better but did worse when they transferred to RSF. Despite the disadvantage of having to be cautious in generalizing from RSF to SF, we conclude that RSF has many advantages, which include accommodating new PC hardware and new training techniques. A monograph that presents the methodology used in creating RSF, details on its performance and validation, and directions on how to download free copies of the system may be downloaded from www .psychonomic.org/archive/.
We examine methods for measuring performance in signal-detection-like tasks when each participant provides only a few observations. Monte Carlo simulations demonstrate that standard statistical techniques applied to ad’ analysis can lead to large numbers of Type I errors (incorrectly rejecting a hypothesis of no difference). Various statistical methods were compared in terms of their Type I and Type II error (incorrectly accepting a hypothesis of no difference) rates. Our conclusions are the same whether these two types of errors are weighted equally or Type I errors are weighted more heavily. The most promising method is to combine an aggregated’ measure with a percentile bootstrap confidence interval, a computerintensive nonparametric method of statistical inference. Researchers who prefer statistical techniques more commonly used in psychology, such as a repeated measurest test, should useγ (Goodman & Kruskal, 1954), since it performs slightly better than or nearly as well asd’. In general, when repeated measurest tests are used,γ is more conservative thand’: It makes more Type II errors, but its Type I error rate tends to be much closer to that of the traditional .05 α level. It is somewhat surprising thatγ performs as well as it does, given that the simulations that generated the hypothetical data conformed completely to thed’ model. Analyses in which H—FA was used had the highest Type I error rates. Detailed simulation results can be downloaded fromwww.psychonomic.org/archive/Schooler-BRM-2004.zip.
This study compared four common methods for scoring a popular working memory span task, Daneman and Carpenter’s (1980) reading span test. More continuous measures, such as the total number of words recalled or the proportion of words per set averaged across all sets, were more normally distributed, had higher reliability, and had higher correlations with criterion measures (reading comprehension and Verbal SAT) than did traditional span scores that quantified the highest set size completed or the number of words in correct sets. Furthermore, creation of arbitrary groups (e.g., high-span and low-span groups) led to poor reliability and greatly reduced predictive power. It is recommended that researchers score span tasks with continuous measures and avoid post hoc dichotomization of working memory span groups.
Contrasting linguistic and nonlinguistic processing has been of interest to many researchers with different scientific, theoretical, or clinical questions. However, previous work on this type of comparative analysis and experimentation has been limited. In particular, little is known about the differences and similarities between the perceptual, cognitive, and neural processing of nonverbal environmental sounds and that of speech sounds. With the aim of contrasting verbal and nonverbal processing in the auditory modality, we developed a new on-line measure that can be administered to subjects from different clinical, neurological, or sociocultural groups. This is an on-line task of sound to picture matching, in which the sounds are either environmental sounds or their linguistic equivalents and which is controlled for potential task and item confounds across the two sound types. Here, we describe the design and development of our measure and report norming data for healthy subjects from two different adult age groups: younger adults (18–24 years of age) and older adults (54–78 years of age). We also outline other populations to which the test has been or is being administered. In addition to the results reported here, the test can be useful to other researchers who are interested in systematically contrasting verbal and nonverbal auditory processing in other populations.
There is a strong relationship between evaluation and methods for automatically training language processing systems, where generally the same resource and metrics are used both to train system components and to evaluate them. To date, in dialogue systems research, this general methodology is not typically applied to the dialogue manager and spoken language generator. However, any metric for evaluating system performance can be used as a feedback function for automatically training the system. This approach is motivated with examples of the application of reinforcement learning to dialogue manager optimization, and the use of boosting to train the spoken language generator.
Ontologies are recognised as important tools, not only for effective and efficient information sharing, but also for information extraction and text mining. In the biomedical domain, the need for a common ontology for information sharing has long been recognised, and several ontologies are now widely used. However, there is confusion among researchers concerning the type of ontology that is needed for text mining , and how it can be used for effective knowledge management, sharing, and integration in biomedicine. We argue that there are several different ways to define an ontology and that, while the logical view is popular for some applications, it may be neither possible nor necessary for text mining. We propose a text-centered approach for knowledge sharing, as an alternative to formal ontologies. We argue that a thesaurus (i.e. an organised collection of terms enriched with relations) is more useful for text mining applications than formal ontologies.
Reviewed by: Word sense disambiguation: The case for combinations of knowledge sources by Mark Stevenson Cornelia Tschichold Word sense disambiguation: The case for combinations of knowledge sources. By Mark Stevenson. (CSLI studies in computational linguistics.) Stanford: CSLI Publications, 2003. Pp. 175. ISBN 1575863901. $25. Disambiguating words is easy for human beings, but difficult for computers. Computational linguistics has developed methods to reliably find the correct part of speech for the large majority of words in running text, but the disambiguation of polysemous words and homonyms (bat as animal, sports tool, or blink of the eye) is a more complex process. This difference is due mainly to the lack of sufficiently complete and formalized data about word senses. Stevenson shows how progress can be achieved by reusing existing lexical databases and combining them in an optimal way. Ch. 1 introduces the problem of polysemy and points out the potential areas of application for word sense disambiguation (WSD). Ch. 2 gives some historical background on the area, intended for readers unfamiliar with the field. In Ch. 3, lexicographic problems associated with polysemous words and attempts at arriving at suitable databases (such as Word-Net) are discussed. As it does not seem likely that machines can take over any significant part of the lexicographic work involved in the production of semantic databases, the re-use of machine-readable dictionaries appears to be the only viable solution for the immediate future. S refutes a number of criticisms that have been made against the use of a machine-readable dictionary for WSD, mainly due to their lack of alternatives, and proposes methods for at least partially remedying the known shortcomings. Ch. 4 describes the knowledge sources that can be used for WSD, that is, syntactic, semantic, and pragmatic information, and the conditions needed to combine them. WordNet and the Longman dictionary of contemporary English (LDOCE) are identified as two potentially useful on-line lexicographic databases. In Ch. 5, the computational similarities and differences of part-of-speech tagging and WSD are explained. In Ch. 6, S explains how his system combining the various knowledge sources was implemented: the preprocessing stage filters out proper names, tokenizes the input text, and identifies the part of speech for each word (using a Brill-type tagger). This is followed by a shallow syntactic analysis and finally the lexical look-up stage. At the disambiguation stage, the part-of-speech tags are used to filter out any (syntactically) incompatible senses, before a number of partial (semantic) taggers are brought into play. The first of these uses LDOCE senses, with any subsenses grouped where possible; the second uses categories of synonyms, and the last selectional restrictions. Known collocations are also taken into account. The implementation involved a memory-based machine learning system that was first trained on annotated data and then used to combine all the knowledge sources for WSD. Chs. 7 and 8 deal with evaluation of the author’s and other known systems for WSD. S illustrates the unsatisfactory state of evaluation tools and procedures in the area of WSD, before demonstrating that his system achieves better results thanks to the combination of a number of available lexical resources. [End Page 1022] The book is a readable introduction and description of the problems WSD poses for computational linguistics, making a clear case for a hybrid approach that uses knowledge-based and corpus-based sources of information to identify the sense of ambiguous words. Cornelia Tschichold University of Wales Swansea, Great Britain Copyright © 2005 Linguistic Society of America
The paper deals with the preliminary findings from the morphologically annotated corpus of Lithuanian language (1 million running words). It was compiled and processed at the Center of Computational Linguistics, Vytautas Magnus University. Each annotation for an inflected word form of the corpus contains a lemma and a set of morphological features. The paper presents the strategy for automatic and manual annotation. Automatic annotation was carried out with the help of analyser-lemmatiser. Disambiguation of the homoforms was performed manually. Tag sets and the most prominent features of Lithuanian morphology are discussed in detail. The annotated corpus allowed us to measure the usage of parts of speech and their morphological features in contemporary Lithuanian language. The annotated corpus is of great importance for future development of parsing tools, treebanks and other NLP tools and resources for Lithuanian language.
The aim of this paper is to investigate a case of transfer within the context of language death. By examining data from Jersey Norman French (known to its speakers as Jèrriais) it illustrates the difficulty in determining linguistic norms for this relatively undocumented variety and suggests possible strategies to overcome this problem. The study compares systematically the occurrence of overt and covert transfer in the speech of a sample of fifty native speakers of Jèrriais via the analysis of a number of linguistic variables. The extent to which transfer-induced changes are themselves becoming established as norms-within this speech community will also be considered. (PsycINFO Database Record (c) 2016 APA, all rights reserved)
It is important that a book on adolescence should contain a chapter on slang and swearing because these are two features of adolescent linguistic behavior that attract more attention than they perhaps warrant, symbolizing as they do, the freedom that young people have at this stage of their lives to challenge linguistic norms, and at the same time to test interpersonal bonds and institutional constraints with their parents as they seek to establish new identities and relationships within their changing worlds. This chapter will start by providing a brief background description of the linguistic characteristics of slang and swearing before moving to a more focused discussion of why (although such language is also commonly used by many adult groups who find themselves living in close proximity in institutionalized contexts such as prisons or military camps), it may be particularly typical of adolescent speech. Subsequent sections of the chapter explore how the use of slang and expletives by young people can be viewed as a means of building cultural capital within their networks, while at the same time establishing boundaries between new adolescent in-groups ('us') and parents and nonmembers ('them'). A section on the use of pejorative terms further develops this theme of 'us' and 'them.' In addition, I also consider gender-based patterns of the ways young people use slang and expletives, and the subtle coercive effect such words have in regulating typical patterns of in-group behavior. Finally some suggestions are made regarding research questions that still require attention. (PsycINFO Database Record (c) 2016 APA, all rights reserved)
We have constructed a large scale and detailed database of lexical types in Japanese from a treebank that includes detailed linguistic information. The database helps treebank annotators and grammar developers to share precise knowledge about the grammatical status of words that constitute the treebank, allowing for consistent large scale treebanking and grammar development. In this paper, we report on the motivation and methodology of the database construction. 1
In order to realize the full potential of dependency-based syntactic parsing, it is desirable to allow non-projective dependency structures. We show how a data-driven deterministic dependency parser, in itself restricted to projective structures, can be combined with graph transformation techniques to produce non-projective structures. Experiments using data from the Prague Dependency Treebank show that the combined system can handle non-projective constructions with a precision sufficient to yield a significant improvement in overall parsing accuracy. This leads to the best reported performance for robust non-projective parsing of Czech.
TextTrees, introduced in (Newman, 2005), are skeletal representations formed by systematically converting parser output trees into unlabeled indented strings with minimal bracketing. Files of TextTrees can be read rapidly to evaluate the results of parsing long documents, and are easily edited to allow limited-cost treebank development. This paper reviews the TextTree concept, and then describes the implementation of the almost parser- and grammar-independent TextTree generator, as well as auxiliary methods for producing parser review files and inputs to bracket scoring tools. The results of some limited experiments in TextTree usage are also provided.
Natural languages encode gender distinctions in various ways. We investigate the differences between English and Hebrew in this respect, our departure point being the relations that are defined between the feminine and the masculine realizations of nouns in the English WordNet. We define a number of distinct classes of English nouns which differ in the way they realize gender distinctions. We then define similar classes of Hebrew nouns and show how to map the Hebrew nouns (and relations defined over them) to the English structure. This establishes a systematic assignment of Hebrew nouns to WordNet synsets, which is consistent with the ideas underlying multilingual extensions of WordNet. The main result is a consistent Hebrew WordNet which is aligned with the English one, but an additional contribution is a set of desiderata for the correct encoding of (systematic) semantic differences among languages. 1
The Proposition Bank project takes a practical approach to semantic representation, adding a layer of predicate-argument information, or semantic role labels, to the syntactic structures of the Penn Treebank. The resulting resource can be thought of as shallow, in that it does not represent coreference, quantification, and many other higher-order phenomena, but also broad, in that it covers every instance of every verb in the corpus and allows representative statistics to be calculated. We discuss the criteria used to define the sets of semantic roles used in the annotation process and to analyze the frequency of syntactic/semantic alternations in the corpus. We describe an automatic system for semantic role tagging trained on the corpus and discuss the effect on its performance of various types of information, including a comparison of full syntactic parsing with a flat representation and the contribution of the empty “trace” categories of the treebank.
To empower the general mass through access to information and knowledge, organized efforts are being made to develop relevant content in local languages and provide local language capabilities to utility software. We have developed a Question Answering (QA) System for Hindi documents that would be relevant for masses using Hindi as primary language of education. The user should be able to access information from E-learning documents in a user friendly way, that is by questioning the system in their native language Hindi and the system will return the intended answer (also in Hindi) by searching in context from the repository of Hindi documents. The language constructs, query structure, common words, etc. are completely different in Hindi as compared to English. A novel strategy, in addition to conventional search and NLP techniques, was used to construct the Hindi QA system. The focus is on context based retrieval of information. For this purpose we implemented a Hindi search engine that works on locality-based similarity heuristics to retrieve relevant passages from the collection. It also incorporates language analysis modules like stemmer and morphological analyzer as well as self constructed lexical database of synonyms. The experimental results over corpus of two important domains of agriculture and science show effectiveness of our approach.