Papers reviewed and determined not to be word norm studies. Use the flag icon to report errors or suggest re-inclusion.
18265 papers
<h3>Introduction</h3><br> BOLT Egyptian Arabic Treebank -- Discussion Forum was developed by the Linguistic Data Consortium (LDC) and consists of Egyptian Arabic web discussion forum data with part-of-speech annotation, morphology, gloss and syntactic tree annotation. <br> The DARPA <a href="https://www.ldc.upenn.edu/collaborations/current-projects/bolt">BOLT</a> (Broad Operational Language Translation) program developed machine translation and information retrieval for less formal genres, focusing particularly on user-generated content. LDC supported the BOLT program by collecting informal data sources -- discussion forums, text messaging and chat -- in Chinese, Egyptian Arabic and English. The collected data was translated and annotated for various tasks including word alignment, treebanking, propbanking and co-reference. <br> The unannotated Egyptian Arabic source data is released as BOLT Arabic Discussion Forums (<a href="../../../LDC2018T10">LDC2018T10</a>). <br> The annotations in this release follow Penn Arabic Treebank (PATB) annotation guidelines. The PATB project consists of two distinct phases: (a) part-of-speech tagging which divides the text into lexical tokens and gives relevant information about each token such as lexical category, inflectional features and a gloss; and (b) Arabic treebanking, which characterizes the constituent structures of word sequences, provides categories for each non-terminal node and identifies null elements, co-reference, traces and so on. <br> There are two kinds of morphological analysis synchronized in the corpus. LDC Standard Morphological Analyzer (SAMA) Version 3.1 (<a href="../../../LDC2010L01">LDC2010L01</a>) was used for Modern Standard Arabic tokens, and CALIMA (Columbia Arabic Language and dIalect Morphological Analyzer) was used for Egyptian-Arabic tokens. <br> <h3>Data</h3><br> This release contains 440,448 tokens before clitics were split and 508,548 tree tokens after clitics were split for treebank annotation. The source material is web discussion forums collected by LDC from various sources. <br> Data is presented in a a variety of UTF-8 encoded text formats, specifically plain text, XML, tdf and Penn Treebank. See the included documentation for more information about the specific formats. <br> <h3>Acknowledgement</h3><br> This material is based upon work supported by the Defense Advanced Research Projects Agency (DARPA) under Contract No. HR0011-11-C-0145. The content does not necessarily reflect the position or the policy of the Government, and no official endorsement should be inferred. <br> <h3>Samples</h3><br> Please view the following samples: <br> <ul><br> <li><a href="desc/addenda/LDC2018T23-int.txt">Integrated</a></li><br> <li><a href="desc/addenda/LDC2018T23.tree">Penn Treebank</a></li><br> <li><a href="desc/addenda/LDC2018T23-pos.txt">POS</a></li><br> <li><a href="desc/addenda/LDC2018T23-su_xml.xml">SU Annotation</a></li><br> <li><a href="desc/addenda/LDC2018T23.tdf">SU tdf</a></li><br> <li><a href="desc/addenda/LDC2018T23.xml">Annotation Graph</a></li><br> </ul><br> <h3>Updates</h3><br> None at this time. </br> Portions © 2011-2018 Trustees of the University of Pennsylvania
This chapter gives an overview of the conception(s) of linguistic normativity in Cognitive Linguistics (CL) and the contributions of CL to the study of normativity on the level of language use (discourse). While normativity has never been a central concern of CL, it is most relevant - though in quite different ways - for both cognitive semantics and cognitive grammar models. The concept of sign of those grammar models is based on the assumption that the linguistic units which constitute a language do not only exist as entrenched units in speakers' minds but also as conventional units by virtue of being shared in a speech community. Cognitive semantics, on the other hand, is most relevant in the field of cognitive language criticism (Sprachkritik) and its application to the analysis of public discourse ("political correctness", "gender-neutral language").
Хухуни Е.А.Комарова Московский государственный областной университет НОРМА, СТАНДАРТ, УЗУС В век информационных технологий и глобализации процесс изменения языка ускоряется, что не может не влиять на узус и норму.Такая тенденция вызывает неоднозначную реакцию в русскоговорящем обществе, зачастую выражающуюся в
A recent advance in monolingual dependency parsing is the idea of a treebank embedding vector, which allows all treebanks for a particular language to be used as training data while at the same time allowing the model to prefer training data from one treebank over others and to select the preferred treebank at test time. We build on this idea by 1) introducing a method to predict a treebank vector for sentences that do not come from a treebank used in training, and 2) exploring what happens when we move away from predefined treebank embedding vectors during test time and instead devise tailored interpolations. We show that 1) there are interpolated vectors that are superior to the predefined ones, and 2) treebank vectors can be predicted with sufficient accuracy, for nine out of ten test languages, to match the performance of an oracle approach that knows the most suitable predefined treebank embedding for the test set.
Abstract In this paper, we present a novel lemmatization method based on a sequence-to-sequence neural network architecture and morphosyntactic context representation. In the proposed method, our context-sensitive lemmatizer generates the lemma one character at a time based on the surface form characters and its morphosyntactic features obtained from a morphological tagger. We argue that a sliding window context representation suffers from sparseness, while in majority of cases the morphosyntactic features of a word bring enough information to resolve lemma ambiguities while keeping the context representation dense and more practical for machine learning systems. Additionally, we study two different data augmentation methods utilizing autoencoder training and morphological transducers especially beneficial for low-resource languages. We evaluate our lemmatizer on 52 different languages and 76 different treebanks, showing that our system outperforms all latest baseline systems. Compared to the best overall baseline, UDPipe Future, our system outperforms it on 62 out of 76 treebanks reducing errors on average by 19% relative. The lemmatizer together with all trained models is made available as a part of the Turku-neural-parsing-pipeline under the Apache 2.0 license.
Text structuring is a fundamental step in NLG, especially when generating multi-sentential text. With the goal of fostering more general and data-driven approaches to text structuring, we propose the new and domain-independent NLG task of structuring and ordering a (possibly large) set of EDUs. We then present a solution for this task that combines neural dependency tree induction with pointer networks and can be trained on large discourse treebanks that have only recently become available. Further, we propose a new evaluation metric that is arguably more suitable for our new task compared to existing content ordering metrics. Finally, we empirically show that our approach outperforms competitive alternatives on the proposed measure and is equivalent in performance with respect to previously established measures.
Pretrained multilingual contextual representations have shown great success, but due to the limits of their pretraining data, their benefits do not apply equally to all language varieties. This presents a challenge for language varieties unfamiliar to these models, whose labeled and unlabeled data is too limited to train a monolingual model effectively. We propose the use of additional language-specific pretraining and vocabulary augmentation to adapt multilingual models to low-resource settings. Using dependency parsing of four diverse low-resource language varieties as a case study, we show that these methods significantly improve performance over baselines, especially in the lowestresource cases, and demonstrate the importance of the relationship between such models' pretraining data and target language varieties.
We introduce a new symmetric measure (called pos ) that utilises the non-symmetric KL cpos 3 measure We can set a threshold for this new measure so that a pair of treebanks can be considered harmonious in their annotation if pos does not surpass the threshold. For the calculation of the threshold, we estimate the effects of (i) the size variation, and (ii) the genre variation in the considered pair of treebanks. The estimations are based on data from treebanks of distinct language families, making the threshold less dependent on the properties of individual languages. We demonstrate the utility of the proposed measure by listing the treebanks in Universal Dependencies version 2.5 (UDv2.5) (Zeman et al., 2019) data that are annotated consistently with other treebanks of the same language. However, the measure could be used to assess inter-treebank annotation consistency under other (non-UD) annotation guidelines as well.
Tae Hwan Oh, Ji Yoon Han, Hyonsu Choe, Seokwon Park, Han He, Jinho D. Choi, Na-Rae Han, Jena D. Hwang, Hansaem Kim. Proceedings of the 16th International Conference on Parsing Technologies and the IWPT 2020 Shared Task on Parsing into Enhanced Universal Dependencies. 2020.
This paper reports on the analysis and annotation of Multiword Expressions in the Irish Universal Dependency Treebank. We provide a linguistic discussion around decisions on how to appropri- ately label Irish MWEs using the compound, flat and fixed dependency relation labels within the framework of the Universal Dependencies annotation guidelines. We discuss some nuances of the Irish language that pose challenges for assigning these UD labels and provide this report in support of the Irish UD annotation guidelines. With this we hope to ensure consistency in annotation across the dataset and provide a basis for future MWE annotation for Irish.
Historical linguistics, whether synchronic or diachronic, is by definition based on corpora.Since we do not have access to the intuitions of native speakers we can only test linguistic hypotheses about historical languages by systematically collating information from our corpus of texts.For questions that typically concern linguists, this often means identifying every occurrence of a particular phenomenon in the corpus, analysing, classifying and counting the occurrences and then using this for testing hypotheses about the structure of the language.This can be done manually, but this is time-consuming and error-prone.As Haug (2015) points out, while reading the text and manually collating information from it is essential for hypothesis formation it is much less useful for hypothesis testing.Even if the text is in electronic form, it is easy to overlook an example, record it incorrectly or fail to apply test criteria consistently over time.This paper focuses on treebanks, which are corpora that have been annotated with morphosyntactic information so that we can extract linguistic structures like 'verb with an accusative noun'.High-quality treebanks for a range of historical languages now exist and are widely used in historical linguistic research.This includes treebanks that follow the Penn-style of annotation, e.g. the Penn-Helsinki
The present article presents some challenges posed by lemmatization and PoS tagging of Latin, with reference to the ongoing work to revise the Latin Dependency Treebank. Current options available for lemmatization and morphological analysis of Latin are reviewed and discussed. The pipeline to annotate the morphological layer of the Latin Dependency Treebank is shown to consist of three main steps: (i) tokenization/sentence split, which is performed via a documented rule-based algorithm, (ii) pre-population by means of COMBO, a state-of-the-art joint lemmatizer, PoS tagger, and parser trained on the data of the Latin Dependency Treebank 2.1, and (iii) manual error correction informed by the attempt to identify and document lemmatization and morphology annotation rules.
We present scalable Universal Dependency (UD) treebank synthesis techniques that exploit advances in language representation modeling which leverage vast amounts of unlabeled generalpurpose multilingual text. We introduce a data augmentation technique that uses synthetic treebanks to improve production-grade parsers. The synthetic treebanks are generated using a state-of-the-art biaffine parser adapted with pretrained Transformer models, such as Multilingual BERT (M-BERT). The new parser improves LAS by up to two points on seven languages. The production models' LAS performance improves as the augmented treebanks scale in size, surpassing performance of production models trained on originally annotated UD treebanks.
Most syntactic dependency parsing models may fall into one of two categories: transition- and graph-based models. The former models enjoy high inference efficiency with linear time complexity, but they rely on the stacking or re-ranking of partially-built parse trees to build a complete parse tree and are stuck with slower training for the necessity of dynamic oracle training. The latter, graph-based models, may boast better performance but are unfortunately marred by polynomial time inference. In this paper, we propose a novel parsing order objective, resulting in a novel dependency parsing model capable of both global (in sentence scope) feature extraction as in graph models and linear time inference as in transitional models. The proposed global greedy parser only uses two arc-building actions, left and right arcs, for projective parsing. When equipped with two extra non-projective arc-building actions, the proposed parser may also smoothly support non-projective parsing. Using multiple benchmark treebanks, including the Penn Treebank (PTB), the CoNLL-X treebanks, and the Universal Dependency Treebanks, we evaluate our parser and demonstrate that the proposed novel parser achieves good performance with faster training and decoding.
We show how we can adapt parsing to low-resource domains by combining treebanks across languages for a parser model with treebank embeddings. We demonstrate how we can take advantage of in-domain treebanks from other languages, and show that this is especially useful when only out-of-domain treebanks are available for the target language. The method is also extended to low-resource languages by using out-of-domain treebanks from related languages. Two parameter-free methods for applying treebank embeddings at test time are proposed, which give competitive results to tuned methods when applied to Twitter data and transcribed speech. This gives us a method for selecting treebanks and training a parser targeted at any combination of domain and language.
Savoring is an emotion regulation technique that aims to increase, sustain, and deepen positive emotion. It has been incorporated into several novel, "positive affect" interventions for anxiety, depression, and chronic pain, but has not been studied in a laboratory setting. As such, it is unknown whether savoring can modulate subjective and neural correlates of emotion-processing and whether savoring might exert a persistent effect on stimulus processing (i.e., modulating response at subsequent encounter). Here, 49 participants savored or viewed positive and neutral pictures, before seeing the same pictures again approximately 20 min later without instructions to savor (or view) pictures. Subjective valence and arousal ratings and the picture-elicited late positive potential (LPP) were assessed during both tasks. Results showed that savoring increased participant ratings of picture pleasantness and arousal as well as a picture-elicited LPP. Moreover, pictures that had previously been savored continued to elicit higher ratings during the subsequent picture viewing task. A larger LPP was observed for previously savored positive and neutral pictures during an early portion of picture viewing; later on during picture viewing, this effect was limited to positive pictures only (i.e., it was not evident for neutral pictures). Results validate savoring as an effective and durable means of increasing positive emotion and are discussed in the context of a broader emotion regulation literature, which has primarily examined the downregulation of negative picture processing.
We tackle implicit discourse relation recognition. Both self-attention and interactive-attention mechanisms have been applied for attention-aware representation learning, which improves the current discourse analysis models. To take advantages of the two attention mechanisms simultaneously, we develop a propagative attention learning model using a cross-coupled two-channel network. We experiment on Penn Discourse Treebank. The test results demonstrate that our model yields substantial improvements over the baselines (BiLSTM and BERT).
This paper presents theoretical and methodological questions related to the creation of a Linguistic Database, made up of samples from the Cazumbá Iracema Extractive Reserve, located in the state of Acre, and discusses the main challenges found and contributions to the teaching and learning process of Portuguese. The methodology for collecting and organizing this database is based on the theoretical assumptions of sociolinguistic patterns, the empirical foundations of the Theory of Linguistic Variation and Change, and the methodology for collecting and manipulating data in sociolinguistics. The implementation of the proposal involves the use of software that can be used in education. The results show contributions of this sample use for the creation of teaching proposals, focusing on the language in use, identification of the sociocultural factors that influence the emergence and permanence of linguistic variation and researches in the scope of natural languages.
The large communication cost for exchanging gradients between different nodes significantly limits the scalability of distributed training for large-scale learning models. Motivated by this observation, there has been significant recent interest in techniques that reduce the communication cost of distributed Stochastic Gradient Descent (SGD), with gradient sparsification techniques such as top-k and random-k shown to be particularly effective. The same observation has also motivated a separate line of work in distributed statistical estimation theory focusing on the impact of communication constraints on the estimation efficiency of different statistical models. The primary goal of this paper is to connect these two research lines and demonstrate how statistical estimation models and their analysis can lead to new insights in the design of communication-efficient training techniques. We propose a simple statistical estimation model for the stochastic gradients which captures the sparsity and skewness of their distribution. The statistically optimal communication scheme arising from the analysis of this model leads to a new sparsification technique for SGD, which concatenates random-k and top-k, considered separately in the prior literature. We show through extensive experiments on both image and language domains with CIFAR-10, ImageNet, and Penn Treebank datasets that the concatenated application of these two sparsification methods consistently and significantly outperforms either method applied alone.
Implicit discourse relation recognition is a challenging task due to the lack of connectives as strong linguistic clues. Previous methods primarily encode two arguments separately or extract the specific interaction patterns for the task, which have not fully exploited the annotated relation signal. Therefore, we propose a novel TransS-driven joint learning architecture to address the issues. Specifically, based on the multi-level encoder, we 1) translate discourse relations in low-dimensional embedding space (called TransS), which could mine the latent geometric structure information of argumentrelation instances; 2) further exploit the semantic features of arguments to assist discourse understanding; 3) jointly learn 1) and 2) to mutually reinforce each other to obtain the better argument representations, so as to improve the performance of the task. Extensive experimental results on the Penn Discourse TreeBank (PDTB) show that our model achieves competitive results against several state-of-the-art systems.
for free word order languages such as Myanmar. This paper describes a dependency head annotation scheme with Universal part-of-speech and Universal Dependencies for Myanmar dependency treebank. Currently 22,810 sentences and 680,218 tokens were annotated from three corpora for Myanmar dependency treebank. Some language specific issues are also described with examples. Raw syntactic structures were annotated automatically by UDPipe according to the Universal Dependencies based on Universalpart-of-speech tag scheme. Then unsupervised annotated dependency head structures have been manually updated in post processing. To be reliable and speedy post process with reduced errors for manual updating, selected sentences were added to the training data after being updated. After that the model has been retrained and the remaining sentences were parsed by UDPipe. Post processing was repeated until all sentences were updated. Some specifications of dependency annotation schemes in sentences encountered in post processing are presented with examples. For parsing performance of annotated data, cross validation tests and parsing experiments were performed. Moreover, annotated treebank data have also been evaluated by CoNLL 2017 evaluation script for parsing performance. Results of parsing experiments and evaluation are also reported by unlabeled and labeled attachment scores and demonstrated that the proposed method is a suitable way for building Myanmar dependency trees. Moreover, syntax structures of treebank are also analyzed and syntax information is also presented. This dependency head annotation for dependency treebank is the first work for Myanmar language as far as we know.
We report the results of our system on the Metaphor Detection Shared Task at the Second Workshop on Figurative Language Processing 2020. Our model is an ensemble, utilising contextualised and static distributional semantic representations, along with word-type concreteness ratings. Using these features, it predicts word metaphoricity with a deep multilayer perceptron. We are able to best the stateof-the-art from the 2018 Shared Task by an average of 8.0% F 1, and finish fourth in both subtasks in which we participate.
The deep inside-outside recursive autoencoder (DIORA; Drozdov et al. 2019a) is a selfsupervised neural model that learns to induce syntactic tree structures for input sentences without access to labeled training data. In this paper, we discover that while DIORA exhaustively encodes all possible binary trees of a sentence with a soft dynamic program, its vector averaging approach is locally greedy and cannot recover from errors when computing the highest scoring parse tree in bottom-up chart parsing. To fix this issue, we introduce S-DIORA, an improved variant of DIORA that encodes a single tree rather than a softlyweighted mixture of trees by employing a hard argmax operation and a beam at each cell in the chart. Our experiments show that through fine-tuning a pre-trained DIORA with our new algorithm, we improve the state of the art in unsupervised constituency parsing on the English WSJ Penn Treebank by 2.2 6% F1, depending on the data used for fine-tuning.
This study examined the syntactic impairments of Chinese Alzheimer’s disease patients with a dependency network approach. The dependency treebanks and dependency networks are constructed from the discourses of both the patient group and its healthy peers. By analysing the contrasts in the dependency networks of the two groups, we found that 1) the mean dependency distance (MDD) of the AD group is shorter than that of the HP group; furthermore, the MDDs of both AD and HP groups are far below the standard Chinese MDD; 2) the content words like remember, forget, know, etc. and the negative forms of the verbs like don’t know, can’t remember, can’t say, etc. show highly repetitive uncertain and negative expressions that are typical of the predicates of the clauses of AD patients; 3) the function word vertices in the AD dependency network have distinctive network parameters such as higher ‘betweenness’ centrality, closeness centrality, and clustering coefficients, etc., indicating that the syntax of AD is impaired and features more simplified stereotypes. These results indicate that the syntax of the AD group has been impaired from parts of speech to the whole syntactic structure.
One of the biggest attractions in the tourism industry in Bandung is nature tourism. There is still such a constraint related to get information about nature tourism in Bandung because new attractions in Bandung always appear every year. This is felt particularly for foreign tourists outside of Bandung. Tourists are still confused to find new and popular tourist attractions, which are places that are worth visiting or not. By implementing Cyber-Physical-Social System (CPSS) with a new approach that is emphasized on social aspect in smart tourism based on Service Oriented Architecture (SOA) as methodology can influence other travelers to visit tourist attractions in Bandung. The main results are tourists will get information such as location, route, images, rating, captions of tourist attractions, and the most important thing is to be able to exchange information with others. Smart tourism is more flexible because it is web based and does not depend on the operating system used, does not require database storage, does not take up storage space, and is free. Tourists can access smart tourism anytime and anywhere.
UDon2 is an open-source library for manipulating dependency trees represented in the CoNLL-U format. The library is compatible with the Universal Dependencies. UDon2 is aimed at developers of downstream Natural Language Processing applications that require manipulating dependency trees on the sentence level (in addition to other available tools geared towards working with treebanks).
This article presents a theory of the initiation and incrementation mechanisms whereby individual phonetic innovations become community-wide sound changes. The theory asserts that language learners are community-oriented and momentum-sensitive: they are community-oriented in that they acquire and obey a mental representation of the collective linguistic norm of their speech community, rejecting individual idiosyncrasies; they are momentum-sensitive in that their mental representation of the community norm includes an age vector encoding linguistic differences between age groups. The theory is shown to fulfil four critical desiderata: (i) it accounts for the sporadic and localized occurrence of community-wide sound change, (ii) it incorporates Ohala’s prediction of a lawful relationship between the strength of the phonetic biases driving individual innovation and the typological frequency of the corresponding sound changes, (iii) it explains how community-wide sound change advances by intergenerational incrementation producing adolescent peaks in apparent time, and (iv) it reliably generates monotonic—including sigmoid—diachronic trajectories. Moreover, the hypotheses of community orientation and sensitivity to momentum, combined with the mechanical effects of density of contact, suffice to explain several macroscopic phenomena in the propagation of sound change, including class stratification, the curvilinear pattern in change from below, and the existence of change reversals. During propagation, linguistic variants do acquire indexical value, and so social meaning, but this produces only small-scale attitudinal effects; it is not the force that drives the intergenerational incrementation of sound change.
In natural vision, noisy and distorted visual inputs often change our perceptual strategy in scene perception. However, it is unclear the extent to which the affective meaning embedded in the degraded natural scenes modulates our scene understanding and associated eye movements. In this eye-tracking experiment by presenting natural scene images with different categories and levels of emotional valence (high-positive, medium-positive, neutral/low-positive, medium-negative, and high-negative), we systematically investigated human participants' perceptual sensitivity (image valence categorization and arousal rating) and image-viewing gaze behaviour to the changes of image resolution. Our analysis revealed that reducing image resolution led to decreased valence recognition and arousal rating, decreased number of fixations in image-viewing but increased individual fixation duration, and stronger central fixation bias. Furthermore, these distortion effects were modulated by the scene valence with less deterioration impact on the valence categorization of negatively valenced scenes and on the gaze behaviour in viewing of high emotionally charged (high-positive and high-negative) scenes. It seems that our visual system shows a valence-modulated susceptibility to the image distortions in scene perception.
We describe an approach to statistical parsing with Tree-Wrapping Grammars (TWG). TWG is a tree-rewriting formalism which includes the tree-combination operations of substitution, sisteradjunction and tree-wrapping substitution. TWGs can be extracted from constituency treebanks and aim at representing long distance dependencies (LDDs) in a linguistically adequate way. We present a parsing algorithm for TWGs based on neural supertagging and A * parsing. We extract a TWG for English from the treebanks for Role and Reference Grammar and discuss first parsing results with this grammar.
Parsing sentences into syntax trees can benefit downstream applications in NLP. Transition-based parsers build trees by executing actions in a state transition system. They are computationally efficient, and can leverage machine learning to predict actions based on partial trees. However, existing transition-based parsers are predominantly based on the shift-reduce transition system, which does not align with how humans are known to parse sentences. Psycholinguistic research suggests that human parsing is strongly incremental: humans grow a single parse tree by adding exactly one token at each step. In this paper, we propose a novel transition system called attach-juxtapose. It is strongly incremental; it represents a partial sentence using a single tree; each action adds exactly one token into the partial tree. Based on our transition system, we develop a strongly incremental parser. At each step, it encodes the partial tree using a graph neural network and predicts an action. We evaluate our parser on Penn Treebank (PTB) and Chinese Treebank (CTB). On PTB, it outperforms existing parsers trained with only constituency trees; and it performs on par with state-of-the-art parsers that use dependency trees as additional training data. On CTB, our parser establishes a new state of the art. Code is available at this https URL.
This paper discusses the theoretical bases as well as the pragmatic implementation of the lemmatization of the Late Latin Charter Treebanks (LLCT). LLCT is a set of three dependency treebanks (LLCT1, LLCT2, LLCT3) of Early Medieval Latin documentary texts (charters) written in Italy between AD 714 and 1000 (c. 594,000 tokens). The original model for the lemmatization of LLCT was the Latin Dependency Treebank (LDT), which is mainly Classical standard Latin and based on the entries of Lewis and Short’s Latin Dictionary. Since LLCT reflects later linguistic developments of Latin and contains a plethora of non-standard proper names, particular attention is paid to how non-standard lexemes are lemmatized systematically to make the lemmatization maximally usable. The theoretical underpinnings to manage the lemmatization boil down to two principles: the evolutionary principle and the parsimony principle.
For sequence models with large word-level vocabularies, a majority of network parameters lie in the input and output layers. In this work, we describe a new method, DeFINE, for learning deep word-level representations efficiently. Our architecture uses a hierarchical structure with novel skip-connections which allows for the use of low dimensional input and output layers, reducing total parameters and training time while delivering similar or better performance versus existing methods. DeFINE can be incorporated easily in new or existing sequence models. Compared to state-of-the-art methods including adaptive input representations, this technique results in a 6% to 20% drop in perplexity. On WikiText-103, DeFINE reduces total parameters of Transformer-XL by half with minimal impact on performance. On the Penn Treebank, DeFINE improves AWD-LSTM by 4 points with a 17% reduction in parameters, achieving comparable performance to state-of-the-art methods with fewer parameters. For machine translation, DeFINE improves a Transformer model by 2% while simultaneously reducing total parameters by 26%
The importance of affect processing to human behavior has long driven researchers to pursue its measurement. In this study, we compared the relative fidelity of measurements of neural activation and physiology (i.e., heart rate change) in detecting affective valence induction across a broad continuum of conveyed affective valence. We combined intra-subject neural activation based multivariate predictions of affective valence with measures of heart rate (HR) deceleration to predict predefined normative affect rating scores for stimuli drawn from the International Affective Picture System (IAPS) in a population (n = 50) of healthy adults. In sum, we found that patterns of neural activation and HR deceleration significantly, and uniquely, explain the variance in normative valent scores associated with IAPS stimuli; however, we also found that patterns of neural activation explain a significantly greater proportion of that variance. These traits persisted across a range of stimulus sets, differing by the polar-extremity of their positively and negatively valent subsets, which represent the positively and negatively valent polar-extremity of stimulus sets reported in the literature. Overall, these findings support the acquisition of heart rate deceleration concurrently with fMRI to provide convergent validation of induced affect processing in the dimension of affective valence.
Traditional approaches to set goals in second language (L2) vocabulary acquisition relied either on word lists that were obtained from large L1 corpora or on collective knowledge and experience of L2 experts, teachers, and examiners. Both approaches are known to offer some advantages, but also to have some limitations. In this paper, we try to combine both sources of information, namely the official reference level description for French language and the FLElex lexical database. Our aim is to train a statistical model on the French RLD that would be able to turn the distributional information from FLElex into one of the six levels of the Common European Framework of Reference for languages (CEFR). We show that such approach yields a gain of 29\\% in accuracy compared to the method currently used in the CEFRLex project. Besides, our experiments also offer deeper insights into the advantages and shortcomings of the two traditional sources of information (frequency vs. expert knowledge).
In this paper, we compare four state-of-the-art neural network dependency parsers for the Semitic language Amharic. As Amharic is a morphologically-rich and less-resourced language, the out-of-vocabulary (OOV) problem will be higher when we develop data-driven models. This fact limits researchers to develop neural network parsers because the neural network requires large quantities of data to train a model. We empirically evaluate neural network parsers when a small Amharic treebank is used for training. Based on our experiment, we obtain an 83.79 LAS score using the UDPipe system. Better accuracy is achieved when the neural parsing system uses external resources like word embedding. Using such resources, the LAS score for UDPipe improves to 85.26. Our experiment shows that the neural networks can learn dependency relations better from limited data while segmentation and POS tagging require much data.
Eating is a process that involves all senses. Recent research has shown that both food-intrinsic and extrinsic sensory factors play a role in the taste of the food we consume. Moreover, many studies have explored the relationship between emotional state and taste perception, where certain emotional states have been shown to alter the perception of basic tastes. This opens up a whole new world of possibilities for the design of eating environments which take into account both sensory attributes as well as their emotional associations. Here, we used virtual reality to study the effect of colours and music, with specific emotional associations, on the evaluation of cold brew coffee. Based on an online study (N=76), two colours and two pieces of music with similar emotional arousal but opposing valence ratings were chosen to produce a total of eight virtual coloured environments. Forty participants were recruited for the on-site experiment, which consisted of three blocks. First, a blind tasting of four coffee samples (0%, 2.5%, 5%, 7.5% sucrose) was carried out. Next, participants experienced the eight environments via an HTC Vive Pro headset and evaluated their expected liking, sweetness and bitterness of a mug of coffee presented in VR. Finally, they tasted identical 5% coffee samples in the same eight environments. Results revealed One of the key findings of this study that, when only one factor (colour or music) was manipulated, background colour significantly influenced coffee liking. When colour and music were used in combination, however, we found an overall effect of music valence on coffee sweetness, as well as an interaction effect of colour and music on liking. These results reinforce the importance of the extrinsic sensory and emotion factors on food expectations and liking. Overall, these results are in line with previous research, where positive emotions can lead to increased food liking and higher sweetness compared to negative emotions.
The large communication cost for exchanging gradients between different nodes significantly limits the scalability of distributed training for large-scale learning models. Motivated by this observation, there has been significant recent interest in techniques that reduce the communication cost of distributed Stochastic Gradient Descent (SGD), with gradient sparsification techniques such as top-k and random-k shown to be particularly effective. The same observation has also motivated a separate line of work in distributed statistical estimation theory focusing on the impact of communication constraints on the estimation efficiency of different statistical models. The primary goal of this paper is to connect these two research lines and demonstrate how statistical estimation models and their analysis can lead to new insights in the design of communication-efficient training techniques. We propose a simple statistical estimation model for the stochastic gradients which captures the sparsity and skewness of their distribution. The statistically optimal communication scheme arising from the analysis of this model leads to a new sparsification technique for SGD, which concatenates random-k and top-k, considered separately in the prior literature. We show through extensive experiments on both image and language domains with CIFAR-10, ImageNet, and Penn Treebank datasets that the concatenated application of these two sparsification methods consistently and significantly outperforms either method applied alone.
Contextual information can modulate the conditioned response to a threat signal (conditioned stimulus, CS+): fear responses are either potentiated or attenuated depending on whether the context is threatening or safe. In this study, we investigated the influence of context on conditioned fear as well as on generalization of conditioned fear. Thirty-two participants underwent a cue-in-context learning protocol in virtual reality (VR). On Day 1 (acquisition), participants received a mild painful electric shock (unconditioned stimulus, US) in one virtual room (fear context, CTX+) at the offset of one colored light (CS+), but never at the offset of another colored light (CS-). In a second room (safety context, CTX-), the two lights were also presented, but not the US. Successful cue conditioning was indicated by aversive ratings and startle potentiation but not skin conductance responses (SCR) to CS+ versus CS- in CTX+ and not in CTX-. On Day 2 (generalization), participants re-visited both fear and safety contexts plus a generalization context (G-CTX), which was an equal mix of CTX+ and CTX-. The two CSs were shown again in all three contexts. Generalization of conditioned fear was revealed in affective ratings (CS+ was rated more aversive than CS- in G-CTX), but not in physiological measures (equal startle potentiation to CS+ versus CS- in all contexts). In sum, contextual information modulates the responses to a threat signal such that a safety context can inhibit conditioned fear. Interestingly, generalization processes also depend on contextual information.
In this paper, we introduce the first Universal Dependencies (UD) treebank for standard Albanian, consisting of 60 sentences collected from the Albanian Wikipedia, annotated with lemmas, universal part-of-speech tags, morphological features and syntactic dependencies. In addition to presenting the treebank itself, we discuss a selection of linguistic constructions in Albanian whose analysis in UD is not self-evident, including core arguments and the status of indirect objects, pronominal clitics, genitive constructions, prearticulated adjectives, and modal verbs.
This paper demonstrates how multilingual adolescents initiate language-directed teasing in family interaction and thus contribute to reinforcing or challenging social hierarchies and norms in the family. It investigates the case of a multilingual family living in Northern Norway (two parents and five children ranging from 3 to 18 years old). To a varying extent, and with varying degrees of competence, all family members use three languages in their daily lives: English, Spanish, and Norwegian. The data consists of self-recorded material of family interactions (9 h; 549 min) that were collected over the course of one year. A close interactional analysis shows how the siblings target linguistic production in teasing attacks, and use language-directed teasing as an interactional resource to position themselves and their family members. Drawing on Billig's (2005) theory of disciplinary humor, the article argues that playful corrections of perceived norm transgressions may be understood as situated (re)production and negotiation of social and linguistic norms, through which the young family members participate in the construction of the family as a community of practice.
Affect fluctuates in a moment-to-moment fashion, reflecting the continuous relationship between the individual and the environment. Despite substantial research, there remain important open questions regarding how a stream of sensory input is dynamically represented in experienced affect. Here, approaching affect as a temporally dependent process, we show that momentary affect is shaped by a combination of the affective impact of stimuli (i.e., visual images for the current studies) and previously experienced affect. We also found that this temporal dependency is influenced by uncertainty of the affective context. Participants in each trial viewed sequentially presented images and subsequently reported their affective experience, which was modeled based on images' normative affect ratings and participants' previously reported affect. Study 1 showed that self-reported valence and arousal in a given trial is partly shaped by the affective impact of the given images and previously experienced affect. In Study 2, we manipulated context uncertainty by controlling occurrence probabilities for normatively pleasant and unpleasant images in separate blocks. Increasing context uncertainty (i.e., random occurrence of pleasant and unpleasant images) was associated with increased negative affect. In addition, the relative contribution of the most recent image to experienced pleasantness increased with increasing context uncertainty. Taken together, these findings provide clear behavioral evidence that momentary affect is a temporally dependent and continuous process, which reflects the affective impact of recent input variables and the previous internal state, and that this process is sensitive to the affective context and its uncertainty. (PsycInfo Database Record (c) 2022 APA, all rights reserved).
Keywords play a major role in representing the gist of a document. Therefore, a lot of Natural Language processing tools have been implemented to identify keywords in both structured and unstructured texts. Text that appears in social media platforms such as twitter is mostly unstructured because of the character limitation. Consequently, a lot of short terms and symbols such as emoticons and URLs are included in tweets. Keyword extraction from grammatically ambiguous text is not easy compared to structured text since it is hard to rely on the linguistic features in unstructured texts. But when it comes to news on twitter, it may contain somewhat structured text than informal text does but it depends on the tweeter, the person who posts the tweet. In this paper, a methodology is proposed to extract keywords from a given tweet to retrieve relevant news that has been posted on twitter, for fake news detection. The intention of extracting keywords is to find more related news efficiently and effectively. For this approach, a corpus that contains tweet texts from different domains is built in order to make this approach more generic instead of making it a domain-specific approach. In fact, the Stanford Core NLP tool kit, Wordnet linguistic database and statistical method are used for extracting keywords from a tweet. For the system evaluation, the Turing test which has human intervention is used. The system was able to acquire an accuracy of 67.6% according to the evaluation conducted.
The LiLa: Linking Latin project aims to build a Knowledge Base of language resources for the study of Latin (corpora, digital lexicons, natural-language-processing tools), based on the Linked Open Data paradigm. In this paper, we discuss the goals and motivation of the project. In particular, we focus on the role played by the lemma as a hub node that holds together the network of linguistic information. The architecture of LiLa is therefore based on lemmas and their morphological properties. The paper illustrates the strategies used to build a collection of Latin lemmas and the first experiments to link them to a set of textual resources (Latin treebanks).
In this paper, we explore self-distillation as a means to improve statistical dependency parsing models for Dutch and German over purely supervised training. Self-distillation (Furlanello et al. 2018) trains a new student model on the output of an existing (weaker) teacher model. In contrast to most previous work on self-distillation, we perform distillation using a large, unannotated corpus. We show that in dependency parsing as sequence labeling (Spoustov´a and Spousta 2010, Strzyz et al. 2019), self-distillation plus finetuning provides large improvements over models that use supervised training. We carry out experiments on the German T¨uBa-D/Z universal dependency (UD) treebank (C¸ ¨oltekin et al. 2017) and the UD conversion of the Dutch Lassy Small treebank (Bouma and van Noord 2017). We find that self-distillation improves German parsing accuracy of a bidirectional LSTM parser from 92.23 to 94.33 Labeled Attachment Score (LAS). Similarly, on Dutch we see improvement from 89.89 to 91.84 LAS.