Papers reviewed and determined not to be word norm studies. Use the flag icon to report errors or suggest re-inclusion.
16504 papers
Many adaptive gradient methods have been successfully applied to train deep neural networks, such as Adagrad, Adadelta, RMSprop and Adam. These methods perform local optimization with an element-wise scaling learning rate based on past gradients. Although these methods can achieve an advantageous training loss, some researchers have pointed out that their generalization capability tends to be poor as compared to stochastic gradient descent (SGD) in many applications. These methods obtain a rapid initial training process but fail to converge to an optimal solution due to the unstable and extreme learning rates. In this paper, we investigate the adaptive gradient methods and get the insights on various factors that may lead to poor performance of Adam. To overcome that, we propose a bounded scheduling algorithm for Adam, which can not only improve the generalization capability but also ensure the convergence. To validate our claims, we carry out a series of experiments on the image classification and the language modeling tasks on several standard benchmarks such as ResNet, DenseNet, SENet and LSTM on typical data sets such as CIFAR-10, CIFAR-100 and Penn Treebank. Experimental results show that our method can eliminate the generalization gap between Adam and SGD, meanwhile maintaining a relative high convergence rate during training.
PURPOSE: Corneal confocal microscopy (CCM) is an imaging method to detect loss of nerve fibers in the cornea. The impact of image quality on the CCM parameters has not been investigated. We developed a quality index (QI) with 3 stages for CCM images and compared the influence of the image quality on the quantification of corneal nerve parameters using 2 modes of analysis in healthy volunteers and patients with known peripheral neuropathy. METHODS: Images of 75 participants were a posteriori analyzed, including 25 each in 3 image quality groups (QI 1-QI 3). Corneal nerve fiber length (CNFL) was analyzed using automated and semiautomated software, and corneal nerve fiber density and corneal nerve branch density were quantified using automated image analysis. Three masked raters assessed CCM image quality (QI) independently and categorized images into groups QI 1-QI 3. In addition, statistical analysis was used to compare interrater reliability. Analysis of variance was used for analysis between the groups. Interrater reliability analysis between the image ratings was performed by calculating Fleiss' kappa and its 95% confidence interval. RESULTS: CNFL, corneal nerve fiber density, and corneal nerve branch density increased significantly with QI (P < 0.001, all post hoc tests P < 0.05). CNFL was higher using semiautomated compared with automated nerve analysis, independent of QI. Fleiss kappa coefficient for interrater reliability of QI was 0.72. CONCLUSIONS: The quantification of corneal nerve parameters depends on image quality, and poorer quality images are associated with lower values for corneal nerve parameters. We propose the QI as a tool to reduce variability in quantification of corneal nerve parameters.
To create emotionally expressive robots, designers of human-robot interaction routinely translate emotion theories into instruments through which we estimate, quantify and analyze human emotional responses to robot behaviour. Pragmatically, we often use straightforward models such as Russell's circumplex, treating emotion as a single point in a two-dimensional space. However, this simple metaphor and its consequent representations omit many aspects of real emotional experience, can lead to erroneous data and may undermine computational models that rely on them. Problems with emotion representations currently prevalent in human-robot interaction fall into three categories: (1)Representations are static and singular, whereas real emotions can be dynamic, multi-valued, uncertain or conflicting. (2)The framing of an interaction is unspecified (i.e., in an affective rating task: which part of an interaction involving multiple parties and perspectives the participant is meant to consider). (3) Participant responses captured with instruments and methods that are not well-understood by experimenters nor participants produce data that is hard to interpret. We propose alternative emotion representations to account for dynamic emotions inherent in interactive contexts; scrutinize framing ambiguities in study tasks and argue for mixed-methods approaches to achieve shared understanding of emotion representations between participants and researchers.
Organizing multivariate data spaces by their dimensions or attributes can be a rather difficult task. Most of the work in this area focuses on the statistical aspects such as correlation clustering, dimension reduction, and the like. These methods typically produce hierarchies in which the leaf nodes are labeled by the attribute names while the inner nodes are often represented by just a statistical measure and criterion, such as a threshold. This makes them difficult to understand for mainstream users. Taxonomies in science, biology, engineering, etc. on the other hand, are easy to comprehend since they provide meaningful labels at the inner nodes as well. Labeling inner nodes of taxonomies automatically requires the identification of hypernyms. Our proposed framework, called Taxonomizer, takes a visual analytics approach to meet this challenge. It appeals to the wisdom of humans to liaise with state of the art data analytics, neural word embeddings, and lexical databases. It consists of a set of visual tools that starts out with an automatically computed hierarchy where the leaf nodes are the original data attributes, and it then allows users to sculpt high-quality taxonomies for any multivariate dataset.
Is phonetic information encoded by distributional biases in the lexicon? Are phonotactic constraints robust enough to help a learner infer the phonic pattern of a language? Our work in progress attempts to shed light on these questions via: (i) statistical description of the distributional biases in phone sequences in a lexical database and (ii) connectionist simulation. The simulation focuses on V-to-V relations in V(C)’C(C)V phone strings since both harmony and contour constraints (the tendency for the vowels to share or avoid repetition of phonic properties, respectively) have been found in the distributional study (Albano, 2002).
Characterizing the distribution of crossing dependencies in natural language dependency trees is a crucial task for building parsers and understanding the formal properties of human language. A number of formal restrictions on crossing dependencies have been proposed, including bounds on gap degree, edge degree, and end-point crossings. Here we ask whether the empirical distribution of crossing dependencies in dependency treebanks offers evidence for these formal restrictions as true, independent constraints on dependency trees, or whether the distribution can be explained using other, more generic constraints affecting dependency trees. Specifically, we explore the null hypothesis that crossing dependencies are formally unrestricted, but occur at a low rate. We implement the null hypothesis using random trees where crossing dependencies occur at the same rate as in natural language trees, but without any formal restrictions. We find that this baseline generally does not reproduce the same distribution of gap degree, edge degree, endpoint-crossing, and heads' depth difference as real trees, suggesting that these formal constraints are a consequence of factors beyond the rate of crossing dependencies alone.
Mirror-sensory synaesthetes mirror the pain or touch that they observe in other people on their own bodies. This type of synaesthesia has been associated with enhanced empathy. We investigated whether the enhanced empathy of people with mirror-sensory synesthesia influences the experience of situations involving touch or pain and whether it affects their prosocial decision making. Mirror-sensory synaesthetes ( N = 18, all female), verified with a touch-interference paradigm, were compared with a similar number of age-matched control individuals (all female). Participants viewed arousing images depicting pain or touch; we recorded subjective valence and arousal ratings, and physiological responses, hypothesizing more extreme reactions in synaesthetes. The subjective impact of positive and negative images was stronger in synaesthetes than in control participants; the stronger the reported synaesthesia, the more extreme the picture ratings. However, there was no evidence for differential physiological or hormonal responses to arousing pictures. Prosocial decision making was assessed with an economic game assessing altruism, in which participants had to divide money between themselves and a second player. Mirror-sensory synaesthetes donated more money than non-synaesthetes, showing enhanced prosocial behaviour, and also scored higher on the Interpersonal Reactivity Index as a measure of empathy. Our study demonstrates the subjective impact of mirror-sensory synaesthesia and its stimulating influence on prosocial behaviour. This article is part of the discussion meeting issue ‘Bridging senses: novel insights from synaesthesia’.
Automatic emotion regulation (AER) is an important type of emotion regulation in our daily life. Most of the previous studies concerning AER are done in the conscious level. Little is known about the AER under the subliminal level. The present study was to investigate the AER at the different perceptual levels (i.e., explicitly and implicitly) simultaneously, and the associated neural differences using functional magnetic resonance imaging. Priming paradigm was adopted in which the inhibition or neutral words were used as primes and the negative picutres were used as targets. In the experiment, the duration time of priming words was manipulated at 33 or 50 ms in the implicit level and 3000 ms in the explicit level. The participants were required to make emotional valence rating of the negative pictures while undergoing functional magnetic resonance imaging scanning. The results showed that the participants experienced less negative emotion in inhibition words priming condition contrary to neutral words priming condition. Significant differences were also found in the left ventrolateral prefrontal cortex and left dorsolateral prefrontal cortex at the implicit and explicit AER. The findings of this study demonstrate that inhibition words can automatically and effectively reduce an individual's negative emotion experience, and left ventrolateral prefrontal cortex and left dorsolateral prefrontal cortex have been both implicated in self-control during AER.
Alterations in fear learning/generalization are considered to be relevant mechanisms engendering the development of anxiety disorders being the most prevalent mental disorders. Although anxiety disorders almost exclusively have their first onset in childhood and adolescence, etiological research focuses on adult individuals. In this study, we evaluated findings of a recent meta-analysis of genome-wide association studies in adult anxiety disorders with significant associations of four single nucleotide polymorphisms (SNPs) in a large cohort of 347 healthy children (8-12 years) characterized for dimensional anxiety. We investigated the modulation of anxiety parameters by these SNPs in a discriminative fear conditioning and generalization paradigm in the to-date largest sample of children. Results extended findings of the meta-analysis showing a genomic locus on 2p21 to modulate anxious personality traits and arousal ratings. These SNPs might, thus, serve as susceptibility markers for a shared risk across pathological anxiety, presumably mediated by alterations in arousal.
We describe a cross-lingual transfer method for dependency parsing that takes into account the problem of word order differences between source and target languages. Our model only relies on the Bible, a considerably smaller parallel data than the commonly used parallel data in transfer methods. We use the concatenation of projected trees from the Bible corpus, and the gold-standard treebanks in multiple source languages along with cross-lingual word representations. We demonstrate that reordering the source treebanks before training on them for a target language improves the accuracy of languages outside the European language family. Our experiments on 68 treebanks (38 languages) in the Universal Dependencies corpus achieve a high accuracy for all languages. Among them, our experiments on 16 treebanks of 12 non-European languages achieve an average UAS absolute improvement of 3.3% over a state-of-the-art method.
The paper aims to examine how the acoustic input (the surface form) and the abstract linguistic representation (the underlying representation) interact during spoken word recognition by investigating left-dominant tone sandhi, a tonal alternation in which the underlying tone of the first syllable spreads to the sandhi domain. We conducted two auditory-auditory priming lexical decision experiments on Shanghai left-dominant sandhi words with less-frequent and frequent Shanghai users, in which each disyllabic target was preceded by monosyllabic primes either sharing the same underlying tone, surface tone, or being unrelated to the tone of the first syllable of the sandhi targets. Results showed a surface priming effect but not an underlying priming effect in younger speakers who used Shanghai less frequently, but no surface or underlying priming effect in older speakers who used Shanghai more often. Moreover, the surface priming did not interact with speakers' familiarity ratings to the sandhi targets. These patterns suggest that left-dominant Shanghai sandhi words may be represented in the sandhi form in the mental lexicon. The results are discussed in the context of how phonological opacity, productivity, the non-structure-preserving nature of tone spreading, and speakers' semantic knowledge influence the representation and processing of tone sandhi words.
Lexical simplification (LS) aims to replace complex words in a given sentence with their simpler alternatives of equivalent meaning. Recently unsupervised lexical simplification approaches only rely on the complex word itself regardless of the given sentence to generate candidate substitutions, which will inevitably produce a large number of spurious candidates. We present a simple BERT-based LS approach that makes use of the pre-trained unsupervised deep bidirectional representations BERT. Despite being entirely unsupervised, experimental results show that our approach obtains obvious improvement than these baselines leveraging linguistic databases and parallel corpus, outperforming the state-of-the-art by more than 11 Accuracy points on three well-known benchmarks.
This paper presents a novel crowd-sourced resource for multimodal discourse: our resource characterizes inferences in image-text contexts in the domain of cooking recipes in the form of coherence relations. Like previous corpora annotating discourse structure between text arguments, such as the Penn Discourse Treebank, our new corpus aids in establishing a better understanding of natural communication and common-sense reasoning, while our findings have implications for a wide range of applications, such as understanding and generation of multimodal documents.
We present a novel semantic framework for modeling linguistic expressions of generalization— generic, habitual, and episodic statements—as combinations of simple, real-valued referential properties of predicates and their arguments. We use this framework to construct a dataset covering the entirety of the Universal Dependencies English Web Treebank. We use this dataset to probe the efficacy of type-level and token-level information—including hand-engineered features and static (GloVe) and contextual (ELMo) word embeddings—for predicting expressions of generalization.
Although there is a wide consensus on how sleep processes declarative memories, how sleep affects emotional memories remains elusive. Moreover, studies assessing the long-term effect of sleep on emotional memory consolidation are scarce. Studies testing subclinical populations characterized by REM abnormalities are also lacking. Here we aimed to (i) investigate the fate of emotional memories and the potential unbinding (or preservation) between content and affective tone over time (i.e., 1 week), (ii) explore the role of seven nights of sleep (recorded via actigraphy) in emotional memory consolidation, and (iii) assess whether participants with self-reported mild-moderate depressive symptoms forget less emotional information compared to participants with low depression symptoms. We found that, although at the immediate recognition session emotional information was forgotten more than neutral information, a week later it was forgotten less than neutral information. This effect was observed both in participants with low and mild-moderate depressive symptoms. We also observed an increase in valence rating over time for negative pictures, whereas perceived arousal diminished a week later for both types of stimuli (unpleasant and neutral); an initial decrease was already observable at the immediate recognition session. Interestingly, we observed a negative association between sleep efficiency across the week and change in memory discrimination for unpleasant pictures over time, i.e., participants who slept worse were the ones who forgot less emotional information. Our results suggest that emotional memories are resistant to forgetting, particularly when sleep is disrupted, and they are not affected by non-clinical depression symptomatology.
Recent neuroscience research tried to identify biological markers underlying schizophrenia’s (SZ) symptoms. Results showed a functional hypofrontality in SZ patients during both cognitive and emotional tasks. Here, we submitted an experimental (E) group of patients to a neurofeedback (NF) training during emotion induction (T1) and assessed its efficacy by comparing the frontal neural activity before (T0) and after it (T2), with regard to a control (C) group. Functional near-infrared spectroscopy (fNIRS) was used during an emotional task with valence and arousal rating. Behavioral results showed that patients of both groups could identify pictures’ valence, both in T0 and T2. However, a significant interaction effect revealed that negative and positive stimuli received more positive values in T2 compared to T0 only in E group, as a consequence of an alleged more functional management of negative feelings. Such results were paralleled by imaging data that showed increased O2Hb levels over frontal areas for positive and negative pictures compared to neutral ones, which were even more evident in the E group in T2. The preliminary results of the present study highlight the possible application of NF training to sustain patients’ achieving more awareness and regulation during emotion processing.
Facial expressions are fundamental to interpersonal communication, including social interaction, and allow people of different ages, cultures, and languages to quickly and reliably convey emotional information. Historically, facial expression research has followed from discrete emotion theories, which posit a limited number of distinct affective states that are represented with specific patterns of facial action. Much less work has focused on dimensional features of emotion, particularly positive and negative affect intensity. This is likely, in part, because achieving inter-rater reliability for facial action and affect intensity ratings is painstaking and labor-intensive. We use computer-vision and machine learning (CVML) to identify patterns of facial actions in 4,648 video recordings of 125 human participants, which show strong correspondences to positive and negative affect intensity ratings obtained from highly trained coders. Our results show that CVML can both (1) determine the importance of different facial actions that human coders use to derive positive and negative affective ratings when combined with interpretable machine learning methods, and (2) efficiently automate positive and negative affect intensity coding on large facial expression databases. Further, we show that CVML can be applied to individual human judges to infer which facial actions they use to generate perceptual emotion ratings from facial expressions.
Several studies have attempted to investigate how the brain codes emotional value when processing music of contrasting levels of dissonance; however, the lack of control over specific musical structural characteristics (i.e., dynamics, rhythm, melodic contour or instrumental timbre), which are known to affect perceived dissonance, rendered results difficult to interpret. To account for this, we used functional imaging with an optimized control of the musical structure to obtain a finer characterization of brain activity in response to tonal dissonance. Behavioral findings supported previous evidence for an association between increased dissonance and negative emotion. Results further demonstrated that the manipulation of tonal dissonance through systematically controlled changes in interval content elicited contrasting valence ratings but no significant effects on either arousal or potency. Neuroscientific findings showed an engagement of the left medial prefrontal cortex (mPFC) and the left rostral anterior cingulate cortex (ACC) while participants listened to dissonant compared to consonant music, converging with studies that have proposed a core role of these regions during conflict monitoring (detection and resolution), and in the appraisal of negative emotion and fear-related information. Both the left and right primary auditory cortices showed stronger functional connectivity with the ACC during the dissonant portion of the task, implying a demand for greater information integration when processing negatively valenced musical stimuli. This study demonstrated that the systematic control of musical dissonance could be applied to isolate valence from the arousal dimension, facilitating a novel access to the neural representation of negative emotion.
The affect associated with negative events fades faster than the affect associated with positive events (the fading affect bias). The fading affect bias is present in most participants and is thought to be evidence of a healthy coping mechanism operating in autobiographical memory. Prior research shows that the fading affect bias can be distorted by negative individual difference variables such as dysphoria and anxiety. The goal of this research is to link the fading affect bias to the positive individual difference variable of Grit. A total of 197 participants completed the short Grit Scale and were divided into four groups based on their Grit scores (i.e., low Grit to high Grit). Participants retrieved positive and negative event memories and then made affect ratings for the events. The results show that increased levels of Grit were associated with a stronger fading affect bias.
Recently, it has been shown that various auditory stimuli modulate flavour perception. The present study attempts to understand the effects of environmental sounds (park, food court, fast food restaurant, cafe, and bar sounds) on the perception of chocolate gelato (specifically, sweet, bitter, milky, creamy, cocoa, roasted, and vanilla notes) using the Temporal Check-All-That-Apply (TCATA) method. Additionally, affective ratings of the auditory stimuli were obtained using the Self-Assessment Manikin (SAM) in terms of their valence, arousal, and dominance. In total, 58 panellists rated the sounds and chocolate gelato in a sensory laboratory. The results revealed that bitterness, roasted, and cocoa notes were more evident when the bar, fast food, and food court sounds were played. Meanwhile, sweetness was cited more in the early mastication period when listening to park and café sounds. The park sound was significantly higher in valence, while the bar sound was significantly higher in arousal. Dominance was significantly higher for the fast food restaurant, food court, and bar sound conditions. Intriguingly, the valence evoked by the pleasant park sound was positively correlated with the sweetness of the gelato. Meanwhile, the arousal associated with bar sounds was positively correlated with bitterness, roasted, and cocoa attributes. Taken together, these results clearly demonstrate that people's perception of the flavour of gelato varied with the different real-world sounds used in this study.
Word vectors are at the core of many natural language processing tasks. Recently, there has been interest in post-processing word vectors to enrich their semantic information. In this paper, we introduce a novel word vector post-processing technique based on matrix conceptors (Jaeger 2014), a family of regularized identity maps. More concretely, we propose to use conceptors to suppress those latent features of word vectors having high variances. The proposed method is purely unsupervised: it does not rely on any corpus or external linguistic database. We evaluate the post-processed word vectors on a battery of intrinsic lexical evaluation tasks, showing that the proposed method consistently outperforms existing state-of-the-art alternatives. We also show that post-processed word vectors can be used for the downstream natural language processing task of dialogue state tracking, yielding improved results in different dialogue domains.
Imagining fictional creatures like zombies in survival situations boosts long-term memory for words encoded in these situations more than rating words for pleasantness (zombie effect). Study 1 required word-ratings in a zombie-survival scenario; participants were told they had to protect against either possible zombie attack or contamination. The zombie-survival situations yielded identical recall levels but higher recall rates than pleasantness. Study 2 matched a zombie-survival scenario on perceived fear with scenarios involving ghosts or predators. Perceived disgust in the zombie scenario was higher than in these other survival conditions. Words were remembered better when processed in survival scenarios than when rated for pleasantness, but there was no reliable difference in recall between the scenarios. In neither study did the number of death-related words produced in a word-fragment completion task fit the mortality salience account of the zombie memory effect. Overall findings suggest that this effect relates to the fear system.
Despite the fact that there are a number of researches working on Khmer Language in the field of Natural Language Processing along with some resources regarding words segmentation and POS Tagging, we still lack of high-level resources regarding syntax, Treebanks and grammars, for example. This paper illustrates the semi-automatic framework of constructing Khmer Treebank and the extraction of the Khmer grammar rules from a set of sentences taken from the Khmer grammar books. Initially, these sentences will be manually annotated and processed to generate a number of grammar rules with their probabilities once the Treebank is obtained. In our experiments, the annotated trees and the extracted grammar rules are analyzed in both quantitative and qualitative way. Finally, the results will be evaluated in three evaluation processes including Self-Consistency, 5-Fold Cross-Validation, Leave-One-Out Cross-Validation along with the three validation methods such as Precision, Recall, F1-Measure. According to the result of the three validations, Self-Consistency has shown the best result with more than 92%, followed by the Leave-One-Out Cross-Validation and 5-Fold Cross Validation with the average of 88% and 75% respectively. On the other hand, the crossing bracket data shows that Leave-One-Out Cross Validation holds the highest average with 96% while the other two are 85% and 89%, respectively.
xml treebank Annotated by Toon Van Hal, with student contributions by Mathieu Cuijpers; Sanderijn Gijbels; Yoran Joosten; Yordi Lenaerts; Eva Uffing; Chiara Van der Hasselt; Lisa Vanhee and Jolien Volders (KU Leuven Bachelor 3, 2018-2019). Based on a preparsed text by Alek Keersmaekers. Controlled by Toon Van Hal, Sanderijn Gijbels and Yoran Joosten.
Counterconditioning (CC) is a form of retroactive interference that inhibits expression of learned behavior. But similar to extinction, CC can be a fairly weak and impermanent form of interference, and the original behavior is prone to relapse. Research on CC is limited, especially in humans, but prior studies suggest it is more effective than extinction at modifying some behaviors (e.g., preference or valence ratings) than others (e.g., physiological arousal). Here, we used a within-subjects design to compare the effects of aversive-to-appetitive CC versus standard extinction on two separate tests of long-term memory in human adults: implicit physiological arousal and explicit episodic memory. Participants underwent Pavlovian fear conditioning to two semantic categories (animals, tools) paired with an electric shock. Conditioned stimuli (i.e., category exemplars) from one category were then extinguished, while stimuli from the other category were paired with a positive outcome. Participants returned 24-h later for a test of skin conductance responses (SCR) to the conditioned exemplars, as well as a surprise recognition memory test for stimuli encoded the previous day. Results showed reduced SCRs at a test for unique stimuli from a category that had undergone CC, relative to stimuli from a category that had undergone standard extinction. Additionally, participants selectively remembered more stimuli encoded during CC than extinction. These results provide new evidence that aversive-to-appetitive CC, as compared to extinction, strengthens memory for items directly associated with a positive outcome, which may provide stronger retrieval competition against a fear memory at test to help diminish fear relapse.
This paper proposes a novel Recurrent Neural Network (RNN) language model that takes advantage of character information. We focus on character n-grams based on research in the field of word embedding construction (Wieting et al. 2016). Our proposed method constructs word embeddings from character ngram embeddings and combines them with ordinary word embeddings. We demonstrate that the proposed method achieves the best perplexities on the language modeling datasets: Penn Treebank, WikiText-2, and WikiText-103. Moreover, we conduct experiments on application tasks: machine translation and headline generation. The experimental results indicate that our proposed method also positively affects these tasks
Abstract A puzzling fact about linguistic norms is that they are mainly stable, but the conventional variant sometimes changes. These transitions seem to be mostly S-shaped and, therefore, directed. Previous models have suggested possible mechanisms to explain these directed changes, mainly based on a bias favoring the innovative variant. What is still debated is the origin of such a bias. In this paper, we propose a refined taxonomy of mechanisms of language change and identify a family of mechanisms explaining self-actuated language changes. We exemplify this type of mechanism with the preference-based selection mechanism that relies on agents having dynamic preferences for different variants of the linguistic norm. The key point is that if these preferences align through social interactions, then new changes can be actuated even in the absence of external triggers. We present results of a multi-agent model and demonstrate that the model produces trajectories that are typical of language change.
The current CoNLL version of the second part of the Late Latin Charter Treebank (LLCT2). Early medieval Latin documentary texts with morphological and syntactic annotation. Ancient Language Dependency Treebank (ALDT) compatible linguistic annotation with modifications concerning morphology (see Korkiakangas & Passarotti, 2011, “Challenges in Annotating Medieval Latin Charters”). LLCT2 expands the chronological span of LLCT1 up to AD 897. LLCT2 contains 521 charters and 257,918 tokens. See Korkiakangas, [in print], “Late Latin Charter Treebank: contents and annotation”.
This paper presents work on the creation of a Universal Dependency (UD) treebank for Wolof as the first UD treebank within the Northern Atlantic branch of the Niger-Congo languages. The paper reports on various issues related to word segmentation for tokenization and the mapping of PoS tags, morphological features and dependency relations to existing conventions for annotating Wolof. It also outlines some specific constructions as a starting point for discussing several more general UD annotation guidelines, in particular for noun class marking, deixis encoding, and focus marking.
Abstract Multiword expressions can have both idiomatic and literal occurrences. For instance pulling strings can be understood either as making use of one’s influence, or literally. Distinguishing these two cases has been addressed in linguistics and psycholinguistics studies, and is also considered one of the major challenges in MWE processing. We suggest that literal occurrences should be considered in both semantic and syntactic terms, which motivates their study in a treebank. We propose heuristics to automatically pre-identify candidate sentences that might contain literal occurrences of verbal VMWEs, and we apply them to existing treebanks in five typologically different languages: Basque, German, Greek, Polish and Portuguese. We also perform a linguistic study of the literal occurrences extracted by the different heuristics. The results suggest that literal occurrences constitute a rare phenomenon. We also identify some properties that may distinguish them from their idiomatic counterparts. This article is a largely extended version of Savary and Cordeiro (2018).
In this paper, we propose a novel data augmentation method with respect to the target context of the data via self-supervised learning. Instead of looking for the exact synonyms of masked words, the proposed method finds words that can replace the original words considering the context. For self-supervised learning, we can employ the masked language model (MLM), which masks a specific word within a sentence and obtains the original word. The MLM learns the context of a sentence through asymmetrical inputs and outputs. However, without using the existing MLM, we propose a label-masked language model (LMLM) that can include label information for the mask tokens used in the MLM to effectively use the MLM in data with label information. The augmentation method performs self-supervised learning using LMLM and then implements data augmentation through the trained model. We demonstrate that our proposed method improves the classification accuracy of recurrent neural networks and convolutional neural network-based classifiers through several experiments for text classification benchmark datasets, including the Stanford Sentiment Treebank-5 (SST5), the Stanford Sentiment Treebank-2 (SST2), the subjectivity (Subj), the Multi-Perspective Question Answering (MPQA), the Movie Reviews (MR), and the Text Retrieval Conference (TREC) datasets. In addition, since the proposed method does not use external data, it can eliminate the time spent collecting external data, or pre-training using external data.
Abstract Lexicalized parsing models are based on the assumptions that (i) constituents are organized around a lexical head and (ii) bilexical statistics are crucial to solve ambiguities. In this paper, we introduce an unlexicalized transition-based parser for discontinuous constituency structures, based on a structure-label transition system and a bi-LSTM scoring system. We compare it with lexicalized parsing models in order to address the question of lexicalization in the context of discontinuous constituency parsing. Our experiments show that unlexicalized models systematically achieve higher results than lexicalized models, and provide additional empirical evidence that lexicalization is not necessary to achieve strong parsing results. Our best unlexicalized model sets a new state of the art on English and German discontinuous constituency treebanks. We further provide a per-phenomenon analysis of its errors on discontinuous constituents.
Many advances in Natural Language Processing have been based upon more expressive models for how inputs interact with the context in which they occur. Recurrent networks, which have enjoyed a modicum of success, still lack the generalization and systematicity ultimately required for modelling language. In this work, we propose an extension to the venerable Long Short-Term Memory in the form of mutual gating of the current input and the previous output. This mechanism affords the modelling of a richer space of interactions between inputs and their context. Equivalently, our model can be viewed as making the transition function given by the LSTM context-dependent. Experiments demonstrate markedly improved generalization on language modelling in the range of 3-4 perplexity points on Penn Treebank and Wikitext-2, and 0.01-0.05 bpc on four character-based datasets. We establish a new state of the art on all datasets with the exception of Enwik8, where we close a large gap between the LSTM and Transformer models.
The lack of understanding and definitional inconsistencies regarding agritourism and the importance of cooperation in sustaining this kind of tourism are underlined in the literature. This study analyzes the perceptions of agritourism and cooperation from actors in the sector using a plurality of methods, including unsupervised (a) text mining and (b) sentiment analysis with the use of a lexical database, as well as (c) supervised qualitative data analysis. Based on the assumption that destinations with different geographic characteristics have different features and products, two different destinations as for its accessibility and tourism recognition were selected for comparison: (a) an island—Lesvos in the North Aegean Sea, and (b) a continental mountain region—Plastiras Lake, in Greece. The data were collected from personal in-depth interviews and with the use of semi-structured questionnaires. From a methodological perspective, all three methods provided unique insights on the study’s themes, and the overall image of agritourism and cooperation was positive. A common understanding seems important for cooperation and networking; however, training is needed not only for effective promotion of agritourism, but also for cooperation techniques, benefits, trust-building mechanisms and best practices.
One of the central goals of Recurrent Neural Networks (RNNs) is to learn long-term dependencies in sequential data. Nevertheless, the most popular training method, Truncated Backpropagation through Time (TBPTT), categorically forbids learning dependencies beyond the truncation horizon. In contrast, the online training algorithm Real Time Recurrent Learning (RTRL) provides untruncated gradients, with the disadvantage of impractically large computational costs. Recently published approaches reduce these costs by providing noisy approximations of RTRL. We present a new approximation algorithm of RTRL, Optimal Kronecker-Sum Approximation (OK). We prove that OK is optimal for a class of approximations of RTRL, which includes all approaches published so far. Additionally, we show that OK has empirically negligible noise: Unlike previous algorithms it matches TBPTT in a real world task (character-level Penn TreeBank) and can exploit online parameter updates to outperform TBPTT in a synthetic string memorization task. Code availiable on github.
Meishan Zhang, Yue Zhang, Guohong Fu. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019.
In past studies, the few quantitative approaches to discourse structure were mostly confined to the presentation of the frequency of discourse relations. However, quantitative approaches should take into account both hierarchical and relational layers in the discourse structure. This study considers these factors and addresses the issue of how discourse relations and discourse units are related. It draws upon the available corpora of discourse structure (rhetorical structure theory-discourse treebank (RST-DT)) from a new perspective. Since an RST tree can be converted into a syntactic dependency tree, the data extracted from the RST-DT can be useful for calculating the discourse distance in much the same way as syntactic dependency distance is calculated. Discourse distance is also applicable to measuring the depth of the human processing of discourse. Furthermore, the data derived from the RST-DT are also easily converted into network data. This study finds that discourse structure has its discourse distance minimum and each type of RST relations has its range of discourse distance. The frequency distribution of discourse data basically follows the power law on several levels, while a network approach reveals how discourse units are arranged spatially in regular patterns. The two methods are mutually complementary in revealing the interaction between discourse relations and discourse units in a comprehensive manner, as well as in revealing how people process and comprehend discourse dynamically. Accordingly, we propose merging the two methods so as to yield a computational model for assessing discourse complexity and comprehension.
Language models are a key technology in various tasks, such as, speech recognition and machine translation. They are usually used on texts covering various domains and as a result domain adaptation has been a long ongoing challenge in language model research. With the rising popularity of neural network based language models, many methods have been proposed in recent years. These methods can be separated into two categories: model based and feature based adaptation methods. Feature based domain adaptation has compared to model based domain adaptation the advantage that it does not require domain labels in the corpus. Most existing feature based adaptation methods are based on bias adaptation. We propose a novel feature based domain adaptation technique using hidden layer factorisation. This method is fundamentally different from existing methods because we use the domain features to calculate a linear combination of linear layers. These linear layers can capture domain specific information and information common to different domains. In the experiments, we compare our proposed method with existing adaptation methods. The compared adaptation techniques are based on two different ideas, that is, bias based adaptation and gating of hidden units. All language models in our comparison use state-of-the-art long short-term memory based recurrent neural networks. We demonstrate the effectiveness of the proposed method with perplexity results for the well-known Penn Treebank and speech recognition results for a corpus of TED talks.
The goal of this paper is to use all available Polish language data sets to seek the best possible performance in supervised sentiment analysis of short texts. We use text collections with labeled sentiment such as tweets, movie reviews and a sentiment treebank, in three comparison modes. In the first, we examine the performance of models trained and tested on the same text collection using standard cross-validation (in-domain). In the second we train models on all available data except the given test collection, which we use for testing (one vs rest cross-domain). In the third, we train a model on one data set and apply it to another one (one vs one cross-domain). We compare wide range of methods including machine learning on bag-of-words representation, bidirectional recurrent neural networks as well as the most recent pre-trained architectures ELMO and BERT. We formulate conclusions as to cross-domain and in-domain performance of each method. Unsurprisingly, BERT turned out to be a strong performer, especially in the cross-domain setting. What is surprising however, is solid performance of the relatively simple multinomial Naive Bayes classifier, which performed equally well as BERT on several data sets.
Abstract To objectively and quantitatively study transcribed protocols of design problem solving conversations, we propose a semantic analysis approach based on dynamic semantic networks of nouns constructed with WordNet 3.1 lexical database. We examined the applicability of the semantic approach focused on a dynamic evaluation of the design problem solving process in educational settings. Using a case of real- world design problem-solving conversations, we show that the approach is able to determine the time dynamics of semantic factors such as level of abstraction, polysemy or information content, and quantify convergence/divergence of semantic similarity in design conversations between students, instructors and real clients. The approach can also be used to evaluate the aforementioned semantic factors for successful and unsuccessful ideas generated in the process of design problem solving, or to assess the effect of external feedback on the developed design solution. The proposed semantic analysis approach allows fast computation of the semantic factors in real time thereby demavonstrating a potential for both monitoring and support of the design problem solving process.
This article contends that researchers can and should be active participants in making sound archives more accessible. In fact, such advocacy can be consequential in setting up possibilities for creative research on race within radio history and sound studies. Using the example of NPR’s All Things Considered archive spanning from 1971–1983, I demonstrate how academics and archivists can work together to make possible the preservation and accessibility of sound archives. This particular collaboration offers an opportunity to take a cultural approach to understanding newsroom diversity, more specifically: the cultural constraints of linguistic norms and the emergent cultures that arise as aberrations from such norms. The article reflects on this project’s implications for other scholars who work with archives that wish to invest in sound archive preservation and use.
This paper describes Stanford's system at the CoNLL 2018 UD Shared Task. We introduce a complete neural pipeline system that takes raw text as input, and performs all tasks required by the shared task, ranging from tokenization and sentence segmentation, to POS tagging and dependency parsing. Our single system submission achieved very competitive performance on big treebanks. Moreover, after fixing an unfortunate bug, our corrected system would have placed the 2nd, 1st, and 3rd on the official evaluation metrics LAS,MLAS, and BLEX, and would have outperformed all submission systems on low-resource treebank categories on all metrics by a large margin. We further show the effectiveness of different model components through extensive ablation studies.
The slowness of legal proceedings in the common law legal system is a widely known fact. Any tool which could help reduce the time taken for the resolution of a case is invaluable. Common legal systems place a great importance on precedents and retrieving the correct set of precedents is considerably time consuming. Hence, for any case whose proceedings are in progress, if there are suitable prior cases, then the court has to follow the same interpretations that were passed in the prior cases. This is to ensure that similar situations receive similar treatment, thus maintaining uniformity amongst the legal proceedings across all courts at all times. Hence, precedent cases are treated as important as any other written law (a statute) in this legal system. In this paper, we propose two new approaches to solve this information retrieval problem wherein the system accepts the current case document as the query and returns the relevant precedent cases as the result. The first approach is to calculate the document similarity using Wordnet, which is a lexical database that could be leveraged to quantify the semantic relatedness between two documents, using a semantic network. The second approach is the use of a Siamese Manhattan Long Short Term Memory network, which is a supervised model trained to understand the underlying similarity between two documents.