Papers reviewed and determined not to be word norm studies. Use the flag icon to report errors or suggest re-inclusion.
18625 papers
We describe our submission to the CoNLL 2017 shared task, which exploits the shared common knowledge of a language across different domains via a domain adaptation technique. Our approach is an extension to the recently proposed adversarial training technique for domain adaptation, which we apply on top of a graph-based neural dependency parsing model on bidirectional LSTMs. In our experiments, we find our baseline graphbased parser already outperforms the official baseline model (UDPipe) by a large margin. Further, by applying our technique to the treebanks of the same language with different domains, we observe an additional gain in the performance, in particular for the domains with less training data.
Despite the global spread of English, it seems that voices from nonnative English teachers concerning English as an international language (EIL) are under-represented. To address the issue, this study sought to investigate the nonnative teachers’ perceptions of idealized native-speaker linguistic and pragmatic norms in the EIL context. Participants included 125 nonnative English-speaking teachers from the Persian context, falling within the expanding circle. Questionnaires and interviews were used to explore the teachers’ perceptions of native-speaker norms. Findings showed that the nonnative teachers gave preference to native-speaker linguistic norms despite the emerging nonnative EIL norms. Although most of the teachers accepted the existence of a number of accents in English, they preferred the standard American or British accent for language education. As to EIL pragmatic norms, the teachers argued that some degree of flexibility is acceptable with regard to the use of L1 pragmatic norms in the EIL context. The EFL teachers in the present study maintained that the transfer of L1 pragmatic norms to the nativized English makes English a legitimate and culturally appropriate variety in communication between nonnative speakers. The findings contribute to the reappraisal of ELT practices and the premises underpinning teaching EIL.
Norms are essential to the human condition. Whether in the guise of tradition, culture, canon or rules, norms are therefore central to studies in the humanities. This book focuses on Russian language culture of the post-revolutionary and post-Soviet periods, times when norms — linguistic and otherwise — have been eagerly debated, challenged, broken and redefined. Exploring the intersections between linguistic authority and creative response, an international team of scholars examines different realms of linguistic practice (literary fiction, internet slang, literary criticism and aesthetics, writers’ blogs, linguistic play) and various arenas for “talk about talk” (the classroom, blogs, the media, or the courtroom). By combining various approaches and disciplines — linguistics, literary criticism, new media studies — the book as a whole explores the multiplicity of meanings that are accorded to the notion of linguistic norms in the Russian community. The result is both a broad and a detailed picture of important trends in modern Russian language culture.
Translanguaging is a rapidly developing concept in bilingual education. Working from the theoretical background of dynamic bilingualism, a translanguaging lens posits that bilingual learners draw on a holistic linguistic repertoire to make sense of the world and to communicate effectively with texts. What is relatively underdeveloped is the pedagogical aspects of translanguaging. This classroom-based study conducted in the southeastern US asks 2 questions: (a) How might teachers create a translanguaging space for students, and (b) what would this space look like? The authors, 1 classroom teacher and 1 researcher, engaged emergent bilingual students in small group reading of a culturally relevant text and observed students’ active participation through strategic and fluid translanguaging practices. The authors argue that the linguistic norms of schooling should reflect the discursive norms of emergent bilingual students, and that teachers create translanguaging spaces as a path to educational equity
Theoretical models and recent advances in the treatment of anorexia nervosa (AN) have increasingly focused on the role of alterations in the processing and regulation of emotions. To date, however, our understanding of these changes is still limited and reports of emotional dysregulation in AN have been based largely on self-report data, and there is a relative lack of objective experimental evidence or neurobiological data. The current functional magnetic resonance imaging (fMRI) study investigated the hemodynamic correlates of passive viewing and voluntary downregulation of negative emotions by means of the reappraisal strategy detachment in AN patients. Detachment is regarded as adaptive regulation strategy associated with a reduction in emotion-related amygdala activity and increased recruitment of prefrontal brain regions associated with cognitive control processes. Emotion regulation efficacy was assessed via behavioral arousal ratings and fMRI activation elicited by an established experimental paradigm including negative images. Participants were instructed to either simply view emotional pictures or detach themselves from feelings triggered by the stimuli. The sample consisted of 36 predominantly adolescent female AN patients and a pairwise age-matched healthy control group. Behavioral and neuroimaging data analyses indicated a reduction of arousal and amygdala activity during the regulation condition for both patients and controls. However, compared with controls, individuals with AN showed increased activation in the amygdala as well as in the right dorsolateral prefrontal cortex (dlPFC) during the passive viewing of aversive compared with neutral pictures. These results extend previous findings indicative of altered processing of salient emotional stimuli in AN, but do not point to a general deficit in the voluntary regulation of negative emotions. Increased dlPFC activation in AN during passive viewing of negative stimuli is in line with the hypothesis that the disorder may be characterized by excessive self-control. Taken together, the data seem to suggest that reappraisal via detachment may be an effective strategy to reduce negative arousal for individuals with AN.
General treebank analyses are graph structured, but parsers are typically restricted to tree structures for efficiency and modeling reasons. We propose a new representation and algorithm for a class of graph structures that is flexible enough to cover almost all treebank structures, while still admitting efficient learning and inference. In particular, we consider directed, acyclic, one-endpoint-crossing graph structures, which cover most long-distance dislocation, shared argumentation, and similar tree-violating linguistic phenomena. We describe how to convert phrase structure parses, including traces, to our new representation in a reversible manner. Our dynamic program uniquely decomposes structures, is sound and complete, and covers 97.3% of the Penn English Treebank. We also implement a proof-of-concept parser that recovers a range of null elements and trace types.
First paragraph: The Scots language has largely been excluded, historically, within Scottish institutional contexts (Jones 1995: 1-21). This phenomenon typically owes itself, in Bourdieuan terms, to the lack of 'social' and 'cultural capital' certain codes of the language have increasingly held since the eighteenth century onwards in much of Scottish society. The devaluation of the Scots language from this period has been exacerbated in particular by its growing marginalisation within the Scottish education system. Although learning Latin held prestige during the sixteenth and seventeenth centuries, Scots was generally the teaching medium in most Scottish classrooms (Williamson 1982a: 54-77). However the elocution movement during the latter half of the eighteenth century and the Education (Scotland) Act of 1872, both encouraged and eventually required that every child should be educated in English (Bailey 1987: 131-42). Scots became regarded as a 'lazy', parochial dialect of English and Scottish aspirations to reproduce the linguistic norms of 'polite' London helped to suppress the language further (Jones 1995: 2).
Most of the machine learning algorithms requires the input to be denoted as a fixed-length feature vector. In text classifications (bag-of-words) is a popular fixed-length features. Despite their simplicity, they are limited in many tasks; they ignore semantics of words and loss ordering of words. In this paper, we propose a simple and efficient neural language model for sentence-level classification task. Our model employs Recurrent Neural Network Language Model (RNN-LM). Particularly, Long Short-Term Memory (LSTM) over pre-trained word vectors obtained from unsupervised neural language model to capture semantics and syntactic information in a short sentence. We achieved outstanding empirical results on multiple benchmark datasets, IMDB Sentiment analysis dataset, and Stanford Sentiment Treebank (SSTb) dataset. The empirical results show that our model is comparable with neural methods and outperforms traditional methods in sentiment analysis task.
International audience
We describe a simple but effective method for cross-lingual syntactic transfer of dependency parsers, in the scenario where a large amount of translation data is not available. This method makes use of three steps: 1) a method for deriving cross-lingual word clusters, which can then be used in a multilingual parser; 2) a method for transferring lexical information from a target language to source language treebanks; 3) a method for integrating these steps with the density-driven annotation projection method of Rasooli and Collins (2015). Experiments show improvements over the state-of-the-art in several languages used in previous work, in a setting where the only source of translation data is the Bible, a considerably smaller corpus than the Europarl corpus used in previous work. Results using the Europarl corpus as a source of translation data show additional improvements over the results of Rasooli and Collins (2015). We conclude with results on 38 datasets from the Universal Dependencies corpora.
Truncated Backpropagation Through Time (truncated BPTT) is a widespread method for learning recurrent computational graphs. Truncated BPTT keeps the computational benefits of Backpropagation Through Time (BPTT) while relieving the need for a complete backtrack through the whole data sequence at every step. However, truncation favors short-term dependencies: the gradient estimate of truncated BPTT is biased, so that it does not benefit from the convergence guarantees from stochastic gradient theory. We introduce Anticipated Reweighted Truncated Backpropagation (ARTBP), an algorithm that keeps the computational benefits of truncated BPTT, while providing unbiasedness. ARTBP works by using variable truncation lengths together with carefully chosen compensation factors in the backpropagation equation. We check the viability of ARTBP on two tasks. First, a simple synthetic task where careful balancing of temporal dependencies at different scales is needed: truncated BPTT displays unreliable performance, and in worst case scenarios, divergence, while ARTBP converges reliably. Second, on Penn Treebank character-level language modelling, ARTBP slightly outperforms truncated BPTT.
Clinical empathy, which is defined as the ability to understand the patient’s experience and feelings from the patient’s perspective, is acknowledged to be an important aspect of quality healthcare. However, how work experience modulates the empathic responses and brain activation patterns in medical professions remains elusive. This fMRI study recruited one hundred female nurses, who varied the length of work experience, and examined how their neural response, functional connectivity, and subjective evaluations of valence and arousal to perceiving another individual in physical pain are modulated by the situational context in which they occur (i.e., in a hospital or at home). Participants with longer hospital terms evaluated pain as less negative in valence and arousal when occurring in a hospital context, but not in a home context. Physical pain perceived in a hospital compared to a home context produced stronger activity in the right temporoparietal junction. The reverse comparison resulted in an increased activity in the insula and anterior midcingulate cortex. Mediation analysis indicated that reduced personal accomplishment, a symptom of burnout, breaks down the mediation effect of the putamen on context-dependent valence ratings. Overall, the study demonstrates how situational contexts significantly influence individuals’ empathic processing, and that perceiving reward from patient care protects them from burnout.
We introduce an attention-based Bi-LSTM for Chinese implicit discourse relations and demonstrate that modeling argument pairs as a joint sequence can outperform word order-agnostic approaches. Our model benefits from a partial sampling scheme and is conceptually simple, yet achieves state-of-the-art performance on the Chinese Discourse Treebank. We also visualize its attention activity to illustrate the model's ability to selectively focus on the relevant parts of an input sequence.
Universal Dependencies is a project that seeks to develop cross-linguistically consistent treebank annotation for many languages, with the goal of facilitating multilingual parser development, cross-lingual learning, and parsing research from a language typology perspective. The annotation scheme is based on (universal) Stanford dependencies (de Marneffe et al., 2006, 2008, 2014), Google universal part-of-speech tags (Petrov et al., 2012), and the Interset interlingua for morphosyntactic tagsets (Zeman, 2008). This release contains the test data used in the CoNLL 2017 shared task on parsing Universal Dependencies. Due to the shared task the test data was held hidden and not released together with the training and development data of UD 2.0. Therefore this release complements the UD 2.0 release (http://hdl.handle.net/11234/1-1983) to a full release of UD treebanks. In addition, the present release contains 18 new parallel test sets and 4 test sets in surprise languages. The present release also includes the development data already released with UD 2.0. Unlike regular UD releases, this one uses the folder-file structure that was visible to the systems participating in the shared task.
Natural Language Processing (NLP) systems conventionally treat words as distinct atomic symbols. The model can leverage small amounts of information regarding the relationship between the individual symbols. Today when it comes to texts; one common technique to extract fixed-length features is bag-of-words. Despite its popularity the bag-of-words feature has two major weaknesses: it ignores semantics of the words and the order of words. In this paper, we propose a neural language model that relies on Convolutional Neural Network (CNN) and Bidirectional Recurrent Neural Network (BRNN) over pre-trained word vectors. We utilize bidirectional layers as a substitute of pooling layers in CNN in order to reduce the loss of detailed local information, and to capture long-term dependencies across input sequences. We validate the proposed model on two benchmark sentiment analysis datasets, Stanford Large Movie Review (IMDB), and Stanford Sentiment Treebank (SSTb). Our model achieves a competitive advantage compared with neural language models on the sentiment analysis datasets.
Generative neural models have recently achieved state-of-the-art results for constituency parsing. However, without a feasible search procedure, their use has so far been limited to reranking the output of external parsers in which decoding is more tractable. We describe an alternative to the conventional action-level beam search used for discriminative neural models that enables us to decode directly in these generative models. We then show that by improving our basic candidate selection strategy and using a coarse pruning function, we can improve accuracy while exploring significantly less of the search space. Applied to the model of Choe and Charniak (2016), our inference procedure obtains 92.56 F1 on section 23 of the Penn Treebank, surpassing prior state-of-the-art results for single-model systems.
BACKGROUND: Formulaic expressions, including idioms and other fixed expressions, comprise a significant proportion of discourse. Although much has been written about this topic, controversy remains about their psychological status. An important claim about formulaic expressions, that they are known to native speakers, has seldom been directly demonstrated. This study tested the hypothesis that formulaic expressions are known and stored as whole unit mental representations by performing three perceptual experiments. METHOD: Listeners transcribed two kinds of spectrally-degraded spoken sentences, half formulaic, and half novel, newly created expressions, matched for grammar and length. Two familiarity ratings, usage and exposure, were obtained from listeners for each expression. Text frequency data for the stimuli and their constituent words were obtained using a spoken corpus. RESULTS: Participants transcribed formulaic more successfully than literal utterances. Usage and familiarity ratings correlated with accuracy, but formulaic utterances with low ratings were also transcribed correctly. Phrase types differed significantly in text frequency, but word frequency counts did not differentiate the two kinds of expressions. DISCUSSION: These studies provide new converging evidence that formulaic expressions are encoded and processed as whole units, supporting a dual-process model of language processing, which assumes that grammatical and formulaic expressions are differentially processed.
This study examined to what extent children and adults differ in how they process negative emotions during reading, and how they rate their own and protagonists’ emotional states. Results show that both children’s and adults’ processing of target sentences was facilitated when they described negative emotions. Processing of spill-over sentences was facilitated for adults but inhibited for children, suggesting children needed additional time to process protagonists’ emotional states and integrate them into coherent mental representations. Children and adults were similar in their valence and arousal ratings as they rated protagonists’ emotional states as more negative and more intense than their own emotional states. However, they differed in that children rated their own emotional states as relatively neutral, whereas adults’ ratings of their own emotional states more closely matched the negative emotional states of the protagonists. This suggests a possible difference between children and adults in the mechanism underlying emotional inferencing.
Darwin (1872) postulated that emotional expressions contain universals that are retained across species. We recently showed that human rating responses were strongly affected by a listener's familiarity with vocalization types, whereas evidence for universal cross-taxa emotion recognition was limited. To disentangle the impact of evolutionarily retained mechanisms (phylogeny) and experience-driven cognitive processes (familiarity), we compared the temporal unfolding of event-related potentials (ERPs) in response to agonistic and affiliative vocalizations expressed by humans and three animal species. Using an auditory oddball novelty paradigm, ERPs were recorded in response to task-irrelevant novel sounds, comprising vocalizations varying in their degree of phylogenetic relationship and familiarity to humans. Vocalizations were recorded in affiliative and agonistic contexts. Offline, participants rated the vocalizations for valence, arousal, and familiarity. Correlation analyses revealed a significant correlation between a posteriorly distributed early negativity and arousal ratings. More specifically, a contextual category effect of this negativity was observed for human infant and chimpanzee vocalizations but absent for other species vocalizations. Further, a significant correlation between the later and more posteriorly P3a and P3b responses and familiarity ratings indicates a link between familiarity and attentional processing. A contextual category effect of the P3b was observed for the less familiar chimpanzee and tree shrew vocalizations. Taken together, these findings suggest that early negative ERP responses to agonistic and affiliative vocalizations may be influenced by evolutionary retained mechanisms, whereas the later orienting of attention (positive ERPs) may mainly be modulated by the prior experience.
To test how preexisting long-term memory influences visual STM, this study takes advantage of individual differences in participants' prior familiarity with Pokémon characters and uses an ERP component, the contralateral delay activity (CDA), to assess whether observers' prior stimulus familiarity affects STM consolidation and storage capacity. In two change detection experiments, consolidation speed, as indexed by CDA fractional area latency and/or early-window (500-800 msec) amplitude, was significantly associated with individual differences in Pokémon familiarity. In contrast, the number of remembered Pokémon stimuli, as indexed by Cowan's K and late-window (1500-2000 msec) CDA amplitude, was significantly associated with individual differences in Pokémon familiarity when STM consolidation was incomplete because of a short presentation of Pokémon stimuli (500 msec, Experiment 2), but not when STM consolidation was allowed to complete given sufficient encoding time (1000 msec, Experiment 1). Similar findings were obtained in between-group analyses when participants were separated into high-familiarity and low-familiarity groups based on their Pokémon familiarity ratings. Together, these results suggest that stimulus familiarity, as a proxy for the strength of preexisting long-term memory, primarily speeds up STM consolidation, which may subsequently lead to an increase in the number of remembered stimuli if consolidation is incomplete. These findings thus highlight the importance of research assessing how effects on representations (e.g., STM capacity) are in general related to (or even caused by) effects on processes (e.g., STM consolidation) in cognition.
In this contribution, I review Augustinus's (2015) dissertation on the syntax of verb clusters. While the main theoretical aspects of this book (within the HPSG framework) may not be of interest to everyone, the corpus methodology used and the GrETEL tool that was developed in the course of this project are impressive, The book contains previously unknown descriptive details about possible clustering verbs. Due to its interdisciplinary nature, there should be something of interest here for anyone who is interested in verb clusters.
The use of the senses of vision and audition as interactive means has dominated the field of Human-Computer Interaction (HCI) for decades, even though nature has provided us with many more senses for perceiving and interacting with the world around us. That said, it has become attractive for HCI researchers and designers to harness touch, taste, and smell in interactive tasks and experience design. In this paper, we present research and design insights gained throughout an interdisciplinary collaboration on a six-week multisensory display – Tate Sensorium – exhibited at the Tate Britain art gallery in London, UK. This is a unique and first time case study on how to design art experiences whilst considering all the senses (i.e., vision, sound, touch, smell, and taste), in particular touch, which we exploited by capitalizing on a novel haptic technology, namely, mid-air haptics. We first describe the overall set up of Tate Sensorium and then move on to describing in detail the design process of the mid-air haptic feedback and its integration with sound for the Full Stop painting by John Latham (1961). This was the first time that mid-air haptic technology was used in a museum context over a prolonged period of time and integrated with sound to enhance the experience of visual art. As part of an interdisciplinary team of curators, sensory designers, sound artists, we selected a total of three variations of the mid-air haptic experience (i.e., haptic patterns), which were alternated at dedicated times throughout the six-week exhibition. We collected questionnaire-based feedback from 2500 visitors and conducted 50 interviews to gain quantitative and qualitative insights on visitors’ experiences and emotional reactions. Whilst the questionnaire results are generally very positive with only a small variation of the visitors’ arousal ratings across the three tactile experiences designed for the Full Stop painting, the interview data shed light on the differences in the visitors’ subjective experiences. Our findings suggest multisensory designers and art curators can ensure a balance between surprising experiences versus the possibility of free exploration for visitors. In addition, participants expressed that experiencing art with the combination of mid-air haptic and sound was immersive and provided an up-lifting experience of touching without touch. We are convinced that the insights gained from this large-scale and real-world field exploration of multisensory experience design exploiting a new and emerging technology provide a solid starting point for the HCI community, creative industries, and art curators to think beyond conventional art experiences. Specifically, our work demonstrates how novel mid-air technology can make art more emotionally engaging and stimulating, especially abstract art that is often open to interpretation.
We argue that coherence relations (relations between propositions, such as Concession or Purpose) are signalled more frequently and by more means than is generally believed. We examine how coherence relations in text are indicated by all possible textual signals, and whether every relation is signalled. To that end, we conducted a corpus study on the RST Discourse Treebank, a corpus of newspaper articles annotated for rhetorical (or coherence) relations. Results from our corpus study show that most relations in text (over 90%) are signalled and also that most signalled relations (over 80%) are indicated not only by discourse markers (and, but, if, since), but also by a wide variety of signals other than discourse markers, such as reference, lexical, semantic, syntactic and graphical features. These findings suggest that signalling of coherence relations is much more sophisticated than previously thought.
This case study of a team in an international workplace investigates processes of language socialization in a transient multilingual setting. Using interview and observational data, the analysis shows how social and linguistic norms are negotiated, with the newcomer positioned as a catalyst for changing language practices toward more English, with the ultimate aim of creating a “global mindset” in the organization. Language socialization in a transient multilingual setting is shown to focus on and assign positive value to new linguistic norms that experienced members are socialized into in a process that hinges on new members functioning as tools for management to bring about the desired change. The article shows that while the newcomer is used as a catalyst for increased use of English and for the creation of a “global mindset,” she is at the same time socialized into the existing Danish egalitarian workplace culture. The analysis also highlights the vulnerable position of the catalyst. While her status as an expert English speaker with the right “global mindset” may confer power in an internationalizing workplace, the position of catalyst is also linked with feelings of exclusion.
Several recent studies have demonstrated that people intuitively make consistent matches between classical music and specific wines. It is not clear, however, what governs such crossmodal mappings. Here, we assess the role of emotion—specifically different dimensional aspects of valence, arousal, and dominance—in mediating such mappings. Participants matched three different red wines to three different pieces of classical music. Subsequently, they made emotion ratings separately for each wine and each musical selection. The results revealed that certain wine–music pairings were rated as being significantly better matches than others. More importantly, there was evidence that the participants’ dominance and arousal ratings for the wines and the music predicted their matching rating for each wine–music pairing. These results therefore support the view that wine–music associations are not arbitrary but can be explained, at least in part, by common emotional associations.
We present a new, efficient frame-semantic parser that labels semantic arguments to FrameNet predicates. Built using an extension to the segmental RNN that emphasizes recall, our basic system achieves competitive performance without any calls to a syntactic parser. We then introduce a method that uses phrase-syntactic annotations from the Penn Treebank during training only, through a multitask objective; no parsing is required at training or test time. This "syntactic scaffold" offers a cheaper alternative to traditional syntactic pipelining, and achieves state-of-the-art performance.
Unstructured text data produced on the internet grows rapidly, and sentiment analysis for short texts becomes a challenge because of the limit of the contextual information they usually contain. Learning good vector representations for sentences is a challenging task and an ongoing research area. Moreover, learning long-term dependencies with gradient descent is difficult in neural network language model because of the vanishing gradients problem. Natural Language Processing (NLP) systems traditionally treat words as discrete atomic symbols; the model can leverage small amounts of information regarding the relationship between the individual symbols. In this paper, we propose ConvLstm, neural network architecture that employs Convolutional Neural Network (CNN) and Long Short-Term Memory (LSTM) on top of pre-trained word vectors. In our experiments, ConvLstm exploit LSTM as a substitute of pooling layer in CNN to reduce the loss of detailed local information and capture long term dependencies in sequence of sentences. We validate the proposed model on two sentiment datasets IMDB, and Stanford Sentiment Treebank (SSTb). Empirical results show that ConvLstm achieved comparable performances with less parameters on sentiment analysis tasks.
This paper presents a novel neural machine translation model which jointly learns translation and source-side latent graph representations of sentences. Unlike existing pipelined approaches using syntactic parsers, our end-to-end model learns a latent graph parser as part of the encoder of an attention-based neural machine translation model, and thus the parser is optimized according to the translation objective. In experiments, we first show that our model compares favorably with state-of-the-art sequential and pipelined syntax-based NMT models. We also show that the performance of our model can be further improved by pretraining it with a small amount of treebank annotations. Our final ensemble model significantly outperforms the previous best models on the standard Englishto-Japanese translation dataset.
Word embeddings that can capture semantic and syntactic information from contexts have been extensively used for various natural language processing tasks. However, existing methods for learning contextbased word embeddings typically fail to capture sufficient sentiment information. This may result in words with similar vector representations having an opposite sentiment polarity (e.g., good and bad), thus degrading sentiment analysis performance. Therefore, this study proposes a word vector refinement model that can be applied to any pre-trained word vectors (e.g., Word2vec and GloVe). The refinement model is based on adjusting the vector representations of words such that they can be closer to both semantically and sentimentally similar words and further away from sentimentally dissimilar words. Experimental results show that the proposed method can improve conventional word embeddings and outperform previously proposed sentiment embeddings for both binary and fine-grained classification on Stanford Sentiment Treebank (SST).
Discourse parsing is an integral part of understanding information flow and argumentative structure in documents. Most previous research has focused on inducing and evaluating models from the English RST Discourse Treebank. However, discourse treebanks for other languages exist, including Spanish, German, Basque, Dutch and Brazilian Portuguese. The treebanks share the same underlying linguistic theory, but differ slightly in the way documents are annotated. In this paper, we present (a) a new discourse parser which is simpler, yet competitive (significantly better on 2/3 metrics) to state of the art for English, (b) a harmonization of discourse treebanks across languages, enabling us to present (c) what to the best of our knowledge are the first experiments on crosslingual discourse parsing.
Recent papers have shown that neural networks obtain state-of-the-art performance on several different sequence tagging tasks. One appealing property of such systems is their generality, as excellent performance can be achieved with a unified architecture and without task-specific feature engineering. However, it is unclear if such systems can be used for tasks without large amounts of training data. In this paper we explore the problem of transfer learning for neural sequence taggers, where a source task with plentiful annotations (e.g., POS tagging on Penn Treebank) is used to improve performance on a target task with fewer available annotations (e.g., POS tagging for microblogs). We examine the effects of transfer learning for deep hierarchical recurrent networks across domains, applications, and languages, and show that significant improvement can often be obtained. These improvements lead to improvements over the current state-of-the-art on several well-studied tasks.
The Web has accumulated a rich source of information, such as text, image, rating, etc, which represent different aspects of user preferences. However, the heterogeneous nature of this information makes it difficult for recommender systems to leverage in a unified framework to boost the performance. Recently, the rapid development of representation learning techniques provides an approach to this problem. By translating the various information sources into a unified representation space, it becomes possible to integrate heterogeneous information for informed recommendation.
Virtual reality (VR) has been proposed as a methodological tool to study the basic science of psychology and other fields. One key advantage of VR is that sharing of virtual content can lead to more robust replication and representative sampling. A database of standardized content will help fulfill this vision. There are two objectives to this study. First, we seek to establish and allow public access to a database of immersive VR video clips that can act as a potential resource for studies on emotion induction using virtual reality. Second, given the large sample size of participants needed to get reliable valence and arousal ratings for our video, we were able to explore the possible links between the head movements of the observer and the emotions he or she feels while viewing immersive VR. To accomplish our goals, we sourced for and tested 73 immersive VR clips which participants rated on valence and arousal dimensions using self-assessment manikins. We also tracked participants' rotational head movements as they watched the clips, allowing us to correlate head movements and affect. Based on past research, we predicted relationships between the standard deviation of head yaw and valence and arousal ratings. Results showed that the stimuli varied reasonably well along the dimensions of valence and arousal, with a slight underrepresentation of clips that are of negative valence and highly arousing. The standard deviation of yaw positively correlated with valence, while a significant positive relationship was found between head pitch and arousal. The immersive VR clips tested are available online as supplemental material.
We systematically explore regularizing neural networks by penalizing low\nentropy output distributions. We show that penalizing low entropy output\ndistributions, which has been shown to improve exploration in reinforcement\nlearning, acts as a strong regularizer in supervised learning. Furthermore, we\nconnect a maximum entropy based confidence penalty to label smoothing through\nthe direction of the KL divergence. We exhaustively evaluate the proposed\nconfidence penalty and label smoothing on 6 common benchmarks: image\nclassification (MNIST and Cifar-10), language modeling (Penn Treebank), machine\ntranslation (WMT'14 English-to-German), and speech recognition (TIMIT and WSJ).\nWe find that both label smoothing and the confidence penalty improve\nstate-of-the-art models across benchmarks without modifying existing\nhyperparameters, suggesting the wide applicability of these regularizers.\n
Large-scale distributed training requires significant communication bandwidth for gradient exchange that limits the scalability of multi-node training, and requires expensive high-bandwidth network infrastructure. The situation gets even worse with distributed training on mobile devices (federated learning), which suffers from higher latency, lower throughput, and intermittent poor connections. In this paper, we find 99.9% of the gradient exchange in distributed SGD is redundant, and propose Deep Gradient Compression (DGC) to greatly reduce the communication bandwidth. To preserve accuracy during compression, DGC employs four methods: momentum correction, local gradient clipping, momentum factor masking, and warm-up training. We have applied Deep Gradient Compression to image classification, speech recognition, and language modeling with multiple datasets including Cifar10, ImageNet, Penn Treebank, and Librispeech Corpus. On these scenarios, Deep Gradient Compression achieves a gradient compression ratio from 270x to 600x without losing accuracy, cutting the gradient size of ResNet-50 from 97MB to 0.35MB, and for DeepSpeech from 488MB to 0.74MB. Deep gradient compression enables large-scale distributed training on inexpensive commodity 1Gbps Ethernet and facilitates distributed training on mobile. Code is available at: https://github.com/synxlin/deep-gradient-compression.
With the advent of word embeddings, lexicons are no longer fully utilized for sentiment analysis although they still provide important features in the traditional setting. This paper introduces a novel approach to sentiment analysis that integrates lexicon embeddings and an attention mechanism into Convolutional Neural Networks. Our approach performs separate convolutions for word and lexicon embeddings and provides a global view of the document using attention. Our models are experimented on both the SemEval'16 Task 4 dataset and the Stanford Sentiment Treebank and show comparative or better results against the existing state-of-the-art systems. Our analysis shows that lexicon embeddings allow building high-performing models with much smaller word embeddings, and the attention mechanism effectively dims out noisy words for sentiment analysis.
Recurrent neural networks (RNNs), such as long short-term memory networks (LSTMs), serve as a fundamental building block for many sequence learning tasks, including machine translation, language modeling, and question answering. In this paper, we consider the specific problem of word-level language modeling and investigate strategies for regularizing and optimizing LSTM-based models. We propose the weight-dropped LSTM which uses DropConnect on hidden-to-hidden weights as a form of recurrent regularization. Further, we introduce NT-ASGD, a variant of the averaged stochastic gradient method, wherein the averaging trigger is determined using a non-monotonic condition as opposed to being tuned by the user. Using these and other regularization strategies, we achieve state-of-the-art word level perplexities on two data sets: 57.3 on Penn Treebank and 65.8 on WikiText-2. In exploring the effectiveness of a neural cache in conjunction with our proposed model, we achieve an even lower state-of-the-art perplexity of 52.8 on Penn Treebank and 52.0 on WikiText-2.
Because of the importance of the information conveyed by the clinical documents and owing to the large quantity of raw texts produced in the healthcare system, it became a determinant challenge, in the NLP research field, to arrange the extraction and the management of meaningful data, starting from real text occurrences. In this paper we approach a corpus of 5000 medical diagnoses with sophisticated linguistic and computational devices, which are able to access the semantic dimension of words and sentences contained in it. Our morphosemantic method is grounded on a list of neoclassical formative elements pertaining to the medical domain which has been used for the automatic creation and population of medical lexical resources. The outcomes of this work are automatically built electronic dictionaries and thesauri and an annotated corpus for the NLP in the medical domain.
Because of the importance of the information conveyed by the clinical documents and owing to the large quantity of raw texts produced in the healthcare system, it became a determinant challenge, in the NLP research field, to arrange the extraction and the management of meaningful data, starting from real text occurrences. In this paper we approach a corpus of 5000 medical diagnoses with sophisticated linguistic and computational devices, which are able to access the semantic dimension of words and sentences contained in it. Our morphosemantic method is grounded on a list of neoclassical formative elements pertaining to the medical domain which has been used for the automatic creation and population of medical lexical resources. The outcomes of this work are automatically built electronic dictionaries and thesauri and an annotated corpus for the NLP in the medical domain.
This paper presents Autobank, a prototype tool for constructing a widecoverage Minimalist Grammar (MG) The front end of the tool is a graphical user interface which facilitates the rapid development of a seed set of MG trees via manual reannotation of PTB preterminals with MG lexical categories. The system then extracts various dependency mappings between the source and target trees, and uses these in concert with a non-statistical MG parser to automatically reannotate the rest of the corpus. Autobank thus enables deep treebank conversions (and subsequent modifications) without the need for complex transduction algorithms accompanied by cascades of ad hoc rules; instead, the locus of human effort falls directly on the task of grammar construction itself.
For the data-driven approach in Natural Language Processing (NLP) applications, good quality linguistic resources considered as a main factor to obtain good results. Although Arabic language is one of the main languages in the world, it is considered as low-resourced language in term of good quality and free linguistic resources. This work presents the first stage of building a new open source dependency treebank for Arabic language. It describes the prototype of the new dependency treebank that are inspired by Lexical Functional Grammar (LFG). This paper shows a main approach of developing a newly treebank and put lines the future work needed to complete this novel linguistic resource.
The Universal Dependencies Project1\n(Nivre, [9]; Nivre et al., [10]) is an\nongoing effort towards creating a set of harmonised dependency treebanks\nthat are annotated and structured according to universal guidelines. This paper reports on the addition of morphological features to the Irish Universal\nDependencies Treebank (IUDT). Our feature set subscribes to the feature inventory of the UD Project and has been mapped from Irish morpho-syntactic\ntags – the output of a Finite State Morphological Analyser for Irish (Uí Dhonnchadha and van Genabith [16]). Irish, a Celtic language, has some relatively unusual morphological features that require language-specific labels\nnot covered by the universal feature set. In this paper, we summarise the\nIrish-specific features that we have added to this set by explaining the linguistic properties that they each describe. We also report on the first parsing\nexperiments using the IUDT by assessing the effect that the inclusion of morphological features has on parsing accuracy.