Papers reviewed and determined not to be word norm studies. Use the flag icon to report errors or suggest re-inclusion.
16504 papers
Proof theory began in the 1920's as a part of Hilbert's program, which aimed to secure the foundations of mathematics by modeling infinitary mathematics with formal axiomatic systems and proving those systems consistent using restricted, finitary means. The program thus viewed mathematics as a system of reasoning with precise linguistic norms, governed by rules that can be described and studied in concrete terms. Today such a viewpoint has applications in mathematics, computer science, and the philosophy of mathematics.
Discourse parsing has long been treated as a stand-alone problem independent from constituency or dependency parsing. Most attempts at this problem are pipelined rather than end-to-end, sophisticated, and not self-contained: they assume goldstandard text segmentations (Elementary Discourse Units), and use external parsers for syntactic features.
In this chapter we describe the web application PaQu (Parse and Query), and carry out a small case study to illustrate its use. PaQu is an application for searching in Dutch treebanks and for analysing the search results. One can search in the LASSY and CGN treebanks, or upload one’s own Dutch corpus, which is then parsed and made available for search and analysis. PaQu offers, next to an interface to formulate Xpath queries, a dedicated interface for searching for dependency triples. This makes it easy to search in treebanks for grammatical dependencies, which would otherwise require very complex queries. It offers extensive functionality for analysing the search results. The dedicated search interface makes PaQu a prime example of the kind of applications that CLARIN promotes. The case study provides an analysis of the syntactic selectional differences between two near-synonymous verbs.
Abstract In this article we present a novel linguistically driven evaluation method and apply it to the main approaches of Machine Translation (Rule-based, Phrase-based, Neural) to gain insights into their strengths and weaknesses in much more detail than provided by current evaluation schemes. Translating between two languages requires substantial modelling of knowledge about the two languages, about translation, and about the world. Using English-German IT-domain translation as a case-study, we also enhance the Phrase-based system by exploiting parallel treebanks for syntax-aware phrase extraction and by interfacing with Linked Open Data (LOD) for extracting named entity translations in a post decoding framework.
Unsupervised dependency parsing, which tries to discover linguistic dependency structures from unannotated data, is a very challenging task. Almost all previous work on this task focuses on learning generative models. In this paper, we develop an unsupervised dependency parsing model based on the CRF autoencoder. The encoder part of our model is discriminative and globally normalized which allows us to use rich features as well as universal linguistic priors. We propose an exact algorithm for parsing as well as a tractable learning algorithm. We evaluated the performance of our model on eight multilingual treebanks and found that our model achieved comparable performance with state-of-the-art approaches.
We introduce a tree-structured attention neural network for sentences and small phrases and apply it to the problem of sentiment classification. Our model expands the current recursive models by incorporating structural information around a node of a syntactic tree using both bottomup and top-down information propagation. Also, the model utilizes structural attention to identify the most salient representations during the construction of the syntactic tree. To our knowledge, the proposed models achieve state of the art performance on the Stanford Sentiment Treebank dataset.
International audience
From an evolutionary perspective, environmental threats relevant for survival constantly challenged human beings. Current research suggests the evolution of a fear processing module in the brain to cope with these threats. Recently, humans increasingly encountered modern threats (e.g., guns or car accidents) in addition to evolutionary threats (e.g., snakes or predators) which presumably required an adaptation of perception and behavior. However, the neural processes underlying the perception of these different threats remain to be elucidated. We investigated the effect of image content (i.e., evolutionary vs. modern threats) on the activation of neural networks of emotion processing. During functional magnetic resonance imaging (fMRI) 41 participants watched affective pictures displaying evolutionary-threatening, modern-threatening, evolutionary-neutral and modern-neutral content. Evolutionary-threatening stimuli evoked stronger activations than modern-threatening stimuli in left inferior frontal gyrus and thalamus, right middle frontal gyrus and parietal regions as well as bilaterally in parietal regions, fusiform gyrus and bilateral amygdala. We observed the opposite effect, i.e., higher activity for modern-threatening than for evolutionary-threatening stimuli, bilaterally in the posterior cingulate and the parahippocampal gyrus. We found no differences in subjective arousal ratings between the two threatening conditions. On the valence scale though, subjects rated modern-threatening pictures significantly more negative than evolutionary-threatening pictures, indicating a higher level of perceived threat. The majority of previous studies show a positive relationship between arousal rating and amygdala activity. However, comparing fMRI results with behavioral findings we provide evidence that neural activity in fear processing areas is not only driven by arousal or valence, but presumably also by the evolutionary content of the stimulus. This has also fundamental methodological implications, in the sense to suggest a more elaborate classification of stimulus content to improve the validity of experimental designs.
This paper describes the IIT Kharagpur dependency parsing system in CoNLL-2017 shared task on Multilingual Parsing from Raw Text to Universal Dependencies. We primarily focus on the lowresource languages (surprise languages). We have developed a framework to combine multiple treebanks to train parsers for low resource languages by a delexicalization method. We have applied transformation on the source language treebanks based on syntactic features of the low-resource language to improve performance of the parser. In the official evaluation, our system achieves macro-averaged LAS scores of 67.61 and 37.16 on the entire blind test data and the surprise language test data respectively.
<strong>Berkeley parser model for Korean: Sejong treebank</strong> Jungyeul Park, Jeen-Pyo Hong, and Jeong-Won Cha (2016). Korean Language Resources for Everyone. In Proceedings of the 30th Pacific Asia Conference on Language, Information and Computation (PACLIC 30). Seoul, Korea. [pdf] @inproceedings{park-hong-cha:2016:PACLIC, <br> address = {Seoul, Korea}, <br> author = {Park, Jungyeul and Hong, Jeen-Pyo and Cha, Jeong-Won}, <br> booktitle = {Proceedings of the 30th Pacific Asia Conference on Language, Information and Computation (PACLIC 30)}, <br> pages = {49--58}, <br> title = {{Korean Language Resources for Everyone}}, <br> year = {2016} <br> } It requires Espresso's POS tagging results for input. Espresso is available at https://zenodo.org/record/884606
In this paper, we propose efficient and less resource-intensive strategies for parsing of code-mixed data. These strategies are not constrained by in-domain annotations, rather they leverage pre-existing monolingual annotated resources for training. We show that these methods can produce significantly better results as compared to an informed baseline. Besides, we also present a data set of 450 Hindi and English code-mixed tweets of Hindi multilingual speakers for evaluation. The data set is manually annotated with Universal Dependencies.
A set of Python scripts that convert function-head style encodings in dependency treebanks in a content-head style encoding (as used in the UD treebanks) and vice versa (for adpositions, copula and coordination). For more information, see (Rehbein, Steen, Do & Frank 2017).
We describe our submission to the CoNLL 2017 shared task, which exploits the shared common knowledge of a language across different domains via a domain adaptation technique. Our approach is an extension to the recently proposed adversarial training technique for domain adaptation, which we apply on top of a graph-based neural dependency parsing model on bidirectional LSTMs. In our experiments, we find our baseline graphbased parser already outperforms the official baseline model (UDPipe) by a large margin. Further, by applying our technique to the treebanks of the same language with different domains, we observe an additional gain in the performance, in particular for the domains with less training data.
Despite the global spread of English, it seems that voices from nonnative English teachers concerning English as an international language (EIL) are under-represented. To address the issue, this study sought to investigate the nonnative teachers’ perceptions of idealized native-speaker linguistic and pragmatic norms in the EIL context. Participants included 125 nonnative English-speaking teachers from the Persian context, falling within the expanding circle. Questionnaires and interviews were used to explore the teachers’ perceptions of native-speaker norms. Findings showed that the nonnative teachers gave preference to native-speaker linguistic norms despite the emerging nonnative EIL norms. Although most of the teachers accepted the existence of a number of accents in English, they preferred the standard American or British accent for language education. As to EIL pragmatic norms, the teachers argued that some degree of flexibility is acceptable with regard to the use of L1 pragmatic norms in the EIL context. The EFL teachers in the present study maintained that the transfer of L1 pragmatic norms to the nativized English makes English a legitimate and culturally appropriate variety in communication between nonnative speakers. The findings contribute to the reappraisal of ELT practices and the premises underpinning teaching EIL.
Norms are essential to the human condition. Whether in the guise of tradition, culture, canon or rules, norms are therefore central to studies in the humanities. This book focuses on Russian language culture of the post-revolutionary and post-Soviet periods, times when norms — linguistic and otherwise — have been eagerly debated, challenged, broken and redefined. Exploring the intersections between linguistic authority and creative response, an international team of scholars examines different realms of linguistic practice (literary fiction, internet slang, literary criticism and aesthetics, writers’ blogs, linguistic play) and various arenas for “talk about talk” (the classroom, blogs, the media, or the courtroom). By combining various approaches and disciplines — linguistics, literary criticism, new media studies — the book as a whole explores the multiplicity of meanings that are accorded to the notion of linguistic norms in the Russian community. The result is both a broad and a detailed picture of important trends in modern Russian language culture.
Translanguaging is a rapidly developing concept in bilingual education. Working from the theoretical background of dynamic bilingualism, a translanguaging lens posits that bilingual learners draw on a holistic linguistic repertoire to make sense of the world and to communicate effectively with texts. What is relatively underdeveloped is the pedagogical aspects of translanguaging. This classroom-based study conducted in the southeastern US asks 2 questions: (a) How might teachers create a translanguaging space for students, and (b) what would this space look like? The authors, 1 classroom teacher and 1 researcher, engaged emergent bilingual students in small group reading of a culturally relevant text and observed students’ active participation through strategic and fluid translanguaging practices. The authors argue that the linguistic norms of schooling should reflect the discursive norms of emergent bilingual students, and that teachers create translanguaging spaces as a path to educational equity
Theoretical models and recent advances in the treatment of anorexia nervosa (AN) have increasingly focused on the role of alterations in the processing and regulation of emotions. To date, however, our understanding of these changes is still limited and reports of emotional dysregulation in AN have been based largely on self-report data, and there is a relative lack of objective experimental evidence or neurobiological data. The current functional magnetic resonance imaging (fMRI) study investigated the hemodynamic correlates of passive viewing and voluntary downregulation of negative emotions by means of the reappraisal strategy detachment in AN patients. Detachment is regarded as adaptive regulation strategy associated with a reduction in emotion-related amygdala activity and increased recruitment of prefrontal brain regions associated with cognitive control processes. Emotion regulation efficacy was assessed via behavioral arousal ratings and fMRI activation elicited by an established experimental paradigm including negative images. Participants were instructed to either simply view emotional pictures or detach themselves from feelings triggered by the stimuli. The sample consisted of 36 predominantly adolescent female AN patients and a pairwise age-matched healthy control group. Behavioral and neuroimaging data analyses indicated a reduction of arousal and amygdala activity during the regulation condition for both patients and controls. However, compared with controls, individuals with AN showed increased activation in the amygdala as well as in the right dorsolateral prefrontal cortex (dlPFC) during the passive viewing of aversive compared with neutral pictures. These results extend previous findings indicative of altered processing of salient emotional stimuli in AN, but do not point to a general deficit in the voluntary regulation of negative emotions. Increased dlPFC activation in AN during passive viewing of negative stimuli is in line with the hypothesis that the disorder may be characterized by excessive self-control. Taken together, the data seem to suggest that reappraisal via detachment may be an effective strategy to reduce negative arousal for individuals with AN.
Interpreting is generally recognized as a particularly demanding language processing task for the cognitive system. Dependency distance, the linear distance between two syntactically related words in a sentence, is an index of sentence complexity and is also able to reflect the cognitive constraints during various tasks. In the current research, we examine the difference in dependency distance among three interpreting types, namely, simultaneous interpreting, consecutive interpreting and read-out translated speech based on a treebank comprising these types of interpreting output texts with dependency annotation. Results show that different interpreting renditions yield different dependency distances, and consecutive interpreting texts entail the smallest dependency distance other than those of simultaneous interpreting and read-out translated speech, suggesting that consecutive interpreting bears heavier cognitive demands than simultaneous interpreting. The current research suggests for the first time that interpreting is an extremely demanding cognitive task that can further mediate the dependency distance of output sentences. Such findings may be due to the minimization of dependency distance under cognitive constraints.
The Universal Dependencies (UD) project aims to develop a consistent annotation framework for treebanks across many languages. In this paper we present the UD scheme for Afrikaans and we describe the conversion of the AfriBooms treebank to this new format. We will compare the conversion to UD to the conversion of related syntactic structures in typologically similar languages.
Natural Language Processing (NLP) systems conventionally treat words as distinct atomic symbols. The model can leverage small amounts of information regarding the relationship between the individual symbols. Today when it comes to texts; one common technique to extract fixed-length features is bag-of-words. Despite its popularity the bag-of-words feature has two major weaknesses: it ignores semantics of the words and the order of words. In this paper, we propose a neural language model that relies on Convolutional Neural Network (CNN) and Bidirectional Recurrent Neural Network (BRNN) over pre-trained word vectors. We utilize bidirectional layers as a substitute of pooling layers in CNN in order to reduce the loss of detailed local information, and to capture long-term dependencies across input sequences. We validate the proposed model on two benchmark sentiment analysis datasets, Stanford Large Movie Review (IMDB), and Stanford Sentiment Treebank (SSTb). Our model achieves a competitive advantage compared with neural language models on the sentiment analysis datasets.
This natural language processing toolkit provides language-agnostic 'tokenization', 'parts of speech tagging', 'lemmatization' and 'dependency parsing' of raw text. Next to text parsing, the package also allows you to train annotation models based on data of 'treebanks' in 'CoNLL-U' format as provided at <<a href="https://universaldependencies.org/format.html" target="_top">https://universaldependencies.org/format.html</a>>. The techniques are explained in detail in the paper: 'Tokenizing, POS Tagging, Lemmatizing and Parsing UD 2.0 with UDPipe', available at <<a href="https://doi.org/10.18653%2Fv1%2FK17-3009" target="_top">doi:10.18653/v1/K17-3009</a>>. The toolkit also contains functionalities for commonly used data manipulations on texts which are enriched with the output of the parser. Namely functionalities and algorithms for collocations, token co-occurrence, document term matrix handling, term frequency inverse document frequency calculations, information retrieval metrics (Okapi BM25), handling of multi-word expressions, keyword detection (Rapid Automatic Keyword Extraction, noun phrase extraction, syntactical patterns) sentiment scoring and semantic similarity analysis.
Ongoing innovations in recurrent neural network architectures have provided a steady influx of apparently state-of-the-art results on language modelling benchmarks. However, these have been evaluated using differing code bases and limited computational resources, which represent uncontrolled sources of experimental variation. We reevaluate several popular architectures and regularisation methods with large-scale automatic black-box hyperparameter tuning and arrive at the somewhat surprising conclusion that standard LSTM architectures, when properly regularised, outperform more recent models. We establish a new state of the art on the Penn Treebank and Wikitext-2 corpora, as well as strong baselines on the Hutter Prize dataset.
Predicting valence-arousal ratings for words and phrases is very useful for constructing affective resources for dimensional sentiment analysis. Since the existing valence-arousal resources of Chinese are mainly in word-level and there is a lack of phrase-level ones, the Dimensional Sentiment Analysis for Chinese Phrases (DSAP) task aims to predict the valence-arousal ratings for Chinese affective words and phrases automatically. In this task, we propose an approach using a densely connected LSTM network and word features to identify dimensional sentiment on valence and arousal for words and phrases jointly. We use word embedding as major feature and choose part of speech (POS) and word clusters as additional features to train the dense LSTM network. The evaluation results of our submissions (1st and 2nd in average performance) validate the effectiveness of our system to predict valence and arousal dimensions for Chinese words and phrases.
We describe our entry, C2L2, to the CoNLL 2017 shared task on parsing Universal Dependencies from raw text. Our system features an ensemble of three global parsing paradigms, one graph-based and two transition-based. Each model leverages character-level bidirectional LSTMs as lexical feature extractors to encode morphological information. Though relying on baseline tokenizers and focusing only on parsing, our system ranked second in the official end-toend evaluation with a macro-average of 75.00 LAS F1 score over 81 test treebanks. In addition, we had the top average performance on the four surprise languages and on the small treebank subset.
Singlish can be interesting to the ACL community both linguistically as a major creole based on English, and computationally for information extraction and sentiment analysis of regional social media. We investigate dependency parsing of Singlish by constructing a dependency treebank under the Universal Dependencies scheme, and then training a neural network model by integrating English syntactic knowledge into a state-of-the-art parser trained on the Singlish treebank. Results show that English knowledge can lead to 25% relative error reduction, resulting in a parser of 84.47% accuracies. To the best of our knowledge, we are the first to use neural stacking to improve cross-lingual dependency parsing on low-resource languages. We make both our annotation and parser available for further research.
Important advances have recently been made using computational semantic models to decode brain activity patterns associated with concepts; however, this work has almost exclusively focused on concrete nouns. How well these models extend to decoding abstract nouns is largely unknown. We address this question by applying state-of-the-art computational models to decode functional Magnetic Resonance Imaging (fMRI) activity patterns, elicited by participants reading and imagining a diverse set of both concrete and abstract nouns. One of the models we use is linguistic, exploiting the recent word2vec skipgram approach trained on Wikipedia. The second is visually grounded, using deep convolutional neural networks trained on Google Images. Dual coding theory considers concrete concepts to be encoded in the brain both linguistically and visually, and abstract concepts only linguistically. Splitting the fMRI data according to human concreteness ratings, we indeed observe that both models significantly decode the most concrete nouns; however, accuracy is significantly greater using the text-based models for the most abstract nouns. More generally this confirms that current computational models are sufficiently advanced to assist in investigating the representational structure of abstract concepts in the brain.
We describe the process of creating NUDAR, a Universal Dependency treebank for Arabic. We present the conversion from the Penn Arabic Treebank to the Universal Dependency syntactic representation through an intermediate dependency representation. We discuss the challenges faced in the conversion of the trees, the decisions we made to solve them, and the validation of our conversion. We also present initial parsing results on NUDAR.
In this paper, we attempt a comparison between new school transition-based parsers that use neural networks and their classical old school coun-terpart. We carry out experiments on treebanks fr...
Svetlana Toldova, Dina Pisarevskaya, Margarita Ananyeva, Maria Kobozeva, Alexander Nasedkin, Sofia Nikiforova, Irina Pavlova, Alexey Shelepov. Proceedings of the 6th Workshop on Recent Advances in RST and Related Formalisms. 2017.
International audience
Accurate sentiment analysis models encode the sentiment of words and their combinations to predict the overall sentiment of a sentence. This task becomes challenging when applied to morphologically rich languages (MRL). In this article, we evaluate the use of deep learning advances, namely the Recursive Neural Tensor Networks (RNTN), for sentiment analysis in Arabic as a case study of MRLs. While Arabic may not be considered the only representative of all MRLs, the challenges faced and proposed solutions in Arabic are common to many other MRLs. We identify, illustrate, and address MRL-related challenges and show how RNTN is affected by the morphological richness and orthographic ambiguity of the Arabic language. To address the challenges with sentiment extraction from text in MRL, we propose to explore different orthographic features as well as different morphological features at multiple levels of abstraction ranging from raw words to roots. A key requirement for RNTN is the availability of a sentiment treebank; a collection of syntactic parse trees annotated for sentiment at all levels of constituency and that currently only exists in English. Therefore, our contribution also includes the creation of the first Arabic Sentiment Treebank (A r S en TB) that is morphologically and orthographically enriched. Experimental results show that, compared to the basic RNTN proposed for English, our solution achieves significant improvements up to 8% absolute at the phrase level and 10.8% absolute at the sentence level, measured by average F1 score. It also outperforms well-known classifiers including Support Vector Machines, Recursive Auto Encoders, and Long Short-Term Memory by 7.6%, 3.2%, and 1.6% absolute respectively, all models being trained with similar morphological considerations.
OBJECTIVE: Intrusive negative affect and concurrent deficits in positive affect are hallmarks of posttraumatic stress disorder (PTSD). We sought to further extend the extant literature by exploring the experience of negative affect intrusion upon potentially positive situations (here termed, "negative affect interference," NAI). METHOD: Two studies with adults endorsing at least 1 traumatic event (Study 1, N = 294; Study 2, N = 286) examined how NAI and more general hedonic deficits (HD) relate to psychopathology, trauma exposure characteristics, and ratings of normed visual stimuli. RESULTS: Study 1 found that NAI and HD were positively correlated with PTSD symptoms and childhood trauma, and NAI incremented over depressive symptoms in predicting PTSD severity. Study 2 results indicated additional strong positive correlations between NAI and HD and anhedonia, affect regulation problems, negative affect, and neuroticism. NAI and HD were found to increment over trait NA in predicting PTSD symptoms. Individuals endorsing elevated NAI and HD rated positively valenced pictures (including food and erotic images) as less arousing, although not more negative. CONCLUSIONS: These findings expand conceptualizations of anhedonia and emotional numbing by drawing attention to negative affect in otherwise positive contexts. (PsycINFO Database Record
The literary genre of poetry is inherently related to the expression and elicitation of emotion via both content and form. To explore the nature of this affective impact at an extremely basic textual level, we collected ratings on eight different general affective meaning scales—valence, arousal, friendliness, sadness, spitefulness, poeticity, onomatopoeia, and liking—for 57 German poems (“die verteidigung der wölfe”) which the contemporary author H. M. Enzensberger had labeled as either “friendly”, “sad”, or “spiteful”. Following Jakobson’s (1960) view on the vivid interplay of hierarchical text levels, we used multiple regression analyses to explore the specific influences of affective features from three different text levels (sublexical, lexical, and inter-lexical) on the perceived general affective meaning of the poems using three types of predictors: 1) Lexical predictor variables capturing the mean valence and arousal potential of words; 2) Inter-lexical predictors quantifying peaks, ranges and dynamic changes within the lexical affective content; 3) Sublexical measures of basic affective tone according to sound-meaning correspondences at the sublexical level (see Aryani, Kraxenberger, Ullrich, Jacobs, & Conrad, 2016). We find the lexical predictors to account for a major amount of up to 50 % of the variance in affective ratings. Moreover, inter-lexical and sublexical predictors account for a large portion of additional variance in the perceived general affective meaning. Together, the affective properties of all used textual features account for 43 to 70 % of the variance in the affective ratings and still for 23 to 48 % of the variance in the more abstract aesthetic ratings. In sum, our approach represents a novel method that successfully relates a prominent part of variance in perceived general affective meaning in this corpus of German poems to quantitative estimates of affective properties of textual components at the sublexical, lexical, and inter-lexical level.
Emotional stimuli attract attention and lead to increased activity in the visual cortex. The present study investigated the impact of personal relevance on emotion processing by presenting emotional words within sentences that referred to participants' significant others or to unknown agents. In event-related potentials, personal relevance increased visual cortex activity within 100 ms after stimulus onset and the amplitudes of the Late Positive Complex (LPC). Moreover, personally relevant contexts gave rise to augmented pupillary responses and higher arousal ratings, suggesting a general boost of attention and arousal. Finally, personal relevance increased emotion-related ERP effects starting around 200 ms after word onset; effects for negative words compared to neutral words were prolonged in duration. Source localizations of these interactions revealed activations in prefrontal regions, in the visual cortex and in the fusiform gyrus. Taken together, these results demonstrate the high impact of personal relevance on reading in general and on emotion processing in particular.
Several theories conceptualise emotions along two main dimensions: valence (a continuum from negative to positive) and arousal (a continuum that varies from low to high). These dimensions are typically treated as independent in many neuroimaging experiments, yet recent behavioural findings suggest that they are actually interdependent. This result has impact on neuroimaging design, analysis and theoretical development. We were interested in determining the extent of this interdependence both behaviourally and neuroanatomically, as well as teasing apart any activation that is specific to each dimension. While we found extensive overlap in activation for each dimension in traditional emotion areas (bilateral insulae, orbitofrontal cortex, amygdalae), we also found activation specific to each dimension with characteristic relationships between modulations of these dimensions and BOLD signal change. Increases in arousal ratings were related to increased activations predominantly in voice-sensitive cortices after variance explained by valence had been removed. In contrast, emotions of extreme valence were related to increased activations in bilateral voice-sensitive cortices, hippocampi, anterior and midcingulum and medial orbito- and superior frontal regions after variance explained by arousal had been accounted for. Our results therefore do not support a complete segregation of brain structures underpinning the processing of affective dimensions.
We present methodology for using dynamic evaluation to improve neural sequence models. Models are adapted to recent history via a gradient descent based mechanism, causing them to assign higher probabilities to re-occurring sequential patterns. Dynamic evaluation outperforms existing adaptation approaches in our comparisons. Dynamic evaluation improves the state-of-the-art word-level perplexities on the Penn Treebank and WikiText-2 datasets to 51.1 and 44.3 respectively, and the state-of-the-art character-level cross-entropies on the text8 and Hutter Prize datasets to 1.19 bits/char and 1.08 bits/char respectively.
We formulate language modeling as a matrix factorization problem, and show that the expressiveness of Softmax-based models (including the majority of neural language models) is limited by a Softmax bottleneck. Given that natural language is highly context-dependent, this further implies that in practice Softmax with distributed word embeddings does not have enough capacity to model natural language. We propose a simple and effective method to address this issue, and improve the state-of-the-art perplexities on Penn Treebank and WikiText-2 to 47.69 and 40.68 respectively. The proposed method also excels on the large-scale 1B Word dataset, outperforming the baseline by over 5.6 points in perplexity.
Darwin (1872) postulated that emotional expressions contain universals that are retained across species. We recently showed that human rating responses were strongly affected by a listener's familiarity with vocalization types, whereas evidence for universal cross-taxa emotion recognition was limited. To disentangle the impact of evolutionarily retained mechanisms (phylogeny) and experience-driven cognitive processes (familiarity), we compared the temporal unfolding of event-related potentials (ERPs) in response to agonistic and affiliative vocalizations expressed by humans and three animal species. Using an auditory oddball novelty paradigm, ERPs were recorded in response to task-irrelevant novel sounds, comprising vocalizations varying in their degree of phylogenetic relationship and familiarity to humans. Vocalizations were recorded in affiliative and agonistic contexts. Offline, participants rated the vocalizations for valence, arousal, and familiarity. Correlation analyses revealed a significant correlation between a posteriorly distributed early negativity and arousal ratings. More specifically, a contextual category effect of this negativity was observed for human infant and chimpanzee vocalizations but absent for other species vocalizations. Further, a significant correlation between the later and more posteriorly P3a and P3b responses and familiarity ratings indicates a link between familiarity and attentional processing. A contextual category effect of the P3b was observed for the less familiar chimpanzee and tree shrew vocalizations. Taken together, these findings suggest that early negative ERP responses to agonistic and affiliative vocalizations may be influenced by evolutionary retained mechanisms, whereas the later orienting of attention (positive ERPs) may mainly be modulated by the prior experience.
To test how preexisting long-term memory influences visual STM, this study takes advantage of individual differences in participants' prior familiarity with Pokémon characters and uses an ERP component, the contralateral delay activity (CDA), to assess whether observers' prior stimulus familiarity affects STM consolidation and storage capacity. In two change detection experiments, consolidation speed, as indexed by CDA fractional area latency and/or early-window (500-800 msec) amplitude, was significantly associated with individual differences in Pokémon familiarity. In contrast, the number of remembered Pokémon stimuli, as indexed by Cowan's K and late-window (1500-2000 msec) CDA amplitude, was significantly associated with individual differences in Pokémon familiarity when STM consolidation was incomplete because of a short presentation of Pokémon stimuli (500 msec, Experiment 2), but not when STM consolidation was allowed to complete given sufficient encoding time (1000 msec, Experiment 1). Similar findings were obtained in between-group analyses when participants were separated into high-familiarity and low-familiarity groups based on their Pokémon familiarity ratings. Together, these results suggest that stimulus familiarity, as a proxy for the strength of preexisting long-term memory, primarily speeds up STM consolidation, which may subsequently lead to an increase in the number of remembered stimuli if consolidation is incomplete. These findings thus highlight the importance of research assessing how effects on representations (e.g., STM capacity) are in general related to (or even caused by) effects on processes (e.g., STM consolidation) in cognition.
In this contribution, I review Augustinus's (2015) dissertation on the syntax of verb clusters. While the main theoretical aspects of this book (within the HPSG framework) may not be of interest to everyone, the corpus methodology used and the GrETEL tool that was developed in the course of this project are impressive, The book contains previously unknown descriptive details about possible clustering verbs. Due to its interdisciplinary nature, there should be something of interest here for anyone who is interested in verb clusters.
The use of the senses of vision and audition as interactive means has dominated the field of Human-Computer Interaction (HCI) for decades, even though nature has provided us with many more senses for perceiving and interacting with the world around us. That said, it has become attractive for HCI researchers and designers to harness touch, taste, and smell in interactive tasks and experience design. In this paper, we present research and design insights gained throughout an interdisciplinary collaboration on a six-week multisensory display – Tate Sensorium – exhibited at the Tate Britain art gallery in London, UK. This is a unique and first time case study on how to design art experiences whilst considering all the senses (i.e., vision, sound, touch, smell, and taste), in particular touch, which we exploited by capitalizing on a novel haptic technology, namely, mid-air haptics. We first describe the overall set up of Tate Sensorium and then move on to describing in detail the design process of the mid-air haptic feedback and its integration with sound for the Full Stop painting by John Latham (1961). This was the first time that mid-air haptic technology was used in a museum context over a prolonged period of time and integrated with sound to enhance the experience of visual art. As part of an interdisciplinary team of curators, sensory designers, sound artists, we selected a total of three variations of the mid-air haptic experience (i.e., haptic patterns), which were alternated at dedicated times throughout the six-week exhibition. We collected questionnaire-based feedback from 2500 visitors and conducted 50 interviews to gain quantitative and qualitative insights on visitors’ experiences and emotional reactions. Whilst the questionnaire results are generally very positive with only a small variation of the visitors’ arousal ratings across the three tactile experiences designed for the Full Stop painting, the interview data shed light on the differences in the visitors’ subjective experiences. Our findings suggest multisensory designers and art curators can ensure a balance between surprising experiences versus the possibility of free exploration for visitors. In addition, participants expressed that experiencing art with the combination of mid-air haptic and sound was immersive and provided an up-lifting experience of touching without touch. We are convinced that the insights gained from this large-scale and real-world field exploration of multisensory experience design exploiting a new and emerging technology provide a solid starting point for the HCI community, creative industries, and art curators to think beyond conventional art experiences. Specifically, our work demonstrates how novel mid-air technology can make art more emotionally engaging and stimulating, especially abstract art that is often open to interpretation.