by Sentimenti Team | Aug 5, 2021 | Conferences, Market research, Scientific publications
Place of publication:
- Conference: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing
Title:
Personal Bias in Prediction of Emotions Elicited by Textual Opinions
Authors:
Jan Kocoń, Piotr Miłkowski, Damian Grimling, Marcin Gruza, Kamil Kanclerz, Przemysław Kazienko
Abstract:
Analysis of emotions elicited by opinions, comments, or articles commonly exploits annotated corpora, in which the labels assigned to documents average the views of all annotators, or represent a majority decision. The models trained on such data are effective at identifying the general views of the population. However, their usefulness for predicting the emotions evoked by the textual content in a particular individual is limited. In this paper, we present a study performed on a dataset containing 7,000 opinions, each annotated by about 50 people with two dimensions: valence, arousal, and with intensity of eight emotions from Plutchik’s model. Our study showed that individual responses often significantly differed from the mean. Therefore, we proposed a novel measure to estimate this effect – Personal Emotional Bias (PEB). We also developed a new BERT-based transformer architecture to predict emotions from an individual human perspective. We found PEB a major factor for improving the quality of personalized reasoning. Both the method and measure may boost the quality of content recommendation systems and personalized solutions that protect users from hate speech or unwanted content, which are highly subjective in nature.
Link: ResearchGate
by Sentimenti Team | Jul 22, 2021 | Scientific publications
Place of publication:
Behavior Research Methods
Title:
Emotion norms for 6000 Polish word meanings with a direct mapping to the Polish wordnet
Authors:
Jan Kocoń, Arkadiusz Janz, Piotr Miłkowski, Monika Riegel, Małgorzata Wierzba, Artur Marchewka, Agnieszka Czoska, Damian Grimling, Barbara Konat, Konrad Juszczyk, Katarzyna Klessa, Maciej Piasecki
Abstract:
Emotion lexicons are useful in research across various disciplines, but the availability of such resources remains limited for most languages. While existing emotion lexicons typically comprise words, it is a particular meaning of a word (rather than the word itself) that conveys emotion. To mitigate this issue, we present the Emotion Meanings dataset, a novel dataset of 6000 Polish word meanings. The word meanings are derived from the Polish wordnet (plWordNet), a large semantic network interlinking words by means of lexical and conceptual relations. The word meanings were manually rated for valence and arousal, along with a variety of basic emotion categories (anger, disgust, fear, sadness, anticipation, happiness, surprise, and trust). The annotations were found to be highly reliable, as demonstrated by the similarity between data collected in two independent samples: unsupervised ( n = 21,317) and supervised ( n = 561). Although we found the annotations to be relatively stable for female, male, younger, and older participants, we share both summary data and individual data to enable emotion research on different demographically specific subgroups. The word meanings are further accompanied by the relevant metadata, derived from open-source linguistic resources. Direct mapping to Princeton WordNet makes the dataset suitable for research on multiple languages. Altogether, this dataset provides a versatile resource that can be employed for emotion research in psychology, cognitive science, psycholinguistics, computational linguistics, and natural language processing.
Link: ResearchGate
by Agnieszka Czoska | Apr 9, 2020 | Sentimenti research
You’ve been in Sentimenti from the beginning. What was it like in 2016?
The business idea for the study of emotions in the text came from W3A.PL company from Poznan. After consultations with the environment of Poznań psychologists, cognitive scientists and linguists, a draft of the project for NCBiR (National Centre for Research and Development) was prepared and the search for subcontractors started. After estimating the market, it turned out that two units are capable of undertaking such advanced research work: LOBI IBD PAS and Language Technology Group of Wrocław University of Technology.
Once you got the grant, how did you start working?
As a research manager I was responsible for organizing the work of the team. It was important for me to combine the scientific teams of subcontractors and the business team into one team. The interface between business and science is not easy. In the Sentimenti team everyone – presidents, PhDs and MSc – speaks to each other by name, each person has the right to express their opinion and make decisions.
You are the research manager and scrum master of our team – how much did you have to learn to become one?
I learned the Scrum management methodology for R&D projects in the UK, where I worked in the Argument Analytics project conducted in cooperation with the University of Dundee and financed by Innovate UK, the British equivalent of NCBiR. I understood then that the key issue in the cooperation between science and business is good communication. A common team, preferably working in one place, frequent meetings and evaluation of results to check if this is really what we want – this is the heart of good projects. Many other R&D projects that I have observed did not achieve their goals precisely because of such a lack of communication.
How does the scrum method differ from your previous project experience?
I am a scientist and I have gained most of my experience in academic work and basic research. The transition to applied research was not easy, but I was given a lot by the British culture of openness, communication and respect – the values that are inscribed in Scrum and that we transfer to our team. The three pillars of Scrum are also important: transparency, inspection and adaptation. Transparency means that every person in the team – even new and unfamiliar with the subject – has access to all information (except, of course, confidential information). This helps a lot in overcoming crises, looking for a solution.
And what are inspection and adaptation?
An inspection is a frequent and short “review” meeting, during which we check what has already been completed, whether we do not have any obstacles that the project management should deal with, whether someone has too much or too little work. This helps to master the natural feature of research projects – unpredictability. When the results are different from we expected or when we get information from the business that a solution is not working – we can quickly adapt.
How do you see further development of Sentimenti?
In February, we have already finished our research work and moved on to development work, i.e. we use the collected knowledge and data in the work on Sentitol – our main tool for text analysis. Thanks to the fact that we use an iterative approach, we implement functionalities by adding them in subsequent versions of the product, and simultaneously – according to the Scrum methodology – we finish each Sprint (stage of work in Scrum) with a working product. At the moment, we have working software that recognizes eight emotions in texts in Polish, thanks to research on over 20 thousand people. This is already a solution that exceeds the scope of other solutions present on the market, and we are preparing two more versions.
In the next version of Sentimenti we will include a module using LSS (Lexical Syntactic Structures), i.e. elements of the language that affect the evaluation, e.g. good + no, + very, + a little. Then we will include a module that uses deep neural networks technology, or more precisely – BiLSTM (bidirectional long short-term memory neural networks), so that it can evaluate the emotions throughout the text immediately – and this is a unique solution on a Polish scale, but also worldwide. Our scientific publication about this module will be published soon.
Therefore, in the project we use fast prototyping, and in parallel to the work of the scientific team, the company implements any new solution for customers – because we have a great interest in our solutions. Thanks to this we have already achieved much better results (and faster) than we planned at the beginning.
