by Damian Grimling | Oct 24, 2023 | Politics and Social
The results of the October 15, 2023 elections have had a stimulating effect on the prices of many financial instruments. Sentimenti analysts surveyed with the proprietary tool SENTITOOL, the period from October 11 to 18, measuring the level of the investor confidence index, defined as the intensity of confidence emotions expressed in posts made on social media. The study was based on an analysis of 1,517 such posts relating to diversified financial instruments.
Changes in the trust index
The final days of the campaign and the ordered election silence in effect especially on Saturday, October 14 caused investors to take a distinctly wait-and-see position. This resulted in a drop in the daily confidence index by an average of 3 percentage points from the level recorded as recently as October 12. As expected, only the successively announced results on voter turnout and preliminary exit poll data caused a decisive change in the level of the confidence index, which reached an increase of 11 percentage points on October 15.
Such a change in sentiment has boosted financial markets. A prime example of such a revival was the WIG20 bourse, which had already risen 5.3% from its last quote on October 16. In the following days, too, a cause-and-effect relationship can be seen between the confidence index and the WIG20 bourse.
Confidence index on Election Day
There was an interesting development of the confidence index on election day itself,October 15. An hour-by-hour analysis shows what dynamic fluctuations it underwent. Thus, after the opening of polling stations, it had an upward trend, which was slowed down by the turnout data at 12:00 p.m. The subsequent data at 5:00 p.m., combined with exit poll data, resulted in a dynamic (by 11 percentage points) increase in the confidence index. The 9:00 pm hour brought preliminary exit poll results. This, combined with turnout data, further boosted the index by another 16 percentage points. The 23:00 hour brought a record high for the confidence index – a level of 71% was recorded. This data brought the average daily confidence index on October 15 to 32%. It was 11 percentage points higher compared to the previous day.
Whether the positive momentum generated by the election results will have a more lasting trend in the financial markets and the economy as a whole, time will tell. Perhaps it will depend, among other things, on the pace at which the election promises are implemented. Promises made by the coalition that will take over our country.
The analysis was carried out thanks to the materials provided, in cooperation with the IBIMS Institute for Internet and Social Media Research.
by Damian Grimling | Sep 2, 2023 | Sentimenti research
Recently, the so-called generative artificial intelligence (GenAI) has received great attention. It is capable of generating new data, images, text or other content based on patterns or input, to quote ChatGPT. Or put another way, it can create new data such as text, images, videos, and music. It relies on machine learning algorithms that allow it to analyze existing data and then generate new data that is similar to the original data, following Google Bard’s definition.
These language models are often treated by users as an “oracle that knows the answers to all questions.” The creators of these solutions allow their tools to describe and review almost all spheres of our lives, including our emotions, and users take advantage of this eagerly.
But are these descriptions, answers to questions and reviews fully reliable, especially in terms of emotions? Let’s check it out!
Testing the efficiency of generative AI
A team of scientists from the department of linguistic engineering at Wroclaw University of Technology, which also included scholars working on our solution “Sentimenti – an analyzer of emotions in text,” subjected ChatGPT and other available language models (including sentiment analysis systems) to a comparative study.
As the study by the Wroclaw team showed, GenAI performed the worst on pragmatic tasks requiring knowledge of the world and precisely on emotion assessment.
Our tests at Sentimenti
Measuring sentiment, emotion and emotional arousal is the essence of Sentimenti. That’s why we were tempted to take a simple test to check.
We analyzed four randomly selected texts using “Sentimenti analyzer”; ChatGPT 4; ChatGPT 3.5 and GOOGLE BARD for the content of emotions they emanate. As a unit of measurement of the quality of the performance of the indicated programs was adopted:
- MSE (mean squared error) – mean squared error as a coefficient of error evaluation (which is the smaller, the more favorable for the software under test) and
- R² – the quality coefficient of the model fit (the larger its value, the better the fit of the software under test).
The benchmark for checking the quality of the performance of the selected programs was the results of the emotive evaluation of the texts. There were selected for analysis carried out on respondents (people) participating during the development of our emotion analyzer. Since the emotion values presented by the ChatGPT 4, ChatGPT 3.5 and Google Bard programs are shown in two decimal places, we made analogous rounding in the “Sentimenti analyzer” and in the results of the survey conducted on respondents.
Randomly selected texts, subjected to GenAI analysis
Text 1
As I have stayed at this hotel many times, my review will not change much. Underneath the rather ugly and neglected exterior of the hotel, there is a nicely renovated interior and friendly staff. The rooms are clean, bright and quite spacious. Breakfasts are excellent. Unfortunately, the restaurant menu, as in all Novotels, is terrible, and the dishes are modest, unpalatable and very expensive. Fortunately, there are many places in Krakow where you can eat well. I definitely do not recommend the restaurant, but the hotel as much as possible. It would be worthwhile to think about renewing the body of the hotel, as it is from a different era. It was the same in my student days, 25 years ago!!!!
Text 2
The hotel in an ideal location as far as the boat is concerned, two minutes from Piotrowska in the center, but as far as cleanliness is a tragedy the bathroom stank and the bedding dirty, for this class of hotel it is unacceptable
Text 3
The hotel only for a one-night stay for business and nothing else. After a recent stay of several days with family, we decided that this would be our last visit to this hotel. Food standard for all Ibis, rooms also. Service at the reception – roulette. Upon checking in, we were given perhaps the smallest corner room in the entire hotel despite a prior phone reservation for specific rooms on specific floors, supposedly due to occupancy. By a strange coincidence, a few minutes later during service by another receptionist our rooms were found.” In the rooms not very clean, housekeeping does not pay attention to the types of water put despite numerous requests. The restaurant leaves much to be desired.”
Text 4
The hotel is located near the center of Wroclaw. Room and bathroom clean, toiletries ok, maybe a little too soft bed, but it was not bad. Breakfast quite good. Parking additionally paid 25 zl. On a very big plus I wanted to evaluate the service. At the reception everything efficiently and with a smile. However, during breakfast, one of the waitress ladies surprised me very positively. She very quickly organized a table for 8 people, immediately brought a feeding chair for the baby, asked the children how they liked Wroclaw, etc. At one point my 7-year-old daughter asked if there was ketchup to which the lady with a smile: sure there is, I’m already going to bring you. A few words and gestures and it makes an impression.
Results
A simple test conducted confirmed the conclusions of the Wrocław researchers that when it comes to analyzing emotions, chatbots are significantly inferior to specialized programs such as “Sentimenti.”
The starting point is the results on individuals (Respondents) and we compare the other models to them: Sentimenti, ChatGPT 4.0, ChatGPT 3.5 and Google Bard.
As can be seen in the case of Sentimenti Analyzer, the values of the MSE and R² indicators are significantly better than those of the other programs.
MSE indicator – in the case of the “Sentimenti analyzer” its value is measured in the third and even in the fourth decimal place, showing the approach of the error (in the operation of the analyzer) to zero. The situation is different in other programs, where the value of this indicator is measured in the second and even in the first decimal place, showing a large area of error in the operation of these programs.
R² index – in the case of the “Sentimenti analyzer,” the values of this index range from 0.8217 to 0.9563, where the range 0.8 – 0.9 is defined as a good fit, and the range 0.9 – 1.0 as an excellent fit.
For the remaining programs, the highest R² index measures for:
- ChatGPT 4.0 – 0,3981
- ChatGPT 3.5 – 0,3769
- GOOGLE BARD – 0,4609
indicate an unsatisfactory fit.
GenAI hallucinations
There is another problem with generative models that insightful analysts point out. This is the tendency for the program to throw out erroneous information.
If this phenomenon, referred to as “hallucinations,” were a recurring one, there is a danger that ChatGPT, the next time it is asked about the level of sentiment for a certain financial asset, could give a completely different answer than it originally gave. In such a case, the investment strategies adopted by investors would be fed with erroneous data that could affect the results of financial transactions.
There is no such danger with the “Sentimenti analyzer”. Here, regardless of the number of queries about the level of emotions in a particular (same) text, we get identical results every time.
GenAI vs. Sentimenti in financial markets.
As part of the Sentistocks project, we use emotion intensity measurement to predict the future values of selected financial instruments. Measuring the intensity of emotion allows us to determine where the mood of the market, commonly referred to as investor sentiment, is headed.
Correlating the intensity of this indicator (built, after all, based on the emotions that investors feel) with financial data makes it possible, using learned models, to predict the future values of financial instruments. Currently, the model analyzes the sentiment of the cryptocurrency market (prediction of Bitcoin price in 15-minute intervals).
There are also reports of ChatGPT being used to predict stock market returns using sentiment analysis of news headlines.
As the authors of the cited publication point out:
…analysis shows that ChatGPT sentiment scores show statistically significant predictive power on daily stock market returns. Using news headline data and sentiment generation, we find a strong correlation between ChatGPT scores and subsequent daily stock market returns in our sample.
Noteworthy is the phrase in the publication:
ChatGPT sentiment scores show statistically significant predictive power.
Our brief test for ChatGPT (4.0 and 3.5) showed that the quality measures (MSE and R²) for these programs used to measure emotions that account for sentiment and its direction are unsatisfactory. And yet, the authors of the referenced publication obtained positive results from their study.
One might then ask how much better the results would have been if our Sentimenti analyzer had been used to measure investor sentiment? Where, in particular, the R² ratio was more than twice as high for our solution as we recorded it for ChatGPT?
Summary
Trying to be objective in our simplistic comparison, we unfortunately see several risks or even exclusions for using generative artificial intelligence models in professional applications, especially in areas based on sentiment and emotion measurement. In our opinion, this is due to both the design of the solution itself, the source data provided for model learning, and the concept of sentiment and emotion measurement itself.
In particular:
- differences in the construction of language models: the Sentimenti model was built from scratch based on data obtained from perhaps the largest research of its kind in the world, with human participants. Such prepared and targeted research usually results in a narrowed functionality. However, with, at the same time, a much higher quality of results compared to the approach of acquiring data in fact from every possible area to provide the model with as much “knowledge” as possible.
- differences in the way sentiment and emotion are interpreted: as part of the Sentimenti project, subjects answered the question of what emotion the text shown to them arouses in them – a minimum of 50 different people for each text. Thus, the reader’s (message recipient’s) reception of the text was measured. In the case of generative AI models, we get a “subjective” evaluation of a given language model. Thus, it is closer to trying to guess the intentions of the author of the message (the sender of the message) to evoke certain emotions rather than how that message might be received. Such interpretation is not, in our opinion, suitable for objective determination of emotions and for use in tools using sentiment and emotion measurement.
- hallucinations: a generally known problem of generative models. Unlike the Sentimenti model, focused on the evaluation of 11 emotive indicators, where such a difficulty does not occur and the evaluation of a given text will always give the same result, GenAI can relatively often give an answer that is not true. In our opinion, this precludes the use of GenAI for professional linguistic applications.
by Damian Grimling | Oct 7, 2022 | Sentistocks
Credit Suisse’s stock price has been on the decline for many months. Confusion over the bank’s financial condition is growing. There are further reports in the media about the potential insolvency of the institution, which is the second largest Swiss bank. Could its share price have been predicted? Yes – thanks to artificial intelligence.
Safe as a Swiss bank. Credit Suisse CDS price is a real trouble
Credit Suisse’s stock price has been on the decline for many months. As Sentimenti deals with, among other things, predicting the prices of financial instruments or cryptocurrencies, we decided to take a look at the atmosphere and mood on the web around Credit Suisse.
The analysis looked at the level of intensity of emotion and emotional arousal among authors of online opinions and comments, which were written in English and German. That’s a total of more than 40,000 mentions from May to September 2022, which our artificial intelligence looked at. The results are surprising.
It was not surprising to find a high level of intensity of emotions generally understood as negative emotions somewhat related to uncertainty (fear and sadness) or neutral emotions (surprise and anticipation). However, it was a big surprise to find an increasing level of intensity of emotions conceived as positive (trust and joy).
A detailed analysis of the content of the entries showed that the entries characterized by a high intensity of these favorable emotions are those expressing – generally speaking – satisfaction with Credit Suisse’s troubles. This state of affairs significantly affected the level of the correlation index of the intensity of emotions and the bank’s stock price.
The study of emotion diads (which form according to Pluchik’s theory) also leads to interesting observations, including showing how market attitudes changed. Thus, emotions correlated most strongly with Credit Suisse stock prices:
in May of expectation and trust – forming a diad of fatalism;
in June and July of fear and sadness – forming a diad of despair;
in August expectation and sadness – forming a diad of pessimism;
in September of expectation and disgust – forming a diad of cynicism.
Using Pearson correlation coefficient analysis, the interaction between stock prices and the intensity of emotion around Credit Suisse on the Internet was examined.
Credit Suisse problems. Are we in for a repeat of Lehman Brothers?
The analysis showed that there was a rare case where all emotions (including those considered positive) correlated negatively with Credit Suisse share prices. This meant that, in this case, a recorded increase in the intensity of emotions was linked to a decrease in share prices. In September, the strongest correlations with the share prices of Switzerland’s second largest bank were revulsion and expectation forming – according to Pluchik’s theory – a diad of cynicism.
A prototype stock price prediction model was fed for Credit Suisse with financial data (share prices) and data on the intensity of emotions, sentiment and emotional arousal around the company. A model built on an artificial neural network using BiLSTM (bidirectional long short-term memory) was used.
Using our predictive model (Sentistocks service), a success rate of up to 88% in predicting the trend of Credit Suisse’s share price over the period under study was achieved. This shows that potential investors could have successfully predicted the quotation solely on the basis of online discussions about the bank’s financial situation.
Investors remember the great collapse of Lehman Brothers. 14 years ago, it contributed to the outbreak of the global financial crisis. Will Credit Suisse face the same finale? We do not know. But the cost of CDS (insurance against bankruptcy) for the institution has approached the highest levels in almost 15 years. The bank itself foresees bumpy times ahead.
Credit Suisse CDS price. How emotions suggest the direction of a share price trend?
Sentistocks has developed effective predictive models for instruments in the cryptocurrency market. These models use both financial and emotive data to predict future prices (rates). The high success rate of the predictions developed with our tool confirms the huge role played by emotions in financial markets.
by Damian Grimling | Nov 6, 2021 | Sentimenti research
When sentiment analysis began to be used for marketing activities in the early 2000s, it opened up vast opportunities for the marketing and advertising industries. Understanding consumer sentiment allowed for better validation of actions and more precise targeting of target groups. Today, 20 years later, new doors are opening—rather than just measuring sentiment, we can now analyze emotions, and this offers even more benefits. What are they?
FROM THIS ARTICLE, YOU WILL LEARN: |
-
What is the difference between sentiment and emotion? -
What is the difference between sentiment analysis and emotion analysis? -
How do the results of both algorithms look in practice? -
Emotion analysis is not only effective in content marketing -
Perspectives on the development of emotion analysis tools |
Differences Between Sentiment and Emotions: Full List of Emotions
First, let’s define sentiment and emotions—the most important differences arise at this basic level because sentiment and emotions are two distinct phenomena.
-
- Sentiment is the state in which a person experiencing emotions can name what they are feeling, connect that experience with thoughts about it, and then make a conscious decision regarding the source of the stimulus. Sentiment, therefore, is the sum of physiological reactions (from the body) and cognitive processes triggered by the experience of emotions. Since it is a conscious state, it can be sustained by the person experiencing it. In general, sentiment is a broad mental attitude towards a particular experience.
- Emotions, on the other hand, are the physiological and, consequently, psychological brain responses to an external stimulus or an experience related to such a stimulus. The combination of the experience and the body’s reaction (hormonal activity) causes a short-lived and unconscious state, leading to specific actions such as fight, flight, freeze, awe, disgust, etc. What’s more—each emotion is tied to a different type of reaction, resulting in different behaviors.
As you can see, without experiencing emotions, you cannot enter a state of sentiment. Emotions operate at the lowest, behavioral level, while sentiment is their reflection, consideration, and evaluation by the person experiencing them. Emotions cause spontaneous reactions, while sentiment leads to conscious and controlled actions. The most significant difference: there are only eight basic emotions (according to Plutchik’s theory: joy, sadness, anger, fear, disgust, surprise, anticipation, and trust), which, when combined, form secondary, higher emotions that lead to the creation of feelings. In the case of sentiment, we are dealing only with positive and negative sentiment, sometimes also neutral.
Exploring Emotions: Media Monitoring vs. Sentitool – A Tool Comparison
Just as the phenomena of sentiment and emotions differ, so do the tools used to analyze them. These tools leverage advanced technologies such as neurolinguistic programming, machine learning, and other AI algorithms. Currently, in the Polish market for sentiment analysis in online content, several reputable companies exist, but in the field of emotion analysis, only one stands out as a true pioneer—Sentimenti. Below is a comparison of the general capabilities of sentiment and emotion analysis tools.
| |
-
examining the general tone of statements made by Internet users, article authors, etc. -
determine the user's impression and attitude after reading the text -
ease of processing and estimation of analysis results -
Useful for evaluating online brand mentions, social media management and customer communications, including complaint handling -
crisis communication support -
comparing consumer attitudes toward our own and competitors' products | -
to examine the instinctive reactions of Internet users and the intentions of the creators -
The ability to get to the emotions behind a particular sentiment and determine the future actions of those commenting -
Much greater range of information collected (8 emotions, sentiment analysis and emotional arousal) -
ability to work on huge groups of respondents -
assessment of consumer sentiment before and after the advertising campaign -
validation of marketing strategies -
monitoring emotions about the company in online mentions -
-
comparing brand perceptions -
analysis of emotions associated with influencers, YouTubers, bloggers, etc. to accurately select a brand ambassador |
Conclusion: The common (even within the industry) usage of the terms sentiment analysis and emotion analysis is incorrect. Sentiment is a much narrower concept, indicating in practice only the tone of a statement and possibly the mood of the author. Emotion analysis, on the other hand, describes the level of individual emotions (providing a percentage result for eight components, as well as the type of sentiment and the level of emotional arousal); with this data, consumer reactions and behaviors can be predicted with high accuracy.
As you can see, this data is more detailed; its analysis is more challenging, but it is also more accurate and comprehensive. Sentimenti’s algorithm includes as many as 30,000 words and phrases collected from a group of 22,000 people. The algorithm itself was developed in collaboration with the Wrocław University of Science and Technology and the Brain Imaging Lab of the Polish Academy of Sciences.
How Do the Results of Both Algorithms Look in Practice?
Time for some practice! Take a look at the following analysis of a real review:
I will never buy anything from smelly V(…) again. I recently decided I didn’t want to buy tragic clothes from chain stores, so I bought from there, and the supposedly perfect condition item, without signs of wear, had a stain that was visible. The woman claims that when she sent it, the stain wasn’t there.
This review is authentic and taken directly from a portal. What can you read from it when measuring sentiment versus emotion?
Results: Anger – 70%, Fear – 45%, Anticipation – 24%, Surprise – 57%, Trust – 14%, Sadness – 60%, Disgust – 62%, Joy – 10%, Positive sentiment – 10%, Negative sentiment – 70%, Emotional arousal – 72%.
The comment comes from a disappointed customer who, instead of making a purchase from one of the popular boutiques, opted for what she considered higher-quality used clothes. The purchase turned out to be unsuccessful, and the item was stained despite being described as in good condition. The buyer is clearly upset (emotional arousal at 72%), angry (a high 70%), feeling disgust (62%) and sadness (60%)—as she expected a good purchase. Notice that the woman also exhibits a high level of surprise (57%) and fear (45%)—when you add the disgust result, you can understand that this is her physical reaction to the experience of being deceived.
Now look at the same review from a sentiment analysis perspective: you receive two results—negative (70%) and positive (10%). The negative sentiment is evident even without automatic analysis, given phrases like smelly, tragic, stain, woman. The phrases supposedly perfect condition and without signs of wear build the low positive sentiment score.
The key information, however, is hidden—the phrase smelly refers directly to the shopping platform, tragic to the quality of chain store clothes, and woman to the seller. The overall tone of the review is set by the epithets unrelated directly to the purchased item, yet the sentiment analysis result remains negative.
Conclusion: With percentage results for basic emotions and knowledge of the typical physiological reactions they correspond to, you can estimate consumer behavior in this situation. Anger is associated with an attack response (hence the comment on the portal), disgust and fear with flight, and sadness with freezing. The emotional consumer will likely switch to another platform or opt for in-person shopping. She will probably not use this platform again.
Emotion Analysis is Effective Beyond Content Marketing
You’ve just seen how sentiment and emotion analysis results are interpreted in a specific example, and what kind of insights you can gain from them. As you’ve likely noticed, emotion analysis is a far more comprehensive, complete solution tailored to customer needs. But are emotion analysis algorithms limited to marketing, PR, or customer service?
Definitely not. With the development of machine learning technology and the implementation of increasingly sophisticated AI algorithms, the possibilities of emotion analysis extend into other industries. Today, emotions can be analyzed, for example, to forecast stock market prices or investment opportunities in the cryptocurrency market.
Future Prospects for Emotion Analysis Tools
If the above information hasn’t convinced you of the superiority of emotion analysis over sentiment analysis, look at the development potential of the former. Artificial intelligence is already highly advanced. Today, AI is being implemented not only to study but also to create emotionally engaging content aimed at achieving set goals.
These goals include increasing conversion rates from marketing activities, acquiring more effective leads, or providing customer service tailored to consumer needs (including potential crisis situations and ways to avoid them). The latter goal significantly improves user experience, increasing customer loyalty to the brand, which, in turn, strengthens the brand’s position in the market. That’s the potential of this technology—sentiment analysis can’t do that.
by Damian Grimling | Nov 6, 2021 | Categorising comments, SentiBrand
Shopping is a constant part of our lives. Every day, we come into contact with a huge number of more or less recognizable brands and companies. However, only some of them we choose more often and more willingly. Sometimes it happens completely automatically. Why does this happen and what determines our choice? It is probably emotions.
Brand loyalty vs. brand strength. What determines our choice?
There’s no denying and it’s been proven that our purchasing decisions are mostly influenced by emotions and the bond that a brand has developed with us. A recent study conducted by Deloitte entitled “From disparate signals to transformative action. The latest research by Deloitte “From disparate signals to transformative action” shows that emotional attachment to a brand is a decisive factor when making purchases for a group of 80% of consumers. In turn, 62% of them declared that they have some kind of relationship with a given brand.
– The brand-consumer relationship is a particularly important factor in sales strategy because it is directly related to customer satisfaction. We as manufacturers are constantly challenged to create better and better offers that meet consumer expectations,” – reads the report.
For what purpose do we shop? Consumers often shop primarily to improve their mood. Nowadays, the purchase of a particular good or service does not have to be dictated by discounts, promotions or low prices. The key here is the mentioned emotions. They are the reason why some of our purchasing decisions are not very rational, made on impulse and at the moment.
So, what can a brand do to generate emotions and become even more liked and recognized by consumers? What is the secret and power of the biggest brands and their messages? Apart from emotions, it is important to build positive associations in the recipients. It is worth remembering to adjust the communication strategy to the target group. An advertising campaign addressed to teenagers should look different from one addressed to people aged 40+.
Visual identity also plays a huge role in consumers’ perception of a brand. We live in a world of short information, slogans and signs. That is why logos, colors, symbols and general aesthetics of products are important. All this also creates the image of the company according to the buyer and has a bearing on building a relationship between him and the brand.
The basis is brand loyalty and trust
Today’s market abounds in various brands, and more and more are created all the time. We are almost flooded with various products and symbols, often not knowing what it all means. However, companies that enjoy the greatest recognition and consumer choice are not resting on their laurels and are constantly developing their offers. The key is to constantly work on the image because without this, it is impossible to maintain the leadership position.
In the aforementioned Deloitte report we read that if a brand wants not only to stay on the market, but above all to boast of constant trust, it needs constant work and listening to consumers. This means staying up to date with the ever-present changes and new trends. Without this, it will not be possible to maintain the highest standards and respond to the preferences of as many customers as possible.
The consumer who has established a bond with the brand is then not only a recurring generator of profit, but also an engaged recipient. Building long-term partnerships with consumers means, above all, responding responsibly and effectively to their needs. More and more companies emphasize personalization and considering individual preferences of consumers to provide them with the most suitable benefits.
Many factors contribute to the choice of a particular brand, but first there are positive emotions and associations. To build them, we need a dialogue with consumers, a trusting answer to their needs as well as building the opinion of a unique brand, positively distinguishing itself from the competition. We can easily learn about emotions by analyzing media, including blogs, news portals, internet forums and most of all social networking sites. That’s what Sentimenti tools are for.
