by Damian Grimling | | Politics and Social
The results of the October 15, 2023 elections have had a stimulating effect on the prices of many financial instruments. Sentimenti analysts surveyed with the proprietary tool SENTITOOL, the period from October 11 to 18, measuring the level of the investor confidence index, defined as the intensity of confidence emotions expressed in posts made on social media. The study was based on an analysis of 1,517 such posts relating to diversified financial instruments.
Changes in the trust index
The final days of the campaign and the ordered election silence in effect especially on Saturday, October 14 caused investors to take a distinctly wait-and-see position. This resulted in a drop in the daily confidence index by an average of 3 percentage points from the level recorded as recently as October 12. As expected, only the successively announced results on voter turnout and preliminary exit poll data caused a decisive change in the level of the confidence index, which reached an increase of 11 percentage points on October 15.
Such a change in sentiment has boosted financial markets. A prime example of such a revival was the WIG20 bourse, which had already risen 5.3% from its last quote on October 16. In the following days, too, a cause-and-effect relationship can be seen between the confidence index and the WIG20 bourse.
Confidence index on Election Day
There was an interesting development of the confidence index on election day itself,October 15. An hour-by-hour analysis shows what dynamic fluctuations it underwent. Thus, after the opening of polling stations, it had an upward trend, which was slowed down by the turnout data at 12:00 p.m. The subsequent data at 5:00 p.m., combined with exit poll data, resulted in a dynamic (by 11 percentage points) increase in the confidence index. The 9:00 pm hour brought preliminary exit poll results. This, combined with turnout data, further boosted the index by another 16 percentage points. The 23:00 hour brought a record high for the confidence index – a level of 71% was recorded. This data brought the average daily confidence index on October 15 to 32%. It was 11 percentage points higher compared to the previous day.
Whether the positive momentum generated by the election results will have a more lasting trend in the financial markets and the economy as a whole, time will tell. Perhaps it will depend, among other things, on the pace at which the election promises are implemented. Promises made by the coalition that will take over our country.
The analysis was carried out thanks to the materials provided, in cooperation with the IBIMS Institute for Internet and Social Media Research.
by Damian Grimling | | Sentimenti research
Recently, the so-called generative artificial intelligence (GenAI) has received great attention. It is capable of generating new data, images, text or other content based on patterns or input, to quote ChatGPT. Or put another way, it can create new data such as text, images, videos, and music. It relies on machine learning algorithms that allow it to analyze existing data and then generate new data that is similar to the original data, following Google Bard’s definition.
These language models are often treated by users as an “oracle that knows the answers to all questions.” The creators of these solutions allow their tools to describe and review almost all spheres of our lives, including our emotions, and users take advantage of this eagerly.
But are these descriptions, answers to questions and reviews fully reliable, especially in terms of emotions? Let’s check it out!
Testing the efficiency of generative AI
A team of scientists from the department of linguistic engineering at Wroclaw University of Technology, which also included scholars working on our solution “Sentimenti – an analyzer of emotions in text,” subjected ChatGPT and other available language models (including sentiment analysis systems) to a comparative study.
As the study by the Wroclaw team showed, GenAI performed the worst on pragmatic tasks requiring knowledge of the world and precisely on emotion assessment.
Our tests at Sentimenti
Measuring sentiment, emotion and emotional arousal is the essence of Sentimenti. That’s why we were tempted to take a simple test to check.
We analyzed four randomly selected texts using “Sentimenti analyzer”; ChatGPT 4; ChatGPT 3.5 and GOOGLE BARD for the content of emotions they emanate. As a unit of measurement of the quality of the performance of the indicated programs was adopted:
- MSE (mean squared error) – mean squared error as a coefficient of error evaluation (which is the smaller, the more favorable for the software under test) and
- R² – the quality coefficient of the model fit (the larger its value, the better the fit of the software under test).
The benchmark for checking the quality of the performance of the selected programs was the results of the emotive evaluation of the texts. There were selected for analysis carried out on respondents (people) participating during the development of our emotion analyzer. Since the emotion values presented by the ChatGPT 4, ChatGPT 3.5 and Google Bard programs are shown in two decimal places, we made analogous rounding in the “Sentimenti analyzer” and in the results of the survey conducted on respondents.
Randomly selected texts, subjected to GenAI analysis
Text 1
As I have stayed at this hotel many times, my review will not change much. Underneath the rather ugly and neglected exterior of the hotel, there is a nicely renovated interior and friendly staff. The rooms are clean, bright and quite spacious. Breakfasts are excellent. Unfortunately, the restaurant menu, as in all Novotels, is terrible, and the dishes are modest, unpalatable and very expensive. Fortunately, there are many places in Krakow where you can eat well. I definitely do not recommend the restaurant, but the hotel as much as possible. It would be worthwhile to think about renewing the body of the hotel, as it is from a different era. It was the same in my student days, 25 years ago!!!!
Text 2
The hotel in an ideal location as far as the boat is concerned, two minutes from Piotrowska in the center, but as far as cleanliness is a tragedy the bathroom stank and the bedding dirty, for this class of hotel it is unacceptable
Text 3
The hotel only for a one-night stay for business and nothing else. After a recent stay of several days with family, we decided that this would be our last visit to this hotel. Food standard for all Ibis, rooms also. Service at the reception – roulette. Upon checking in, we were given perhaps the smallest corner room in the entire hotel despite a prior phone reservation for specific rooms on specific floors, supposedly due to occupancy. By a strange coincidence, a few minutes later during service by another receptionist our rooms were found.” In the rooms not very clean, housekeeping does not pay attention to the types of water put despite numerous requests. The restaurant leaves much to be desired.”
Text 4
The hotel is located near the center of Wroclaw. Room and bathroom clean, toiletries ok, maybe a little too soft bed, but it was not bad. Breakfast quite good. Parking additionally paid 25 zl. On a very big plus I wanted to evaluate the service. At the reception everything efficiently and with a smile. However, during breakfast, one of the waitress ladies surprised me very positively. She very quickly organized a table for 8 people, immediately brought a feeding chair for the baby, asked the children how they liked Wroclaw, etc. At one point my 7-year-old daughter asked if there was ketchup to which the lady with a smile: sure there is, I’m already going to bring you. A few words and gestures and it makes an impression.
Results
A simple test conducted confirmed the conclusions of the Wrocław researchers that when it comes to analyzing emotions, chatbots are significantly inferior to specialized programs such as “Sentimenti.”
The starting point is the results on individuals (Respondents) and we compare the other models to them: Sentimenti, ChatGPT 4.0, ChatGPT 3.5 and Google Bard.
As can be seen in the case of Sentimenti Analyzer, the values of the MSE and R² indicators are significantly better than those of the other programs.
MSE indicator – in the case of the “Sentimenti analyzer” its value is measured in the third and even in the fourth decimal place, showing the approach of the error (in the operation of the analyzer) to zero. The situation is different in other programs, where the value of this indicator is measured in the second and even in the first decimal place, showing a large area of error in the operation of these programs.
R² index – in the case of the “Sentimenti analyzer,” the values of this index range from 0.8217 to 0.9563, where the range 0.8 – 0.9 is defined as a good fit, and the range 0.9 – 1.0 as an excellent fit.
For the remaining programs, the highest R² index measures for:
- ChatGPT 4.0 – 0,3981
- ChatGPT 3.5 – 0,3769
- GOOGLE BARD – 0,4609
indicate an unsatisfactory fit.
GenAI hallucinations
There is another problem with generative models that insightful analysts point out. This is the tendency for the program to throw out erroneous information.
If this phenomenon, referred to as “hallucinations,” were a recurring one, there is a danger that ChatGPT, the next time it is asked about the level of sentiment for a certain financial asset, could give a completely different answer than it originally gave. In such a case, the investment strategies adopted by investors would be fed with erroneous data that could affect the results of financial transactions.
There is no such danger with the “Sentimenti analyzer”. Here, regardless of the number of queries about the level of emotions in a particular (same) text, we get identical results every time.
GenAI vs. Sentimenti in financial markets.
As part of the Sentistocks project, we use emotion intensity measurement to predict the future values of selected financial instruments. Measuring the intensity of emotion allows us to determine where the mood of the market, commonly referred to as investor sentiment, is headed.
Correlating the intensity of this indicator (built, after all, based on the emotions that investors feel) with financial data makes it possible, using learned models, to predict the future values of financial instruments. Currently, the model analyzes the sentiment of the cryptocurrency market (prediction of Bitcoin price in 15-minute intervals).
There are also reports of ChatGPT being used to predict stock market returns using sentiment analysis of news headlines.
As the authors of the cited publication point out:
…analysis shows that ChatGPT sentiment scores show statistically significant predictive power on daily stock market returns. Using news headline data and sentiment generation, we find a strong correlation between ChatGPT scores and subsequent daily stock market returns in our sample.
Noteworthy is the phrase in the publication:
ChatGPT sentiment scores show statistically significant predictive power.
Our brief test for ChatGPT (4.0 and 3.5) showed that the quality measures (MSE and R²) for these programs used to measure emotions that account for sentiment and its direction are unsatisfactory. And yet, the authors of the referenced publication obtained positive results from their study.
One might then ask how much better the results would have been if our Sentimenti analyzer had been used to measure investor sentiment? Where, in particular, the R² ratio was more than twice as high for our solution as we recorded it for ChatGPT?
Summary
Trying to be objective in our simplistic comparison, we unfortunately see several risks or even exclusions for using generative artificial intelligence models in professional applications, especially in areas based on sentiment and emotion measurement. In our opinion, this is due to both the design of the solution itself, the source data provided for model learning, and the concept of sentiment and emotion measurement itself.
In particular:
- differences in the construction of language models: the Sentimenti model was built from scratch based on data obtained from perhaps the largest research of its kind in the world, with human participants. Such prepared and targeted research usually results in a narrowed functionality. However, with, at the same time, a much higher quality of results compared to the approach of acquiring data in fact from every possible area to provide the model with as much “knowledge” as possible.
- differences in the way sentiment and emotion are interpreted: as part of the Sentimenti project, subjects answered the question of what emotion the text shown to them arouses in them – a minimum of 50 different people for each text. Thus, the reader’s (message recipient’s) reception of the text was measured. In the case of generative AI models, we get a “subjective” evaluation of a given language model. Thus, it is closer to trying to guess the intentions of the author of the message (the sender of the message) to evoke certain emotions rather than how that message might be received. Such interpretation is not, in our opinion, suitable for objective determination of emotions and for use in tools using sentiment and emotion measurement.
- hallucinations: a generally known problem of generative models. Unlike the Sentimenti model, focused on the evaluation of 11 emotive indicators, where such a difficulty does not occur and the evaluation of a given text will always give the same result, GenAI can relatively often give an answer that is not true. In our opinion, this precludes the use of GenAI for professional linguistic applications.
by Damian Grimling | | Sentistocks
Credit Suisse’s stock price has been on the decline for many months. Confusion over the bank’s financial condition is growing. There are further reports in the media about the potential insolvency of the institution, which is the second largest Swiss bank. Could its share price have been predicted? Yes – thanks to artificial intelligence.
Safe as a Swiss bank. Credit Suisse CDS price is a real trouble
Credit Suisse’s stock price has been on the decline for many months. As Sentimenti deals with, among other things, predicting the prices of financial instruments or cryptocurrencies, we decided to take a look at the atmosphere and mood on the web around Credit Suisse.
The analysis looked at the level of intensity of emotion and emotional arousal among authors of online opinions and comments, which were written in English and German. That’s a total of more than 40,000 mentions from May to September 2022, which our artificial intelligence looked at. The results are surprising.
It was not surprising to find a high level of intensity of emotions generally understood as negative emotions somewhat related to uncertainty (fear and sadness) or neutral emotions (surprise and anticipation). However, it was a big surprise to find an increasing level of intensity of emotions conceived as positive (trust and joy).
A detailed analysis of the content of the entries showed that the entries characterized by a high intensity of these favorable emotions are those expressing – generally speaking – satisfaction with Credit Suisse’s troubles. This state of affairs significantly affected the level of the correlation index of the intensity of emotions and the bank’s stock price.
The study of emotion diads (which form according to Pluchik’s theory) also leads to interesting observations, including showing how market attitudes changed. Thus, emotions correlated most strongly with Credit Suisse stock prices:
in May of expectation and trust – forming a diad of fatalism;
in June and July of fear and sadness – forming a diad of despair;
in August expectation and sadness – forming a diad of pessimism;
in September of expectation and disgust – forming a diad of cynicism.
Using Pearson correlation coefficient analysis, the interaction between stock prices and the intensity of emotion around Credit Suisse on the Internet was examined.
Credit Suisse problems. Are we in for a repeat of Lehman Brothers?
The analysis showed that there was a rare case where all emotions (including those considered positive) correlated negatively with Credit Suisse share prices. This meant that, in this case, a recorded increase in the intensity of emotions was linked to a decrease in share prices. In September, the strongest correlations with the share prices of Switzerland’s second largest bank were revulsion and expectation forming – according to Pluchik’s theory – a diad of cynicism.
A prototype stock price prediction model was fed for Credit Suisse with financial data (share prices) and data on the intensity of emotions, sentiment and emotional arousal around the company. A model built on an artificial neural network using BiLSTM (bidirectional long short-term memory) was used.
Using our predictive model (Sentistocks service), a success rate of up to 88% in predicting the trend of Credit Suisse’s share price over the period under study was achieved. This shows that potential investors could have successfully predicted the quotation solely on the basis of online discussions about the bank’s financial situation.
Investors remember the great collapse of Lehman Brothers. 14 years ago, it contributed to the outbreak of the global financial crisis. Will Credit Suisse face the same finale? We do not know. But the cost of CDS (insurance against bankruptcy) for the institution has approached the highest levels in almost 15 years. The bank itself foresees bumpy times ahead.
Credit Suisse CDS price. How emotions suggest the direction of a share price trend?
Sentistocks has developed effective predictive models for instruments in the cryptocurrency market. These models use both financial and emotive data to predict future prices (rates). The high success rate of the predictions developed with our tool confirms the huge role played by emotions in financial markets.
by Damian Grimling | | Sentimenti research
When sentiment analysis began to be used in marketing activities in the early 2000s, the marketing and advertising industry opened up a wide range of horizons. Knowing consumer sentiment allowed for better validation of actions and more precise targeting. Today, 20 years later, another door has opened – instead of just studying sentiment, you can now study emotions, which brings even more benefits. What are the benefits?
FROM THIS ARTICLE, YOU WILL LEARN: |
-
What is the difference between sentiment and emotion? -
What is the difference between sentiment analysis and emotion analysis? -
How do the results of both algorithms look in practice? -
Emotion analysis is not only effective in content marketing -
Perspectives on the development of emotion analysis tools
|
Sentiment versus emotion – how are they different?
But first, learn the definitions of sentiment and emotion – the most important differences come at this level, because sentiment and emotion are two different phenomena.
- Sentiment is a state in which the person experiencing the emotion is able to name what he or she is experiencing, connect that experience with thoughts about his or her experience, and then make a conscious decision toward the source of the stimulus. Sentiment is thus the sum of physiological reactions (from the body) and cognitive processes caused by the experience of an emotion. As it is a conscious state, it can be sustained by the person experiencing it. In general, it is a broadly defined mental attitude toward a given experience.
- Emotions are the physiological and, consequently, psychological reaction of the brain to an external stimulus or an experience connected with such a stimulus. The experience combined with the body’s reaction (hormone action) causes a brief and unconscious state that leads to a specific action, such as starting a fight, fleeing, freezing, delight, disgust, and so on. More importantly, each emotion is associated with a different type of response that results in a different behavior.
So, as you can see, without experiencing an emotion, you cannot experience a sentient state. Emotions operate at the lowest, behavioral level, while sentiment is the processing, reflection, and attitude of the experiencer. Emotion causes spontaneous reactions, while sentiment causes conscious and controlled reactions. And the most important difference – in the primary understanding of emotions alone there are 8 primary emotions (according to Plutchik’s theory these are joy, sadness, anger, fear, disgust, surprise, anticipation, and trust), and after the combination there are secondary, higher, leading to the formation of feelings. In the case of sentiment, we will only speak of positive and negative sentiment, sometimes distinguishing neutral sentiment as well.
Sentiment analysis and emotion analysis – comparison of tool capabilities
Just as the two phenomena differ, so will the tools to study them. These tools use advanced technologies, based on neuro-linguistic programming, machine learning and other AI algorithms. Currently on the Polish market of companies dealing with sentiment analysis in web content you will find several significant and even reputable brands, but in the field of emotion research – in practice – only one, but a pioneer – Sentimenti. Below you will find comparison of general capabilities of sentiment and emotion analysis tools.
| |
-
examining the general tone of statements made by Internet users, article authors, etc. -
determine the user's impression and attitude after reading the text -
ease of processing and estimation of analysis results -
Useful for evaluating online brand mentions, social media management and customer communications, including complaint handling -
crisis communication support -
comparing consumer attitudes toward our own and competitors' products
| -
to examine the instinctive reactions of Internet users and the intentions of the creators -
The ability to get to the emotions behind a particular sentiment and determine the future actions of those commenting -
Much greater range of information collected (8 emotions, sentiment analysis and emotional arousal) -
ability to work on huge groups of respondents -
assessment of consumer sentiment before and after the advertising campaign -
validation of marketing strategies -
monitoring emotions about the company in online mentions -
-
comparing brand perceptions -
analysis of emotions associated with influencers, YouTubers, bloggers, etc. to accurately select a brand ambassador
|
Conclusion: the popular (even in the industry) use of the terms sentiment analysis and emotion analysis is incorrect. Sentiment is a much narrower concept, indicating in practice only the overtone of an utterance and – possibly – the mood of its author. Meanwhile, emotion analysis describes the level of particular emotions (we get the percentage result of 8 components, plus the type of sentiment and the value of emotional arousal); having such data we can with high probability predict consumer reactions and behaviour.
As you can see, it is a large amount of data that is detailed; its analysis is more difficult to perform, but more accurate and cross-sectional. As Sentimenti points out, the catalog of their algorithm contains as many as 30 thousand words and phrases collected on a group of 22 thousand people, while the algorithm itself was developed in collaboration with the Wroclaw University of Technology and the Brain Imaging Laboratory of the Polish Academy of Sciences.
How do the results of both algorithms look in practice?
It’s time to practice! Take a look at the analysis below, which was subjected to such an opinion:
I will never buy anything at stinky V(…) again. I recently decided that I do not want to buy tragic clothes in chain stores, so I bought it there and the condition was supposedly perfect, with no sign of use, but with a stain that you can see. The woman wrote me that when she sent it back, it was not there.
This opinion is authentic and taken directly from the portal. What do you read from it when measuring sentiment and what emotions?
Results: anger – 70%, fear – 45%, anticipation – 24%, surprise – 57%, trust – 14%, sadness – 60%, disgust – 62%, joy – 10%, positive sentiment – 10%, negative sentiment – 70%, emotional arousal – 72%.
The statement is a comment from a disappointed customer who, instead of making a purchase in one of the popular boutiques, decided to reach for (in her opinion) better quality second-hand clothes. The purchase turned out to be wrong and the clothes were stained, although they were supposed to be in good condition. The shopper is definitely agitated (72% agitation), angry (high level – 70%), she also feels disgust (62%) and sad (60%) – after all she expected a good purchase. Note that the woman’s expression of emotion is quite high in surprise (57%) and fear (45%) – when you add in the disgust score, you understand that this is her physical reaction to experiencing the scam she fell victim to.
Now look at the same statement in terms of sentiment analysis: you get two results here – negative (70%) and positive (10%). The negative sentiment in the comment is obvious even without automatic analysis of the statement in question – this is evidenced by the phrases stinky, tragic, stain, granny. The phrases condition supposedly perfect and with no sign of use build a low score of positive sentiment.
The most important information for you, however, is hidden – the phrase stinky refers directly to the shopping platform, tragic to the quality of clothes from chain stores, and grandma – to the person selling it. In practice, the tone of the entire statement is set by epithets that are not directly related to the purchased goods, yet you still get a negative result from the sentiment analysis.
Conclusion: Having the percentage scores of basic emotions and knowing the typical physiological reactions corresponding to them, you can estimate the consumer’s behavior in this situation. Anger is equated with an attack response (hence the portal comment), disgust and fear are equated with fleeing, and sadness is equated with a frozen response. An upset consumer is likely to move to another platform or decide to shop in person. She will probably not use this portal again.
Emotion analysis is not only effective in content marketing
You’ve just learned how to interpret the results of sentiment analysis and emotion analysis on an individual example and the range of information you can gain from them. As you’ve also probably noticed, emotion analysis is a much more complete, cross-sectional, and customized solution. But are algorithms for exploring emotions in content only applicable in marketing, piracy or consumer service?
Definitely not. With the development of machine learning technologies and the implementation of better and better artificial intelligence algorithms, the possibilities of emotion analysis are expanding to other industries. Emotions can already be studied, for example, for regular forecasting of stock market prices or investment opportunities of the cryptocurrency market.
Perspectives on the development of emotion analysis tools
If the above information has not yet convinced you of the superiority of emotion analysis over sentiment analysis, then take a look at the development prospects of the former. The possibilities of artificial intelligence are already powerful. Currently, AI has been implemented not so much for research, but for creating emotionally charged content so that you can achieve your goals with it.
These goals include increasing conversions from marketing efforts, acquiring leads more effectively, or creating a customized (sic!) customer experience that takes into account potential crises and how to avoid/bypass them. This last goal definitely improves User Experience and generates greater customer loyalty to the brand. This, in turn, leads to a better image of the brand, and thus its position in the market. So much for technology development perspectives. Your sentiment analysis fails to do that.
by Damian Grimling | | SentiBrand, Categorising comments
Shopping is a constant part of our lives. Every day, we come into contact with a huge number of more or less recognizable brands and companies. However, only some of them we choose more often and more willingly. Sometimes it happens completely automatically. Why does this happen and what determines our choice? It is probably emotions.
Brand loyalty vs. brand strength. What determines our choice?
There’s no denying and it’s been proven that our purchasing decisions are mostly influenced by emotions and the bond that a brand has developed with us. A recent study conducted by Deloitte entitled “From disparate signals to transformative action. The latest research by Deloitte “From disparate signals to transformative action” shows that emotional attachment to a brand is a decisive factor when making purchases for a group of 80% of consumers. In turn, 62% of them declared that they have some kind of relationship with a given brand.
– The brand-consumer relationship is a particularly important factor in sales strategy because it is directly related to customer satisfaction. We as manufacturers are constantly challenged to create better and better offers that meet consumer expectations,” – reads the report.
For what purpose do we shop? Consumers often shop primarily to improve their mood. Nowadays, the purchase of a particular good or service does not have to be dictated by discounts, promotions or low prices. The key here is the mentioned emotions. They are the reason why some of our purchasing decisions are not very rational, made on impulse and at the moment.
So, what can a brand do to generate emotions and become even more liked and recognized by consumers? What is the secret and power of the biggest brands and their messages? Apart from emotions, it is important to build positive associations in the recipients. It is worth remembering to adjust the communication strategy to the target group. An advertising campaign addressed to teenagers should look different from one addressed to people aged 40+.
Visual identity also plays a huge role in consumers’ perception of a brand. We live in a world of short information, slogans and signs. That is why logos, colors, symbols and general aesthetics of products are important. All this also creates the image of the company according to the buyer and has a bearing on building a relationship between him and the brand.
The basis is brand loyalty and trust
Today’s market abounds in various brands, and more and more are created all the time. We are almost flooded with various products and symbols, often not knowing what it all means. However, companies that enjoy the greatest recognition and consumer choice are not resting on their laurels and are constantly developing their offers. The key is to constantly work on the image because without this, it is impossible to maintain the leadership position.
In the aforementioned Deloitte report we read that if a brand wants not only to stay on the market, but above all to boast of constant trust, it needs constant work and listening to consumers. This means staying up to date with the ever-present changes and new trends. Without this, it will not be possible to maintain the highest standards and respond to the preferences of as many customers as possible.
The consumer who has established a bond with the brand is then not only a recurring generator of profit, but also an engaged recipient. Building long-term partnerships with consumers means, above all, responding responsibly and effectively to their needs. More and more companies emphasize personalization and considering individual preferences of consumers to provide them with the most suitable benefits.
Many factors contribute to the choice of a particular brand, but first there are positive emotions and associations. To build them, we need a dialogue with consumers, a trusting answer to their needs as well as building the opinion of a unique brand, positively distinguishing itself from the competition. We can easily learn about emotions by analyzing media, including blogs, news portals, internet forums and most of all social networking sites. That’s what Sentimenti tools are for.
