Does sentiment analysis always tell the truth? Sentiment analysis is a popular tool that helps understand how users react to social media content. However, is it always fully reliable? We analyzed reactions to The New York Times’ post on Platform X regarding the platform’s potential ban in Brazil to see what lies behind the emotion expressed in the comments.
Comments under the above post (formerly a tweet – we miss a bit😉) appeared in both English and Spanish, so we analyzed them all.
What does sentiment analysis say in the comments?
A preliminary analysis of the NYT entry shows that positive sentiment prevailed, with 31% of the content carrying positive emotion and only 12% being negative. At first glance, one would think that the reception of the entry would be similar.
However, an analysis of the comments under the post reveals a very different reality. As many as 62% of the comments were characterized by negative sentiment, while only 38% had positive overtones. This means that despite the positive tone of the post itself, readers’ reactions were mostly negative.
Such data may suggest that the majority of users have negative feelings about a post, but analysis of sentiment alone does not tell us why. Are the comments full of anger, sadness, surprise? Or do they express revulsion or disappointment? To fully understand what viewers really feel, we need a more detailed analysis of emotion.
The emotions behind the negative sentiment
An analysis of the emotions in the comments shows that the main emotions responsible for negative sentiment were anger and sadness. We can trace this by analyzing the average value of each emotion and the comments with the highest and lowest intensity of emotion.
With this tool, we can not only see what emotions are dominant, but also identify exactly towards what or whom they are directed. In our case, anger and sadness were directed towards both the Brazilian government and the person Elon Musk.
For the sake of illustration, we analyzed a number of comments in which anger and sadness dominated in order to understand what specific issues aroused these emotions. Anger mainly concerned the decisions of the Brazilian authorities, which may affect the banning of Platform X. Sadness, on the other hand, came in the context of concerns about the future of freedom of expression and the impact Elon Musk has on the operation of the platform. With such data, we can indicate how strong readers’ views and emotions are.
And now analyzing the content of the comments, we can conclude that the negative sentiment, dominated by anger and sadness, was directed towards both the broader Brazilian authorities and the person of E. Musk. And having data on the intensity of selected emotions, we can even indicate the level of strength with which readers’ views are expressed.
Arousal – a measure of reader engagement
Another important indicator to consider is “arousal,” or the level of emotional involvement in the discussion. Stimulation measures the intensity of emotion – the higher the value, the stronger the reactions of commenters.
In the case of the entry analyzed, the average level of agitation was 58%, which means that the comments were clearly emotional. The range of agitation ranged from 44% to 80%, showing that most commenters were heavily involved in the discussion. As many as 79% of comments had agitation above 50%, indicating strong emotion in the debate.
Conclusion – is sentiment analysis enough?
Based on the above results, we can see that sentiment analysis alone does not give a complete picture of user reactions. While it can indicate whether comments are positive or negative, it does not tell us what specific emotion is behind those reactions.
Analyzing emotions, such as anger, sadness or surprise, is essential to better understand why people react in certain ways. In this case, we can see that anger and sadness were related to specific issues, such as the actions of the Brazilian government or Elon Musk’s attitude.
Therefore, in order to get the full picture, it is worth reaching for more advanced emotion analysis tools, which we are developing at Sentimenti with the help of artificial intelligence. By doing so, they will help us better understand what Internet users really think and are driven by.