Since attending Sentiment Analysis Symposium last month, we’ve been musing on where we see sentiment-focused text analytics headed next. While we’re excited by industry innovation so far, there’s still a few areas where current offerings are lagging behind the hype.

Sentiment analysis can be an incredibly powerful tool, but only when applied to the right use case and the right data. Here are a few areas where we feel the market is still facing challenges.

Authorship is key

 

Sentiment analysis tools use context clues to determine the feeling behind a tweet, post, or document. However sometimes the most valuable contextual information is not included within the written text.

For example, if you were scrolling through your Twitter feed and came across the tweet below, what would you guess the author was feeling?

Being involved in a lawsuit is a pretty unpleasant experience for the majority of people. Recognizing this, our sentiment analysis tool ‘correctly’ identifies the tweet as negative.

Spoiler alert: this is not hypothetical text, but a real tweet posted by hip hop artist Kanye West.

The music video for West’s recent single “Famous” features wax models of West and other celebrities such as Taylor Swift, Caitlin Jenner, and Donald Trump naked in bed together. West published the tweet (which has since been deleted) in anticipation of a public backlash to the controversial video.

A notoriously antagonistic celebrity, West thrives on all publicity, perhaps especially negative publicity. It’s plausible that he genuinely wanted a lawsuit, making this tweet positive. West’s public persona and attitude towards conflict are vital pieces of contextual information that are not contained within the text of the tweet.

For some use cases, this is a manageable problem. If you want to monitor the activity of celebrities, politicians, and other public figures, you could build a knowledge base of personalities and inclinations to take into consideration when assigning sentiment. Machine learning may even be able to recognize and adjust its profiles as personas change over time.

However, if you’re looking to monitor brand engagement and satisfaction among your customers, we enter far murkier waters. The information necessary to create a comprehensive and accurate personality profile of a non-public figure is not publicly available. Furthermore, a major brand may have millions of users to monitor as opposed to a select group of public figures. The data volume would be staggering. Sentiment analysis can get you the gist of consumer feedback, but be aware of where false positives or negatives can slip through the cracks.

So many feelings

Human emotion is astronomically more complex than a simple positive/negative assignment. Accordingly, a recurring image among Sentiment Symposium presentations (not to mention the event’s logo) was Plutchik’s wheel of emotions:

The sentiment analysis of the future will identify a full spectrum of emotion, but unfortunately even this is a first step towards a larger problem: we never feel only one emotion at any given time (Disney’s Inside Out illustrates this reality quite well). So what was Kanye feeling when he wrote this tweet?

• Aggression: West is antagonizing his ‘haters’
• Pride: He feels proud of the music he’s made
• Excitement: He looks forward to the coming media uproar
• Apathy: He genuinely doesn’t mind being sued

These emotions don’t even fall on the same spoke of Plutchik’s wheel, but a human reader can quickly recognize them. For machines to comprehend emotion on a human level, they would need to recognize a diverse spectrum of emotion and how drastically different emotions can be felt at the same time.

A simple positive/negative sentiment assignment can be terribly valuable for the right use cases. When looking for a specific measurable, for example, what your customers think of your most recent advertising campaign or your new customer service chatbot, two-dimensional sentiment is sufficient. However if you’re looking to dive deeper into your customer’s feelings and preferences, a human eye is necessary.

When we don’t say what we mean

Irony and sarcasm are a well known thorn in the side of sentiment analysis providers. We won’t take time here rehashing what has been so eloquently covered by the academic community (read a few of the many articles on the subject here, here, and here). However little has been said on how rapidly sarcasm and irony evolve. This is particularly true of social media, where young people–the drivers of slang and language evolution–make up the majority of users.

For example, an individual who disliked Kanye West’s clothing line in 1998 may have written:

While we can train our tools to recognize this contradictory speech pattern as negative, that wouldn’t prepare us for the same sentiment in 2016, in which “Not!” may be replaced with a meme or string of emojis:

Ask your sentiment analysis provider how they work with sarcasm. Being able to handle irony–a tricky problem in itself–isn’t comprehensive when analyzing social media data. This ability doesn’t yet exist in current sentiment analysis offerings, but could in the future.

Proceed with caution

As a provider of sentiment analysis ourselves (now in Spanish too!), we certainly don’t want to discredit the value it can add to your unstructured data management. However you must have a clear understanding of what you’re hoping to learn from sentiment analysis, and whether your provider (or any provider) can actually deliver. For now, consider using supervised sentiment analysis: the software can highlight anomalies in your data for human review.

And fellow sentiment analysis providers? We love the promise of deep learning sentiment analysis too but we’ve all got some work to do before taking home our big data moonman.