Sentiment Analysis Algorithm – Demystified (In Layman’s Terms)

Let us get the definitions straight first! The sentiment is the attitude of a speaker or a writer towards anything, such as a personality, business, product or even a place. Sentiment analysis is the method of equipping a computer algorithm to determine the writer’s sentiment using Natural Language Processing (NLP).

For example:

“Amazing Phone. Don’t think, Just Go for it.”  – Positive Sentiment

“The signal drops very frequently, and voice quality is also bad” – Negative Sentiment

Different methods including statistical techniques are employed to evaluate sentiment. Few depend ultimately on the algorithm itself; Few employ people to interpret it, and others take a hybrid approach.

Also, not always sentiment is about positive or negative, it can go further deep into capturing emotions of the writer (sad, angry, happy, excited, etc.) and this is not easy as you might see in the above examples because, in real-time, the sentences are always lengthy with lot of connections and also it ‘s hard to understand the context of an author. (Example: Sarcasm) The scope of this blog is to give you a glimpse of how a rule-based sentiment engine is built. The idea is to make a newbie understand. To start with, let’s define the goal of this exercise, we will discuss on a rule-based approach based on the intuition of how a person will interpret the sentiment of a sentence. A note of caution here, there are many ways to deal with it, and this is one way to do so.

Let’s take the above examples,Amazing Phone. Don’t think, Just Go for it Now, If I ask you to determine the sentiment of the above sentence with just an objective answer(Positive | Negative), what will it be? Obviously, it’s going to be “Positive” without no doubt! Now let’s drill down the answer further, Let’s split the sentence into individual words, “Amazing” + “Phone” + “Don’t” + “think” + “Just” + “Go” + “for” + “it”

Now, if you notice carefully, the following words do not express any sentiments as a single word alone. “Phone”, “Don’t”, “think”, “Just” ,“Go”,“for”,”it” But the word “Amazing” alone defines the sentiment “positive.” Now based on this intuition, if we were to build a small algorithm, all we need is a list of words that we know for sure is positive and the list of words for certain that we are aware that they are negative. For now, forget about how to procure the lists since there are a lot of open-source lists already available on the web.

Now that, we have the library of positive and negative words, can we write a simple algorithm to score a sentence into positive or negative?

Step 1: Split a sentence into independent words. W = {1…n } Step 2: For n = 1 to Number of Words and Initialise sentiment =0 Step 3: If the Current-Word:  W is in positive list add 1 to sentiment else if it is in negative list subtract 1 from sentiment or else do nothing(if the word is not present in either of the list, simply add 0) Now, let’s take the same example, Amazing = 1 Phone = 0 Don’t = 0 think = 0 Just= 0 Go= 0 for= 0 it = 0

So the sentiment = + 1, That is Positive. I guess this article helped you to understand the intuition behind how an algorithm can determine the sentiment of a sentence. Real world examples are more complex, and this simple rule will not take for enhancers and negations. For example, – “very good” should be treated differently when compared to “good” and likewise – “not good” should not be flagged as positive by the algorithm

When we draw in the relationship between words and bring in parts of speech to associate sentiments to multiple objects in the sentence, arriving at the sentiment gets more complicated! But as I said earlier this would be a good starting point for understanding sentiment analysis. I took a webinar on this topic on November 8, 2016. Click here to view the recording.

About the Author:

Deepak Murugaian is the Co-founder and CEO of EdisonLMS, an EdTech startup solving the Quality Education Crisis. He is an analytics professional, helping Fortune companies for deriving actionable insights, especially from unstructured data sources. Deepak also delivers training sessions for renowned research labs and prestigious academic Institutions. He holds a Masters degree in Artificial Intelligence from Heriot-Watt University and a Computer Engineering degree from Anna University.