IntenCheck Blog

IntenCheck Blog

Fine Tuning Sentiment Analysis with Scala

(article from the magazine Voxx. Share the knowledge)

Take a look at Google Trends, and you’ll see a steep spike in interest in sentiment analysis in recent years, buoyed by the explosion of social media and blogging. Without a doubt, if you’ve looked into IBM’s Watson, you’ll have seen the term crop up among its many projects – but there are a number of smaller outfits also looking to perfect the technology. In this interview, Vadim Dekhtyar – President of specialist shop Intentex Ltd. – offers us a snapshot of where he perceives the current limitations lie, the challenges inherent in such analysis, and why his company chose to offer a Scala counter to Watson’s largely Java based backend.

Voxxed: What problems are you trying to solve at Intentex?

Dekhtyar: We have developed a unique text-analysis engine that can reveal the real intent of a text in just a few seconds. Our system will show you information about the preferred communication style of your respondent, and help you easily understand the intent, emotions and attitude hidden in the text you are reading or writing. It will even show if the text seems sincere or not.

We provide accurate, effective and highly innovative sentiment analysis features, which are unique on the market. Our lexicon-based sentiment engine can provide detailed analysis of texts, showing users what effects the words they use have on their readers in ways that by far surpass our competitors. Anyone who wants to communicate more effectively in English and get better results (later otherpopular languages as well, such as German, French, etc) can use our technology. Mainly sales, marketing and advertising specialists, PR specialists, journalists, politicians and speech writers, bloggers, government organisations and enterprises.

Why did you choose a Scala backend?

When we started working on Intencheck we were facing the task of processing significant amounts of text effectively. The most stable server side platform in our opinion is JVM, but task was still daunting. Fortunately with Scala you get all the advantages of functional design with full Java interoperability (when you need it).  Scala’s excellent collection framework allowed much simpler metaphors while designing our services.

Our backend is a set of REST services which are capable of processing high volumes of incoming text concurrently. Internally we use Natural Language Processing algorithms to parse incoming text and then we apply our sentiment analysis to it.  Finally, we compare our results with English norm text, compiled by our linguists.

Were there any particular challenges in building your software?

We had to design a very scalable system, which grows with amount of text and users organically. So, naturally, we were looking for cloud environment. We looked at many and, after weighing the pros and cons, we decided on using Pivotal CloudFoundry. It gives us unlimited scalability and operational agility, and, because deployment, provisioning and managements issues associated with most cloud environments are almost non-existent, it also allows our team to focus on delivering applications, .

What are the current capabilities of sentiment analysis software? Where do you see it going in the future?

According to latest research, the global text analytics market alone has a potential to reach $6.5  billion by 2020, and is predicted to register a growth rate of 25% during 2014-2020 period.  The market size in email communication alone is staggering: over 5.2 billion email accounts projected by end of 2018, and a projection of over 139.4 billion emails sent and received per day by 2018, by businesses alone.

Sentiment analysis is an emerging field that overlaps with many others such as business intelligence, customer service, and brand reputation management. Most types of sentiment  software use a technology known as text analytics, which extracts insight from text, such as in  social media, news articles, or internal documents and databases. The technology makes it  possible to get insights on unstructured text and helps companies formulate business strategies accordingly, or simply analyse consumer feelings.

What was it that drew you to NLP?

In school, we learn how to read, write, and do math. All of these skills of course are essential to our education. However, what we do not learn is how to communicate efficiently; in the real  world that ability is as important if not more so as the ability to write or calculate.

During the last twenty years I have been teaching NLP by holding seminars, doing business consulting, coaching, and private sessions. In reality, I have been decrypting the deep hidden meaning of people’s communication using my knowledge in Neuro Linguistic Programming (NLP) technology.

With the desire to show the real intent of the words we use, I decided to start analysing neuro-linguistic meanings of written texts. It was at first just a small project, but it grew bigger and bigger each year. Now, six years later, we have finished building our IntenCheck engine from the ground up, and what started out as a small project now become Intentex Ltd, a startup  founded by a team of experts across multiple fields, ranging from linguistics and research to business.

Can you take us on a walk-through your technology?

When used purposefully, our text analysis engine can be used to craft messages that have the desired effect on the audience. It can also be used in reverse, to analyse texts written by a person or released by a company/organization. This can result in useful insights about the person, company or organization, their communication preferences and how they affect their audience, or whether they are being truthful or attempting to keep something from public knowledge. The applications are too many to mention here, and we have provided case studies and white papers explaining them in more detail.

Matching language patterns: One of the main features that our software engine offers to its users refers to an idea first introduced in the field of neuro-linguistic programming: that of matching language patterns, particularly communication style (representational systems).

By being aware of a person’s language and then using their preferred communication style, we can build rapport with them faster and get our message across more clearly. It also makes it more likely to have our point of view accepted with less resistance. Essentially, matching language patterns helps us better understand others and how they think and leads to a greater ability to influence. The idea of matching elements of communication gives the user a clear idea about the direction of his/her communication.

Language norms and profiles: In order to be able to compare these elements of communication, our software uses language norms. These are generalized preferences of communication that are common for most people. Language norms can vary depending on the specific contexts – business, psychology, medical, scientific, etc.

Users can also respond to a specific person by using the email(s) they received from their respondent and our engine automatically compiles a unique language norm, which will be used in that case. These unique language norms can also be saved as profiles and improved upon later, by adding more examples of texts or emails written by the same person.                                                                                                                                                                     Lucy

 Editor of