A Simple Introduction To Natural Language Processing

Sandeep Raut

Today, with digitization of everything, 80% of the data being created is unstructured. Audio, video, our social footprints, the data generated from conversations between customer service reps, tons of legal documents, and texts processed in financial sectors are examples of unstructured data stored in Big Data. Organizations are turning to natural language processing (NLP) technology to derive understanding from the myriad unstructured data available online, in call logs, and in other sources.

NLP describes the ability of computers to understand human speech as it is spoken. NLP is a branch of artificial intelligence that has many important implications on the ways that computers and humans interact. Machine learning has helped computers parse the ambiguity of human language. Apache OpenNLP, Natural Language Toolkit (NLTK), and Stanford NLP are various open source NLP libraries used in real world applications.

Here are a few common ways NLP is being used today:

  • Spell check functionality in Microsoft Word is the most basic and well-known application.
  • Text analysis, also known as sentiment analytics, is a key use of NLP. Businesses can use it to learn how their customers feel emotionally and use that data to improve their service.
  • By using email filters to analyze the emails that flow through their servers, email providers can use Naive Bayes spam filtering to calculate the likelihood that an email is spam based its content.
  • Call center representatives often hear the same, specific complaints, questions, and problems from customers. Mining this data for sentiment can produce incredibly actionable intelligence that can be applied to product placement, messaging, design, or a range of other uses.
  • Google, Bing, and other search systems use NLP to extract terms from text to populate their indexes and parse search queries.
  • Google Translate applies machine translation technologies in not only translating words, but also in understanding the meaning of sentences to improve translations.
  • Financial markets use NLP by taking plain-text announcements and extracting the relevant info in a format that can be factored into making algorithmic trading decisions. For example, news of a merger between companies can have a big impact on trading decisions, and the speed at which the particulars of the merger (e.g., players, prices, who acquires who) can be incorporated into a trading algorithm can have profit implications in the millions of dollars.

Since the invention of the typewriter, the keyboard has been the king of the human-computer interface. But that’s changing today because of voice recognition via virtual assistants, such as Amazon’s Alexa, Google’s Now, Apple’s Siri, and Microsoft’s Cortana, which respond to vocal prompts to do things like finding a coffee shop, getting directions to an office, turning on lights, and switching the heat on, etc., depending on how digitized and wired our life is. IBM Watson is the most prominent example of question answering via information retrieval, already helping guide decisions in areas like healthcare, weather, insurance, etc.

From these examples and more, it’s clear that NLP has a very important role in new machine-human interfaces and will be an essential tool for leading-edge analytics in the near future.

Learn more about the pros and cons of artificial intelligence at work. See An AI Shares My Office. 

This article originally appeared in Simplified Analytics.