In its most basic sense, natural language processing can be defined as a machine’s ability to process something that is said, structure the received information, determine an appropriate response, and then respond as needed. Often abbreviated as NLP, natural language processing has been one of the most essential elements of computing science and programming dating back to its earliest days in the 1950s.
But what are the applications for natural language processing today? Obviously, the past seventy years have turned the field from one of dreams and hypotheticals to household uses that are familiar to us all. Google Translate, voice assistants, language learning programs - all are modern applications of NLP that have been extended into homes throughout the world.
However, these uses are just a sampling of the many broader uses of NLP that continue to emerge in a variety of tech sectors and market verticals. This guide will explore the basic history and development of natural language processing, and then review five of the major areas where NLP and its potential are particularly exciting and important.
What is Natural Language Processing? Where Did it Come From?
Famed British mathematician Alan Turing, in his landmark 1950 article “Computing Machinery and Intelligence”, first proposed the idea of measuring machine intelligence via a machine’s ability to “learn”. His method, known famously as the Turing Test, provided a set of metrics for programmers to measure and grow machine intelligence through a variety of factors.
Until the 1980s, natural language processing developed very slowly, as researchers encountered hurdle after hurdle (both technical and functional) in creating machines capable of actually performing all of the basic steps of NLP listed above. As researchers began to move away from fixed, “decision-tree” programming rules and toward probabilistic decision-making has led to the sequence of advancements leading to today’s voice assistants and other manifestations of “talking and listening machines”.
More recently, the past decade’s advances in deep learning and neural-network methods have allowed machines (with significantly more computing power than their predecessors) to achieve higher-level language modeling, parsing, and predictive capabilities. Because of these innovations, applications for NLP over the coming years are significantly broader than they ever have been before.
So what are the main applications for natural language processing in the coming years? Here are five of the areas that most industry specialists and researchers are focusing on.
From Siri to Cortana to “Ok Google”, voice assistants have gone from a science fiction dream to a regular part of everyday life in just a few short years. However, anyone using voice assistants frequently knows that while they work well in theory, there are still significant limitations that prevent these technologies from truly being the comprehensive “virtual assistant” that they are marketed as.
In terms of natural language processing, speech recognition offers perhaps the most exciting field of growth in terms of mass impact. Developing methods and frameworks for machines to go beyond recognizing simple commands and performing associated tasks and actually developing more genuine, human-like reactions in voice assistants is the next frontier of focus.
The potential in this field goes beyond simply setting reminders, adding items to shopping lists, or controlling the lighting and air conditioning. Improved voice assistants could give advice or “coach” in real-time, serve as a guide while wandering through a city or driving across country, and actively remember and personalize their commands to a broad array of users, beyond just the owner of the device. While some people are still wary of human-like voice assistants that rely on speech recognition technology, continued advancement and “humanizing” of these tools could make adoption grow exponentially and thus, make companies significantly higher revenue around selling these technologies.
Sentiment analysis (also known as opinion mining or emotion AI) is the technical term for developing machines capable of not just reading and understanding language, but also interpreting subtleties and hidden meanings behind it. The task of this analysis is to identify subjective information in the text. In plain terms, this is the difference between a machine simply reading a user’s review of a company and the machine understanding nuance (like satire, or irony) that is inherent in the review itself.
Why is this field so important over the coming years? For companies and organizations looking to get a better sense for user attitudes and desires towards goods and services, accurately interpreting sentiments is the only way to truly mine the data from hundreds of reviews for improving future operations and customer service. The challenge behind achieving truly efficient sentiment analysis is not just an issue of machine learning -- it is also attempting to teach machines a process that even real humans struggle with.
As social media has exploded and more people create their own “content platforms” via different accounts, the older methods of collecting customer opinion (surveys, polls, questionnaires) have become less reliable. However, people willingly share their opinions on social networks about everything from what they eat to what is happening in the political world. Companies striving to make the most of this free, user-offered information are in need of continued improvements in sentiment analysis to realize the benefits of the exponential growth of customer information accessible via social media platforms.
Text Labelling and Automatic Summarization
While most Internet users know the benefits of the ever-expanding amount of information readily accessible and available as tech use becomes ubiquitous, there’s a downside to this growth. Access to useful and relevant articles, research, studies, and sources can be hampered by current limitations (for example, Google algorithms that attempt to predict usefulness may leave out something that a user wants but doesn’t search specifically for).
That’s where text labeling and automatic summarization come in. By teaching machines to “read” long and complex articles (like studies in academic journals, for example), human users would be able to sort through abstracts or summaries that can reduce the expanse of information available to a more easily sorted format. The benefits of this are enormous -- faster research, easier vetting of sources and authors, quicker recall of essential information in a high-risk situation, and limitless potential in higher education are just a few examples of this.
At the moment, the limitations toward realizing truly effective text labeling and automatic summarization are similar to those in other fields of natural language processing -- the machines just are not good enough to recognize nuance, sentiment, or other subtle factors that may lead to misreading and miscategorization of the information. Until these problems are fixed, the fields remain promising but still limited in reliability and efficacy.
Another science fiction-esque vertical of natural language processing focuses on the potential for using machines for efficient, immediate translation between languages (both spoken and text/numeric). There are some obvious, clear upsides to this -- the ability to speak with anyone in the world in real-time, with translations being delivered right into your ear as the conversation happens being the most clear. But advanced language translation also offers the ability to quickly transfer the structure of a web application from one programming language to another, like switching from Python to Ruby.
So where does the potential lie for growing this technology in the near term? Many industry specialists are pointing investors toward funding research in complex translating models, tools that go beyond simply recognizing speech and translation after the fact, but that may have predictive components and draw cues from the environment and other information about the speaker being translated.
On the user-facing side, companies incorporating advanced NLP-powered translation can not only expand their services to a much broader array of users (regardless of nationality or language), but can also better personalize and humanize their customer service and user experience.
Question Answering via Machine Learning
Most recognizable through the proliferation of chatbots and other automated assistants, question answering is yet another field of natural language processing that many individuals have already experienced to varying degrees of usefulness and frustration. Machine-powered question answering is actually one of the older aspects of NLP -- chatbots and automated answering have been used for customer service and communications since the early days of widespread Internet use.
The benefits of these technologies for companies and organizations are plentiful -- from cutting down on staffing costs and training by automating customer service, to using machines as digital coaches or assistants to help people through installation or other processes, automated question-answering offers a responsive and comprehensive option for solving staffing or experience shortages and ensuring customers receive the help and info they need whenever they visit.
However, there is still more to be done in this field to grow towards the true science fiction image of digital workforces and support systems. Chatbots do not yet have the ability to recognize differences in customers aside from information being provided, meaning the technology can be limited if an insufficient amount of background info is provided. Similarly, chatbots have largely functioned on the “decision-tree” model that allowed them to exist in the first place. More innovation around neural networks and deep learning is needed to allow these tools to “think” more clearly for themselves and deliver a more personalized experience for customers and users.
Text-to-Speech (and Speech-to-Text)
Another area of exciting applications for natural language processing is talk-to-text functionality. We often think of programs that allow for dictation and speech commands as the clear example of this, but that is a result of current limitations.
More exciting would be the potential for advanced text-to-speech tools in education, rehabilitation, and workplace environments. From assisting students with reading and processing issues, to allowing individuals who have lost the ability to communicate easily because of physical mishaps, advanced text-to-speech tools would offer accommodations for allowing individuals to work, live, and process information as easily as those without disadvantages.
Growth in this area is largely inhibited by the factors already listed in other areas, as well as the additional limitations of clearly translating spoken word and audio files into text format without error. As anyone who has used voice-to-text on their phone now knows, the machines are not good enough yet to guarantee accuracy of translation and communication for sensitive communications or needs. Continued innovation in improving these tools is key to fully realize the potential benefits of the technology.
While natural language processing has moved significantly from the realm of science fiction into reality, there still remain significant technical and social hurdles preventing its widespread use, and from the larger population enjoying its benefits. Aside from the technological hurdles, a significant shortage of skilled programmers and specialists means that top companies are often deprived of talent and not able to innovate as quickly as they would like.
For further growth in the field as a whole, governments and private organizations need to continue driving investment into the field and rewarding promising ideas with future developmental funding and initiatives. Likewise, a greater pool of talent must be developed, starting with learning the basics of coding before moving to the more advanced fields of natural language processing. The future is bright for NLP, but significant actions are needed in the present to realize that future.