Introduction of Natural Language Processing

An introduction to Natural Langauge Processing basics covering what is NLP, where it is used, what are the applications, and what are the approaches to create a NLP application.
TKTejas Khare24.00
May 03, 2021
Article

"Hey Siri, What's the weather now?" or "Okay Google, Where is the tallest mountain located?" are some of the many operations or questions you ask your smartphone. Siri, Google Assistant, Cortana, and Alexa are all virtual assistants powered by Natural Language Processing (NLP).

Every LG and Nokia phone going forward will have a Google Assistant button

All these assistants are the applications of NLP, which is the main highlight of this article, and of course Machine Learning. NLP is the understanding of natural text like a text in "English" by statistical methods and also with or without the help of scientific study of the language.

So, How does the machine learn or understand the text? 

A computer cannot understand our language and that is why there is a concept of machine language and human language. Did mystery solve right? Well, not exactly. The machine can understand our language but not directly. I will tell you how.

The text that you pass is are divided into computational representations which can be discrete or continuous, such as vectors, tensors, or tress. Google, Apple, Amazon, and many other virtual assistant providers use deep learning to get or understand those representations efficiently from a computational graph and numerical optimization techniques.

What are other uses of NLP, this can't be it?

Yes, of course, there are other many great and handy applications of NLP. And the best part is you are probably knowing most of them already because you unknowingly or knowingly use them every day. I will talk about two main applications of NLP that google provides us every day.

1. Autocorrect or spell check

How to write your own spellchecker and autocorrect algorithm in under 80  lines of code | by Will Sentance | Medium

You must have typed some random word (which is not actually a legit word) and your keyboard or google shows suggestions of the correct word. This is basically an application of NLP. So how does this work?

Basically, the tech giant Google has its own probability model which firstly understands the representation of the incorrectly spelled word, generates several closest representations of the input, and finally, converts them into human language, in the boe image - English.

You also must have seen when you type a non-English word on an English keyboard, it may not give suggestions. But after several uses, the generation or suggestion part becomes smarter. "Smarter" in technical words is nothing but the probability algorithm now has a wide variety of data to use for generation.

2.  Autocomplete

How Google autocomplete works in Search

Google claims that autocomplete reduces typing by 25% and I think it's very accurate. The sentences that you see in the image above are not suggestions but actually, they are the predictions from google's autocorrect API.

How does this magic work?

Though a magician never reveals his/her secret, google has actually published an entire document on how this works. Everything that we type on the google search bar is stored in their database. Furthermore, after we type something which is generally called a 'query', google also records which link we go into. The reason it stores the link is that it considers the link as an answer to that query. There can be multiple answers or links to one question and hence all are stored. Finally, on which link the time spent is more is calculated and it is considered as the best match or best answer for your query.

From this process, the probability algorithm learns by matching the representations and predicts multiple sentences like in the above image.

Okay, so let's get to the technical part that is exactly what is used for the machine to understand those representations that I was talking about repeatedly.

10 Things We Know About the Science of Meditation - Mindful

These are the approaches being used to develop these applications and constantly amaze people like you and me.

1. Rule-Based Algorithms

This is the most basic way of developing simple NLP applications. For example, by simple, I mean a chatbot with constraints on the number of questions asked and the way they are asked. Furthermore, this is mainly used in extracting structured data like applications that are used for finding a particular word from a text or even extracting an entire web page.

Let us understand with a simple code -

dict1 = {"Hi":"\n--> Hello, I am a simple chatbot. How can I help you with?", 
         "What is the weather now?":"\n--> It is 20 degrees outside so a good day for a stroll", 
         "Bye":"\n-->Nice talking to you bye bye."}
q = input("Enter your text: ")

Here we just defined a dictionary where the key represents a question and value represents its answer. Then we take input as a string.

while True:    
    if q == 'Hi':
        print(dict1['Hi'])
        q1 = input("Enter your question: ")
        if q1 == "What is the weather now?":
            print(dict1['What is the weather now?'])
        elif q1 == 'Bye':
            print(dict1['Bye'])
            break
        else:
            print("\n-->Command not recognized")
    else:
        print("Command not recognized")
        q1 = input("Enter your question: ")
        if q1 == 'Bye':
            print(dict1['Bye'])
            break
        else:    
            print("\n--> Command not recognized")

Now, in the above code, we just traverse into an infinite loop in which I have defined some set of rules. As you can see, there are many 'if' and 'else' statements, and when the input matches as per the key in the dictionary, you get the answer.

The loop will exit when I say "Bye", else the loop will continue infinitely.

But this method is highly vague and basically is hardcoding. Just imagine a virtual assistant asking you to only ask specific questions. That's unacceptable right!

Hence, this method is used for only specific tasks which include structured data.

2. Machine Learning approach

The Machine Learning approach eliminates the constraints of the rule-based approach that is it can work on unstructured data. Basically, what this approach does is first tokenizes the sentences. Tokenization is nothing but dividing a paragraph or a sentence into multiple sentences or words respectively. The tokenized output is then vectorized and sent to extract features like word occurrence, length of the word, length of sentence, etc. These features are then sent to a statistical machine learning model to learn the different patterns and finally predict the results as per the requirement.

3. Deep Learning approach

The machine learning approach performs way better than the normal rule-based one. But take this, the deep learning approach is by far the best approach for accurate and real predictions. You don't need to specify the features as it automatically extracts the features. This approach consists of complex neural network architectures like BERT which is one of the best text understanding models, GPT-3 which generates text with arbitrary input. These are state-of-the-art models trained on large corpora.

The potential of Natural Language Processing is limited only to one's own imagination

There it is guys, I hope you found the article interesting and it opened the doors for you to the NLP world. Thank you for reading the article. Cheers :)

3 votes
nlpbertgpt
How helpful was this page?