Components of NLP

Hello everyone!!

In this post I am  going to explain what is NLU and NLG and how three of them works together also techniques used in NLP.

1. NLU - Natural Language Understanding is a smaller part of natural language processing. Once the language has been broken down, it’s time for the program to understand, find meaning, and even perform sentiment analysis.It mainly includes following steps:

• Mapping the given input in natural language into some useful representation.
• Analyzing different aspects of language.

NLU understand every meaning behind sentence formation. It is quite possible that the same text has different meaning whereas different word has same meaning. Also most of the times meaning behind the text changes according to the situation. It uses sentiment detection at such cases then.

 2. NLG - Natural Language Generation is what happens       when computer write language. NLG process turn structured  data into text.

  It mainly includes following steps:

• Text planning - It includes retrieving the relevant content from knowledge base.
• Sentence planning - It includes some steps such as choosing required words , forming meaningful phrases  and also setting tone of the sentence.
• Text realization - It is used for mapping meaningful sentences.

How NLP , NLU and NLG work in hand in hand ?

As u can see in figure , first the input is given and then with the help of automatic speech recognition software speech is converted into text and then passes to NLU. NLU understand the text and convert into structured data. It passes this as input to NLG which turns structured data into text and write information in human language and we get output.

Techniques in NLP

1. Bag of words - It allows you to count no of words in text by creating an occurrence matrix for the sentences.

2.Tokenization - Process of dividing text into set of meaningful pieces called tokens. Also it removes punctuation too.

3. Stop words removal -It includes getting rid of human language articles ,pronouns ,prepositions as "and","the" ,"or" in English.

4. Stemming - It is the process of removing common affixes i.e cutting end or beginning of word.

5 Lemmatization - It has intentions to reduce a word to its base form i.e root form and also combines different combines forms of word.

6 Part of speech - Grammatical type of words is referred to as POS tags. It indicates how a word functions in meaning as well as grammatically into the sentence. 

7. Named entity recognition - A word can have more than one POS . So it removes such kinds of ambiguity. It includes noun phase identification , phrase identification, entity disambiguation.

Thanks for reading my article.
Stay tuned for next aspects of NLP.
By Ashwini Ghode