Lexical Analysis & Syntactic Analysis(Parsing)

In this post, I'm going to discuss two of the most important steps which need to be implemented whenever we deal with NLP. They are Lexical Analysis and Syntactic Analysis.

Lexical Analysis:

It basically is a stage in Natural Language Processing(NLP) where the given raw text is divided/segmented into different chunks of words or other units like paragraphs or sentences.

It involves identifying and analyzing the structure of words. Lexicon of a language basically means the collection of words and phrases in a language. It is also known as Morphological Analysis.

For eg:

" Tom owns an iPhone" could be tokenized into :

Tom

owns

an 

iPhone.

Syntactic Analysis(Parsing):

In simple words, this step does nothing but acts as a grammar checker. It analyses the words in the given sentence for grammar and also checks if the arrangement of different words are in a certain order which satisfies the relationship among the word.

What it basically does is it checks the basic grammar of the sentence.

For eg:

"The College goes to boy"

This given sentence is Syntactically wrong.

It identifies and analyzes the structure of words.

Lexicon of a language means the collection of words and phrases in a language