Introduction to Artificial Intelligence - Old Questions

10.  Explain the steps of Natural Language Processing. 

Natural language processing is a technology which involves converting spoken or written human language into a form which can be processed by computers, and vice versa. NLP is composed of two part: NLU (Natural Language Understanding) and NLG (Natural Language generation).

Steps of Natural Language Processing (NLP)

Input/Source

• The input of a NLP system can be written text or speech.

• Quality of input decides the possible errors in language processing that is high quality input leads to correct language understanding.

Segmentation

• The text inputs are divided in segments (Chunks) and the segments are analyzed. Each such chunk is called frames.

Syntactic Analysis

• Syntactic analysis takes an input sentence and produces a representation of its grammatical structure.
• A grammar describes the valid parts of speech of a language and how to combine them into phrases.
• The grammar of English is nearly context free.

Grammar: A computer grammar specifies which sentences are in a language and their parse trees. A parse tree is a hierarchical structure that shows how the grammar applies to the input. Each level of the tree corresponds to the application of one grammar rule.

Parse Tree:

Semantic Analysis

• Semantic analysis is a process of converting the syntactic representations into a meaning representation.
• This involves the following tasks:

                    - Word sense determination

                    -  Sentence level analysis

Word sense: Words have different meanings in different contexts.

Example:Mary had a bat in her office.

        bat = ``a baseball thing’

        bat = ``a flying mammal’

Sentence Level Meaning:

Once the words are understood, the sentence must be assigned some meaning

Examples:

• She saw her duck.
• I saw a man with a telescope.

Non-examples: Colorless green ideas sleep furiously - >This would be rejected semantically as colorless green would make no sense.

Pragmatic Analysis

• Pragmatics comprises aspects of meaning that depend upon the context or upon facts about real world.
• These aspects include:

                - Pronouns and referring expressions.

                - Logical inferences, that can be drawn from the meanings of a set of propositions.

                - Discourse structure: the meaning of a collection of sentences taken together.

Examples:

Jack fell. Jill brought him a band-aid.

        Jack got hurt and Jill wanted to help.

We got seven letters today

        We got only seven letters (and not eight).