Natural Language Processing (NLP)

Natural Language Processing (NLP)
The Faculty of Industrial Engineering and Management
Spring Semester, 2015
3.5 credits


This course is a graduate introduction to natural language processing. It covers algorithms for the understanding of texts written in human language. Among the covered topics are: tagging tasks (part of speech and named entity), syntactic parsing and lexical semantics. The course will put an emphasis on machine learning algorithms for structured learning and probabilistic graphical models.

The goal of the course is to teach students how to:
• Understand approaches to syntax and semantics in NLP.
• Understand approaches to discourse, generation, dialogue and summarization within NLP.
• Understand current methods for statistical approaches to machine translation.
• Understand machine learning techniques used in NLP, including hidden Markov models, context-free grammars, unsupervised methods, log-linear and discriminative models.

This course is suitable to students who have taken the courses 094223 (Data Structures & Algorithms) and 094411 (Probability & Statistics) or courses 234218 (Data Structures) and 094412 (Probabilities).

Prof. Roi Reichart
Office: Blumfield Building, Rm. 313

Teaching Assistant (TA):


There will be three 50-minute lecture classes per week and one 50-minute recitation per week, led by a teaching assistant (TA).


• Daniel Jurafsky and James Martin, “Speen and Language Processing, 2nd edition,” Prentice-Hall, 2008
• Chris Manning and Hinrich Schutze, “Foundations of Statistical Natural Language Processing,” The MIT Press, 1999

Course Objectives:
At the end of the course the student will know:
1. To formulate the various tasks an algorithm should solve in order to gain understanding of a text.
2. To design and implement in software algorithms for markovian language models.
3. To design and implement in software algorithms for sequential tagging tasks such as part of speech (POS) tagging and named entity recognition (NER).
4. To design and implement in software algorithm for learning complex structures (such as trees) in order to solve problems such as statistical syntactic parsing (dependency as well as phrase structure parsing).
5. To design and implement algorithms that learn the semantics of words, phrases and sentences in the vector space modeling framework.
6. To design and implement dynamic programming algorithms, algorithms for inference in graphical models and parameter estimation algorithms in order to solve sentence and text structure learning problems.

The TA will lead the recitation section, and this time will be used to go over additional problems that are similar to the homework assignments. The TA will hold office hours to go over specific homework questions and questions about homework grading.

Course Expectations & Grading:
There will be 3 assignments during the semester. We take academic honesty extremely seriously, and expect the same of you. Copying of homework assignments will not be tolerated.
In addition, there will be reading assignments for some of the classes.

Homework is mandatory. The weight of each assignment is approx. 33% of the final grade.

Approximate Weekly Schedule:

Week Topic
1 Introduction to natural language processing, review of relevant statistical methods.

2 Markov network structures, methods for revaluation textual parameters.

3 Serial labeling tasks (tagging parts of speech, entities) using tuned graphical models such as Hidden Markov Model (HMM) (generative models).
4 Log-linear classifiers in discriminative statistical models with statistical methods in natural language processing.
5 Serial labeling tasks (tagging parts of speech, entities) using discriminative statistical models.
6 Unsupervised language learning methods with emphasis on serial labeling tasks (tagging parts of speech, entities).
7 Learning statistics syntax using context-free grammar models.
8 Algorithms for reevaluating parameters in discriminative statistical models with network structure; emphasis will be placed on perceptron algorithm.
9 Learning statistics syntax using a dependency grammar model.
10 Vector models to represent the meaning of words, phrases and sentences.
11 Vector models to represent the meaning of words, phrases and sentences – a combination of textual and visual information.
12 Vector models to represent the meaning of words, phrases and sentences – emphasis on expansion of models from the word-level to phrase and sentence levels.
13 Using vector models for semantics in other areas of language processing that were taught in the course like grammar acquisition and language models.

The strength of the university depends on academic and personal integrity. In this course, you must be honest and truthful. Ethical violations include cheating on exams, plagiarism, reuse of assignments, improper use of the Internet and electronic devices, unauthorized collaboration, alteration of graded assignments, forgery and falsification, lying, facilitating academic dishonesty, and unfair competition.