Natural Language Processing · TEE · papers
Q7b · Named entity recognition
2025-02-21 · re-exam · recovered AI paper
Previous examination · complete worked answers
Answer
Named Entity Recognition (NER) identifies text spans referring to entities and assigns their types, such as PERSON, ORGANIZATION and LOCATION. Many practical schemas also include DATE, TIME, MONEY or domain-specific types. The exact allowed labels depend on the application's annotation scheme.
NER has two parts: boundary detection finds where the entity starts and ends; classification assigns the appropriate type. A multiword name should normally be one entity span: “New York” is a location span, not two unrelated location decisions.
Not every noun is a named entity. In “a developer works at Google”, developer is an ordinary job description while Google is a named organization. A specialized HR system may define a JOB_TITLE category, but it must state that schema explicitly. NER identifies mentions; entity linking is the separate task of connecting a mention to a unique database identity.
| Task | Question it answers | Example |
|---|---|---|
| POS | What grammatical role does this token have? | Google/NNP |
| Chunking | Which adjacent words form a phrase? | [NP a skilled developer] |
| NER | Which span names an entity, and of what type? | [Google] ORGANIZATION |
| Entity linking | Which real entity does this mention denote? | Connect Paris to the intended city or person record |
Applications. NER structures search results and documents, extracts organizations/people from news, routes support tickets by product or organization, supports recommendation, and extracts specialized entities from healthcare or legal text. Context is essential: “Apple released a laptop” names an organization, while “she ate an apple” describes fruit.
Yaad rakho: NER mein do kaam: naam kahan se kahan tak, aur kis category ka.
| Approach | How it works | Example | Main limitation |
|---|---|---|---|
| Dictionary / gazetteer | Match text against lists of known names | A city list includes Mumbai and Pune | New names, spelling variants and ambiguous names are missed |
| Rule-based | Use word shapes, regular expressions and context | A title such as Dr. followed by a name-like span suggests PERSON | Manual maintenance; exceptions and domain/language dependence |
| Machine learning | Learn a labeling rule from annotated examples using features | Capitalization, suffixes, POS and surrounding words help a classifier/CRF | Requires labeled data and useful features |
| Deep learning | Learn contextual word/character representations and predict labels | A contextual sequence encoder labels multiword entity spans | Training/domain dependence and resource demands |
A pattern-based rule examines the form of text, such as an email pattern or a date format. A context-based rule examines neighboring words: “Dr.” can support a following PERSON span, while “headquartered in” can support a following LOCATION span. Capitalization alone is insufficient because sentence-initial common nouns are also capitalized.
A dictionary may find “May” as a person's name, but “in May” may refer to a month. A learned model uses surrounding evidence to disambiguate. Hybrid systems can combine dictionaries, rules and learned models; these categories are practical tools rather than mutually exclusive restrictions.
Preprocessing for NER. Tokenize while preserving token offsets and useful capitalization. Blindly removing capital letters or stopwords before recognition can damage names such as “Bank of India”. NER output should preserve the original span so the reader can locate it in the input.
Exam-ready 3-mark answer. NER locates named-entity spans and classifies them into categories such as person, organization and location. A dictionary-based approach matches known name lists; a rule-based approach uses patterns or context; machine-learning and deep-learning approaches learn entity decisions from labeled data and representations. For example, in “Mary works at Google in Mumbai”, Mary is PERSON, Google is ORGANIZATION and Mumbai is LOCATION.