{"edition":"nlp-2026-10-10","routes":{"m2":{"read":[{"id":"foundations","title":"Start here · language & probability","unit":0,"sections":[{"id":"language","title":"What NLP is doing","html":"

Natural Language Processing (NLP) is the study of computational methods for analysing, interpreting and producing human language. An NLP system receives text or speech and produces an output such as a tag, a meaning, a translation or a probability.

For M2, the central question is often: how do counts from a training corpus become probabilities, and how do those probabilities help choose a sentence, a tag or a word sense? You will learn the vocabulary first, then do the arithmetic with complete working.

Training textexamples already observed
→
Countshow often things occur
→
Probabilitiescounts divided by totals
→
Decisioncompare possible answers
TermMeaningConcrete example
CorpusA collection of language data. The plural is corpora.Three sentences saved together form a tiny corpus.
DocumentOne text item within a collection.One review or one article.
TokenOne occurrence produced by tokenization.“cats like cats” has 3 word tokens.
TypeA distinct token value.The same text has 2 types: cats and like.
VocabularyThe set of token types recognised by the model.V = {cats, like}; vocabulary size is 2.
TrainingEstimating rules or numbers using examples.Count how often like follows cats.
TestApplying the learned model to a new example.Calculate a probability for a sentence not used to build the counts.

Three levels you will repeatedly see

LevelQuestion it answersExample
Word sequenceWhich word is likely next?After “drink”, is “water” likely?
SyntaxWhat role or structure does the word have?“book” is a noun in “a book” and a verb in “book tickets”.
SemanticsWhich meaning is intended?“bank” near “deposit” means a financial institution.

Yaad rakho: Word ka order = language model; word ka role = POS; word ka matlab = sense.

Your M2 route

Read the probability refresher below, then N-grams and smoothing → POS/HMM → parsing/chunking/NER → word relations and WSD. Answers sit directly below their questions. The separate Formulas tab is for revision after learning the methods. Both 2-mark questions are compulsory; attempt any two of the three 3-mark questions. Total attempted = 10 marks in one hour.

M2 scope: Unit 3 slides 62–160; Unit 4 slides 1–128; Unit 5 slides 1–57. Earlier-unit material in this short opening chapter is included only to supply prerequisites.

"},{"id":"notation","title":"Tokens, counts and symbols","html":"

A formula is a short way of stating a calculation. First name the quantities; only then insert the numbers. In this guide, C means a count and P means a probability.

NotationRead it asWhat to do
\"C(\\text{cat})=3\"Count of cat is threeCount every occurrence, including repeats.
\"P(\\text{cat})\"Probability of catA number from 0 to 1.
\"P(\\text{cat}\\mid\\text{the})\"Probability of cat given theRestrict attention to occasions where the history is “the”.
\"\\sum_iSum of the x valuesAdd the terms: x₁ + x₂ + …
\"\\prod_iProduct of the x valuesMultiply the terms: x₁ × x₂ × …
\"\\arg\\max_cThe choice c with the largest scoreReturn a label, not the numeric score.
\"\\max_cLargest scoreReturn a number.

Example 1 · tokens and types

For the corpus “we study nlp” and “we like nlp”, find the token count, vocabulary and count of “we”.

Given

Two lowercase sentences. Ignore punctuation; do not add boundary markers.

To find

Token count N, vocabulary V, vocabulary size |V| and C(we).

Formula and why it applies

A token is an occurrence; a type is a distinct word. Add sentence lengths to count tokens; remove duplicates only when finding the vocabulary.

Solution

Step 1. List every occurrence

we, study, nlp, we, like, nlp.

Step 2. Count tokens

\"N=3+3=6\"

Step 3. Keep one copy of each type

V = {we, study, nlp, like}; |V| = 4.

Step 4. Count the requested word

“we” appears once in each sentence, hence C(we) = 2.

N = 6; |V| = 4; C(we) = 2.

Yaad rakho: Token mein repeat gin; type mein duplicate hata.

Example 2 · sum and product

Given probabilities 0.5, 0.2 and 0.1, compute their sum and their product.

Given

Three numbers: 0.5, 0.2, 0.1.

To find

Sum and product; recognise that these are different operations.

Formula and why it applies

\"\\sum_{i=1}^{3}p_i=p_1+p_2+p_3\"
\"\\prod_{i=1}^{3}p_i=p_1p_2p_3\"

Add for a sum. Multiply for a product. A sentence-chain calculation normally uses the product of its conditional probabilities.

Solution

Step 1. Add the first two numbers

\"0.5+0.2=0.7\"

Step 2. Complete the sum

\"0.7+0.1=0.8\"

Step 3. Multiply the first two numbers

\"0.5\\times0.2=0.1\"

Step 4. Complete the product

\"0.1\\times0.1=0.01\"
Sum = 0.8; product = 0.01.
"},{"id":"probability","title":"Probability, without skipped steps","html":"

Empirical probability estimates how likely an event is from its observed frequency. Divide the number of relevant observations by the total number of observations in the selected group. For a conditional probability, the denominator is the count of the condition, not the size of the entire corpus.

\"P(A)=\\frac{C(A)}{N}\"
\"P(B\\mid

Here A is the conditioning event, B is the event being predicted, C(A,B) counts observations containing both in the required relationship, and N is the total number of observations. “Given A” tells you which denominator to use.

Select the condition“given A”
→
Count matching outcomesC(A,B)
→
Divide by condition countC(A)

Example 1 · word probability

A corpus contains 20 tokens; “rain” occurs 5 times. Find the unigram probability of “rain”.

Given

N = 20 and C(rain) = 5.

To find

P(rain).

Formula and why it applies

\"P(w)=\\frac{C(w)}{N}\"

Unigram probability ignores neighbouring words. Every token in the corpus is one observation.

Solution

Step 1. Insert the count and total

\"P(\\text{rain})=\\frac{5}{20}\"

Step 2. Divide

\"\\frac{5}{20}=0.25\"
P(rain) = 0.25, or 25%.

Example 2 · conditional probability

In a corpus, “drink” is followed by another word 8 times. It is followed by “water” 6 times. Find P(water | drink).

Given

C(drink as a history) = 8; C(drink, water) = 6.

To find

Probability that the next word is water, given drink.

Formula and why it applies

\"P(\\text{water}\\mid\\text{drink})=\\frac{C(\\text{drink

The condition is drink, so use the 8 drink-histories as the denominator.

Solution

Step 1. Identify the favourable continuations

Six of the eight continuations are water.

Step 2. Divide

\"P(\\text{water}\\mid\\text{drink})=\\frac68=0.75\"
P(water | drink) = 0.75.

Yaad rakho: Bar ke right mein jo hai, denominator uska count.

Example 3 · multiply a sequence

A model gives P(a | start) = 0.4, P(cat | a) = 0.5 and P(end | cat) = 0.2. Find the probability of the complete sentence “a cat”.

Given

Start → a = 0.4; a → cat = 0.5; cat → end = 0.2.

To find

Probability of the three-transition path.

Formula and why it applies

\"P(\\text{a

Every transition must happen in sequence, so multiply the model’s conditional probabilities.

Solution

Step 1. Write the three factors in order

\"P=0.4\\times0.5\\times0.2\"

Step 2. Multiply the first pair

\"0.4\\times0.5=0.2\"

Step 3. Multiply by the last factor

\"0.2\\times0.2=0.04\"
Complete-sentence probability = 0.04.

Example 4 · compare labels

Two tag candidates have scores noun = 0.036 and verb = 0.048. Which tag does argmax return? If these are the only candidates, find the normalised noun probability.

Given

s(noun) = 0.036 and s(verb) = 0.048.

To find

Winning label and normalised noun probability.

Formula and why it applies

\"\\hat
\"P(c)=\\frac{s(c)}{\\sum_j

A score need not sum to 1. Dividing each score by the sum gives a distribution over the candidates.

Solution

Step 1. Compare scores

0.048 > 0.036, so verb wins.

Step 2. Find their total

\"0.036+0.048=0.084\"

Step 3. Divide the noun score by the total

\"P(\\text{noun})=\\frac{0.036}{0.084}=\\frac37\\approx0.428571\"
argmax returns “verb”. The normalised noun probability is approximately 0.428571.
"},{"id":"bayes","title":"Bayes’ rule: prior, evidence and answer","html":"

Bayes’ rule reverses a conditional probability. A word-sense classifier asks “which sense is most likely given these context words?” Training data more easily supplies “how likely are these words when this sense is used?” Bayes connects the two.

\"P(c\\mid
SymbolMeaning
cA candidate class, tag or word sense.
xThe observed evidence, such as a context word.
P(c)Prior: how common the class is before seeing the evidence.
P(x | c)Likelihood: how compatible the evidence is with that class.
P(c | x)Posterior: probability of the class after observing the evidence.
P(x)Evidence normaliser, shared by all classes for the same observation.

For ranking classes, calculate prior × likelihood. The common denominator does not affect which class is largest. For an actual posterior probability, divide by the sum of all class scores.

A complete Bayes calculation

A word has two senses: financial bank and river bank. Priors are 0.6 and 0.4. The probability of observing “loan” is 0.5 under the financial sense and 0.1 under the river sense. Find the most likely sense and its posterior.

Given

P(F) = 0.6; P(R) = 0.4; P(loan | F) = 0.5; P(loan | R) = 0.1. These two senses are exhaustive.

To find

Scores and P(F | loan).

Formula and why it applies

\"s(c)=P(c)P(\\text{loan}\\mid
\"P(F\\mid\\text{loan})=\\frac{s(F)}{s(F)+s(R)}\"

Multiply prior and likelihood to reverse the conditioning. Then normalise because a posterior was requested.

Solution

Step 1. Score the financial sense

\"s(F)=0.6\\times0.5=0.30\"

Step 2. Score the river sense

\"s(R)=0.4\\times0.1=0.04\"

Step 3. Add the scores

\"P(\\text{loan})=0.30+0.04=0.34\"

Step 4. Normalise the financial score

\"P(F\\mid\\text{loan})=\\frac{0.30}{0.34}=\\frac{15}{17}\\approx0.882353\"
Financial bank wins; its posterior probability is about 0.882353.

Yaad rakho: Pehle se kitna common × evidence kitna suit karta = score.

"},{"id":"calculator","title":"Small probabilities, logs and calculator use","html":"

Repeated multiplication of probabilities produces small numbers. A calculator may show 0.0000125 as 1.25 × 10⁻⁵; they are the same value. A logarithm turns a product into a sum. Log probabilities are often negative because probabilities between 0 and 1 have negative logs.

\"\\log(ab)=\\log
\"\\log_b

Use the log base specified in the question. Natural log is ln (base e); ordinary calculator log is usually base 10. Perplexity is simplest to evaluate using the direct power formula. Do not mix bases in the same calculation.

Calculator example · roots in perplexity

A model assigns probability 1/16 to a test sequence with 4 predicted tokens. Calculate perplexity.

Given

P = 1/16; N = 4 predicted tokens.

To find

Perplexity PP.

Formula and why it applies

\"PP=P^{-1/N}=\\left(\\frac1P\\right)^{1/N}\"

Perplexity takes the reciprocal probability and then its Nth root. N must match the number of predicted events in the product.

Solution

Step 1. Take the reciprocal

\"\\frac{1}{1/16}=16\"

Step 2. Take the fourth root

\"PP=16^{1/4}=2\"

Step 3. Interpret

A perplexity of 2 means an effective average uncertainty comparable to two equally likely alternatives. It is not 2% error.

PP = 2.

Calculator entry order

Enter the full product with parentheses. For the Nth root enter (1 ÷ P)^(1 ÷ N). Keep at least 6 significant digits while working; round the final answer. If your calculator offers Ans, reuse the stored value instead of typing a heavily rounded result.

Yaad rakho: Probability ke factors multiply; log mein add; perplexity mein ulta karke root.

"}],"scope":"Prerequisites for M2 · no earlier NLP knowledge assumed"},{"id":"m2-language-models","title":"N-grams and language models","unit":3,"scope":"Unit 3, pages 62–108; essential probability revision","sections":[{"id":"words-counts-probability","title":"Words, counts and probabilities: the notation you need","html":"

A token is one occurrence of a word. A type is one distinct word. A corpus is a collection of texts used for analysis or training. In “cats chase cats”, there are three word tokens but only two word types: cats and chase. A language model learns regularities from such a corpus to assign probabilities to word sequences. It does not simply count whether a sentence is grammatically correct.

SymbolRead it asConcrete meaning
C(cat)count of catNumber of times cat occurs.
C(the, cat)count of the followed by catCount adjacent ordered pairs; cat the is different.
P(cat)probability of catA number from 0 to 1.
P(cat | the)probability of cat given theAmong contexts ending in the, how often cat follows.
Vvocabulary sizeNumber of distinct outcomes the model may predict.
Nnumber of tokens or predictionsIts exact meaning must be stated for the calculation.

The vertical bar | means “given”. The condition is on its right. In P(cat | the), the model has already seen “the” and is predicting “cat”. This direction matters: P(cat | the) and P(the | cat) usually have different denominators.

Question. A corpus contains 20 word tokens. The word cat occurs 5 times. Find the maximum-likelihood unigram probability of cat.

Given

Total tokens N = 20; count C(cat) = 5.

To find

P(cat).

Formula and why it applies

\"P(w)=\\frac{C(w)}{N}\"

Relative frequency estimates the probability of drawing this word from the corpus. The denominator includes every token, not just distinct words.

Solution

Step 1. Select the matching count

The required outcome is cat, which occurs 5 times.

Step 2. Divide by the total number of trials
\"P(\\text{cat})=\\frac{5}{20}=0.25\"
Step 3. Interpret the result

Under this unigram model, cat receives probability 0.25, or 25%.

Final answer. P(cat) = 0.25.

Yaad rakho: Jo chahiye uska count upar; total possible trials neeche.

Question. The word the has 10 outgoing occurrences in a corpus. It is followed by cat on 4 occasions and dog on 6. Find P(cat | the) and P(dog | the).

Given

C(the, cat) = 4; C(the, dog) = 6; C(the) = 10 outgoing contexts.

To find

Both next-word probabilities.

Formula and why it applies

\"P(w\\mid

The condition h = the restricts attention to its 10 continuations. We do not divide by the total corpus size.

Solution

Step 1. Use the cat continuation
\"P(\\text{cat}\\mid\\text{the})=\\frac{4}{10}=0.4\"
Step 2. Use the dog continuation
\"P(\\text{dog}\\mid\\text{the})=\\frac{6}{10}=0.6\"
Step 3. Check the complete row
\"0.4+0.6=1\"
Final answer. P(cat | the) = 0.4 and P(dog | the) = 0.6.

Yaad rakho: Given waala word denominator choose karata hai.

Question. A collection has 8 sentences; 6 start with I and 2 start with we. Estimate each starting-word probability.

Given

Each sentence receives a start marker <s>. C(<s>) = 8; C(<s>, I) = 6; C(<s>, we) = 2.

To find

P(I | <s>) and P(we | <s>).

Formula and why it applies

\"P(w_1\\mid\\langle

The start marker supplies the context for the first word; one start context is available per sentence.

Solution

Step 1. Calculate the first option
\"P(I\\mid\\langle
Step 2. Calculate the second option
\"P(we\\mid\\langle
Final answer. The estimated probabilities are 0.75 and 0.25.
"},{"id":"n-grams-construction","title":"Unigrams, bigrams and trigrams: construct them correctly","html":"

Definition. An n-gram is a contiguous sequence of n items. For word n-grams, the items are consecutive word tokens. A unigram contains one word; a bigram contains two; a trigram contains three. Character n-grams use characters instead, so always identify the unit being counted.

Tokens: we | study | natural | language
→
Move a window one token at a time
→
Bigrams: we study / study natural / natural language

For m tokens without padding, the number of n-gram positions is max(0, m−n+1). Repeated positions may contain the same n-gram type. A bag-of-n-grams representation counts each n-gram type and loses the overall sentence order beyond the local order inside each n-gram.

\"\\text{Number

Question. List all word unigrams, bigrams and trigrams for “the prime minister of our country”. Do not add boundary tokens.

Given

Six tokens: the, prime, minister, of, our, country. n = 1, 2 and 3.

To find

All three lists and their counts.

Formula and why it applies

\"\\#\\text{positions}=m-n+1\"

Each window must contain consecutive words and move one position. No wraparound or invented final word is allowed.

Solution

Step 1. Take one word at a time

Unigrams: (the), (prime), (minister), (of), (our), (country). Count = 6−1+1 = 6.

Step 2. Take two consecutive words

Bigrams: (the, prime), (prime, minister), (minister, of), (of, our), (our, country). Count = 6−2+1 = 5.

Step 3. Take three consecutive words

Trigrams: (the, prime, minister), (prime, minister, of), (minister, of, our), (of, our, country). Count = 6−3+1 = 4.

Final answer. 6 unigrams, 5 bigrams and 4 trigrams.

Question. Construct bigrams for “I like tea” using one start and one end marker.

Given

Original tokens: I, like, tea. Add <s> and </s> once each.

To find

The boundary-aware bigram sequence.

Formula and why it applies

\"\\#\\text{bigrams}=(m+2)-1=m+1\"

The start context predicts the first word, and the last word predicts that the sentence ends.

Solution

Step 1. Write the padded sequence

<s> I like tea </s>

Step 2. Move a two-token window

(<s>, I), (I, like), (like, tea), (tea, </s>).

Step 3. Count the windows
\"3+1=4\"
Final answer. There are four boundary-aware bigrams.

Question. Construct trigrams for “I like tea” using two start markers and one end marker.

Given

Original sentence has three words. For a trigram model each predicted token needs two prior context tokens.

To find

All trigram prediction events.

Formula and why it applies

\"P(w_i\\mid

Two start markers provide a complete context even for the first word.

Solution

Step 1. Pad with two starts

<s> <s> I like tea </s>

Step 2. Slide a three-token window

(<s>, <s>, I), (<s>, I, like), (I, like, tea), (like, tea, </s>).

Step 3. Count predicted events

The model predicts I, like, tea and the end marker: four predictions.

Final answer. Four trigrams. Start markers are context; they are not ordinary predicted words.

Question. Give the bag-of-bigrams counts for “a b a b” without padding.

Given

Four tokens in their given order.

To find

Bigram positions, distinct bigram types and their counts.

Formula and why it applies

\"\\#\\text{positions}=4-2+1=3\"

A bag stores frequencies; repeated bigrams must not be discarded.

Solution

Step 1. Enumerate all positions

(a,b), (b,a), (a,b).

Step 2. Group identical pairs
Bigram typeCount
a b2
b a1
Final answer. Three bigram positions but two distinct bigram types.

Yaad rakho: n-gram mein n words; context mein n−1 words.

"},{"id":"collocations-and-lm","title":"Collocations and the purpose of a language model","html":"

A collocation is a conventional word combination whose words occur together more strongly than ordinary chance or independent frequencies would suggest. Examples include “strong tea”, “heavy rain” and “red wine”. A bigram is merely two consecutive items; it need not be a collocation. “of the” can be frequent because both words are frequent, while a collocation expresses a particularly strong association.

TermWhat it representsExample
BigramAny adjacent two-token sequencethe book
CollocationA characteristic strongly associated expressionheavy rain
Language modelA distribution used to score or predict sequencesP(rain | heavy)

Definition. A statistical language model assigns a probability to a sequence of words, or equivalently predicts the next word from preceding context. It is useful for autocomplete, speech recognition, machine translation, spelling correction, assistive communication and ranking candidate text. High probability reflects patterns learned from its training distribution; it is not a guarantee of truth or grammatical correctness.

Previous words: please turn your homework
→
Candidate: in / over / elephant
→
Language model gives each candidate a probability
→
Application selects or ranks candidates

Noisy-channel connection. Suppose x is the observed noisy text or speech and y is a candidate intended message. The noisy-channel approach chooses the y that maximizes P(x | y)P(y). The channel model P(x | y) asks how y could produce x; the language model P(y) asks whether y is a plausible message. P(x) is the same for all candidate y and therefore can be omitted when ranking.

\"\\hat

Question. A spelling-correction system considers “flowers” and “flours”. For the observed input, the channel likelihoods are 0.2 and 0.4, and the respective language-model probabilities in context are 0.08 and 0.01. Which candidate does the noisy-channel rule choose?

Given

Candidate flowers: P(x|flowers)=0.2, P(flowers)=0.08. Candidate flours: P(x|flours)=0.4, P(flours)=0.01.

To find

The larger unnormalized candidate score.

Formula and why it applies

\"\\operatorname{score}(y)=P(x\\mid

Both the error mechanism and the language context matter. The same denominator would normalize both candidates.

Solution

Step 1. Score flowers
\"0.2\\times0.08=0.016\"
Step 2. Score flours
\"0.4\\times0.01=0.004\"
Step 3. Compare
\"0.016>0.004\"
Final answer. Choose flowers. The scores are not yet posterior probabilities.

Yaad rakho: Har word-pair bigram hai; har frequent pair meaningful collocation nahi.

"},{"id":"chain-rule-markov","title":"Probability chain rule and the Markov assumption","html":"

The probability chain rule decomposes the probability of a complete sequence into successive conditional probabilities. It is different from the derivative chain rule used in deep learning. Here we multiply probabilities, not derivatives.

\"P(w_1,w_2,w_3)=P(w_1)P(w_2\\mid

Read left to right: probability of the first word; then probability of the second after the first; then probability of the third after both previous words. For a longer sentence, continue the same pattern. The product symbol ∏ simply means “multiply all of these terms”.

\"P(w_1,\\ldots,w_m)=\\prod_{i=1}^{m}P(w_i\\mid

The Markov assumption approximates long history by a limited recent context. A unigram model ignores preceding words. A bigram model keeps the previous word. A trigram model keeps the previous two words. This reduces the number of counts needed, but loses longer-range dependencies.

ModelPrediction ruleFor the next word country after the prime minister of our
UnigramP(wᵢ)P(country)
BigramP(wᵢ | wᵢ₋₁)P(country | our)
TrigramP(wᵢ | wᵢ₋₂,wᵢ₋₁)P(country | of, our)
Full chain ruleP(wᵢ | all previous words)P(country | the, prime, minister, of, our)

Question. For the three-word sentence “I like tea”, P(I)=0.2, P(like|I)=0.5 and P(tea|I,like)=0.4. Find its probability using the full chain rule.

Given

Three conditional factors are given, including the full two-word history for tea. No end probability is requested.

To find

P(I, like, tea).

Formula and why it applies

\"P(I,like,tea)=P(I)P(like\\mid

The chain rule supplies one factor per word in the specified sequence.

Solution

Step 1. Insert the three supplied factors
\"P=0.2\\times0.5\\times0.4\"
Step 2. Multiply the first two
\"0.2\\times0.5=0.1\"
Step 3. Multiply by the third
\"0.1\\times0.4=0.04\"
Final answer. The sentence probability is 0.04.

Question. A bigram model gives P(I|<s>)=0.5, P(like|I)=0.6, P(tea|like)=0.25 and P(</s>|tea)=0.8. Find the full sentence probability.

Given

All four conditional probabilities, including the end event, are supplied.

To find

P(I like tea </s> | <s>).

Formula and why it applies

\"P=P(I\\mid\\langle

Under the bigram assumption, every factor uses one preceding token.

Solution

Step 1. Insert the factors in sentence order
\"P=0.5\\times0.6\\times0.25\\times0.8\"
Step 2. Start and second word
\"0.5\\times0.6=0.3\"
Step 3. Third word
\"0.3\\times0.25=0.075\"
Step 4. End the sentence
\"0.075\\times0.8=0.06\"
Final answer. The complete sentence probability is 0.06.

Yaad rakho: Sequence probability = saare required transitions ka product.

Typical 2-mark answer. An n-gram language model predicts a word from its previous n−1 words using a Markov approximation. For example, a bigram model uses P(wᵢ | wᵢ₋₁). It estimates these probabilities from corpus counts and multiplies the appropriate factors to score a sentence.

"}]},{"id":"m2-count-to-probability","title":"From a corpus to sentence probabilities","unit":3,"scope":"Unit 3, pages 97–120 and appendix","sections":[{"id":"unigram-estimation","title":"Unigram estimation and complete worked examples","html":"

Maximum likelihood estimation (MLE) chooses probabilities equal to observed relative frequencies. In a unigram model, every word is predicted independently of its surrounding words. If a word occurs c times in N training tokens, its estimate is c/N. To score m specified words, multiply their m unigram probabilities.

\"P_{\\mathrm{MLE}}(w)=\\frac{C(w)}{N}\\qquad

Choose a tokenization convention before counting. Lowercasing, punctuation handling and boundary markers can change the vocabulary and counts. Never count words with one convention and form the denominator with another. A unigram model gives the same product to rearrangements of the same word multiset; that is one of its limitations.

Question. Train an unpadded, case-insensitive unigram model on “I like tea” and “I like coffee”. Find the probability of “I like tea”.

Given

Training sentences: I like tea; I like coffee. Use lowercase words; exclude punctuation and sentence boundaries.

To find

The vocabulary, word probabilities and test-sentence probability.

Formula and why it applies

\"P(w)=C(w)/N,\\qquad

The unigram approximation ignores order and uses total token count for every denominator.

Solution

Step 1. Count every training token

Each sentence contains three words, so N = 3+3 = 6.

Step 2. Count types and frequencies
WordCountMLE probability
i22/6 = 1/3
like22/6 = 1/3
tea11/6
coffee11/6

Vocabulary size V = 4.

Step 3. Write one factor per test word
\"P=\\frac{2}{6}\\times\\frac{2}{6}\\times\\frac{1}{6}\"
Step 4. Multiply the fractions
\"P=\\frac{4}{216}=\\frac{1}{54}\\approx0.018519\"
Final answer. The unigram sentence probability is approximately 0.018519.

Question. A corpus contains counts a: 4, b: 3 and c: 1. Find the unigram probability of “a b a”.

Given

C(a)=4, C(b)=3, C(c)=1. No boundary markers.

To find

P(a b a).

Formula and why it applies

\"P(a\\,b\\,a)=P(a)P(b)P(a)\"

A repeated test word contributes a probability factor at every occurrence; do not include it just once.

Solution

Step 1. Find the total
\"N=4+3+1=8\"
Step 2. Estimate the required probabilities
\"P(a)=4/8=1/2,\\qquad
Step 3. Use both occurrences of a
\"P=\\frac12\\times\\frac38\\times\\frac12\"
Step 4. Simplify
\"P=\\frac3{32}=0.09375\"
Final answer. P(a b a) = 0.09375.

Yaad rakho: Repeated word ka probability bhi repeat multiply hoga.

Question. An unsmoothed unigram model is trained on “red blue red”. Find the probability of “red green”.

Given

N=3. Counts: red=2, blue=1, green=0. The test contains red and green.

To find

The unsmoothed test probability and the reason for its value.

Formula and why it applies

\"P(red\\

MLE assigns zero to an unseen outcome. Smoothing is needed if an unseen in-vocabulary item must receive probability.

Solution

Step 1. Estimate red
\"P(red)=2/3\"
Step 2. Estimate the unseen word
\"P(green)=0/3=0\"
Step 3. Multiply
\"(2/3)\\times0=0\"
Final answer. Unsmoothed probability = 0. This reflects missing training evidence, not proof that the phrase is impossible.
"},{"id":"bigram-count-table","title":"Bigram counts: build the table before multiplying","html":"

A bigram probability table uses the previous word as its row and the next word as its column. Divide each cell count by its row's total outgoing count. With one end marker on every sentence, each ordinary training word has a continuation, so its outgoing count equals its token frequency. Do not create a bigram that crosses from one sentence into the next.

\"P(w\\mid
Add boundaries to each sentence separately
→
List adjacent pairs
→
Count repeated pairs
→
Divide each row by its total
→
Follow the test sentence through the table

Question. Train a bigram model on “I like tea”, “I like coffee” and “you like tea”. Use one start and one end marker. Find the probability of “I like tea”.

Given

Corpus: <s> I like tea </s>; <s> I like coffee </s>; <s> you like tea </s>. Preserve these tokens.

To find

All nonzero bigram estimates and the test probability.

Formula and why it applies

\"P(w\\mid

Every sentence starts and ends independently. Four probability factors are required for the three test words plus end.

Solution

Step 1. List the training pairs

Sentence 1: (<s>,I), (I,like), (like,tea), (tea,</s>).
Sentence 2: (<s>,I), (I,like), (like,coffee), (coffee,</s>).
Sentence 3: (<s>,you), (you,like), (like,tea), (tea,</s>).

Step 2. Create the nonzero count table
History → nextPair countHistory countProbability
<s> → I232/3
<s> → you131/3
I → like221
you → like111
like → tea232/3
like → coffee131/3
tea → </s>221
coffee → </s>111
Step 3. Write the test path
\"P=P(I\\mid\\langle
Step 4. Substitute each matching row
\"P=\\frac23\\times1\\times\\frac23\\times1\"
Step 5. Multiply
\"P=\\frac49\\approx0.444444\"
Final answer. P(I like tea) = 4/9 ≈ 0.444444.

Question. Using the same three-sentence corpus given here—“I like tea”, “I like coffee”, “you like tea”—find the bigram probabilities of “you like coffee” and “tea like I”. Include boundaries.

Given

Corpus with one start/end marker per sentence. Counts: start→you=1/3; you→like=1; like→coffee=1/3; coffee→end=1. No training sentence starts with tea.

To find

The two test probabilities.

Formula and why it applies

\"P(\\text{sentence})=\\prod

A model can assign a nonzero probability to a sentence it never saw if all of its local bigrams were seen.

Solution

Step 1. Score you like coffee
\"P=\\frac13\\times1\\times\\frac13\\times1=\\frac19\\approx0.111111\"
Step 2. Inspect tea like I
\"P(tea\\mid\\langle
Step 3. Apply the zero factor

The full product is zero, regardless of the remaining factors.

Final answer. P(you like coffee) = 1/9; P(tea like I) = 0. A new sentence is not necessarily a zero-probability sentence.

Question. Obtain the unsmoothed bigram probability of “students are from Vellore” from this corpus: “I am from Vellore”; “I am a teacher”; “students are good and are from various cities”; “students from Vellore do engineering”. Use sentence boundaries.

Given

Four training sentences, each padded with one <s> and </s>. Test sequence: <s> students are from Vellore </s>.

To find

The five required conditional probabilities and their product.

Formula and why it applies

\"P(w\\mid

Count all occurrences in the entire corpus. In particular, are occurs twice inside the third sentence, and Vellore is followed by end once and do once.

Solution

Step 1. Count the histories
HistoryOccurrences with continuation
<s>4
students2
are2
from3
Vellore2
Step 2. Count the required pairs
Required pairCountProbability
<s> students22/4
students are11/2
are from11/2
from Vellore22/3
Vellore </s>11/2
Step 3. Write the complete product
\"P=\\frac24\\times\\frac12\\times\\frac12\\times\\frac23\\times\\frac12\"
Step 4. Multiply progressively
\"\\frac24\\times\\frac12=\\frac14,\\quad
\"\\frac18\\times\\frac23=\\frac1{12},\\quad
Final answer. P = 1/24 ≈ 0.0416667.

Yaad rakho: Same sentence mein word do baar aaye toh count bhi do.

"},{"id":"trigram-estimation","title":"Trigram estimates: two-word history, not two denominators","html":"

A trigram model predicts the next word from the two preceding words. The numerator is the count of a triple. The denominator is the count of its two-word history with an available next token. Do not multiply two independent bigram probabilities and call that a trigram probability.

\"P(w_i\\mid

Question. A corpus contains 12 occurrences of “prime minister”. Eight are followed by “of”, three by “said”, and one by “visited”. Estimate the three trigram probabilities.

Given

History h = (prime, minister). Count of this history = 12. Triple counts for of, said, visited are 8, 3, 1.

To find

The conditional distribution of the next word.

Formula and why it applies

\"P(w\\mid

All three candidates share one two-word history, so they share denominator 12.

Solution

Step 1. Calculate of
\"P(of\\mid
Step 2. Calculate said
\"P(said\\mid
Step 3. Calculate visited
\"P(visited\\mid
Step 4. Confirm normalization
\"8/12+3/12+1/12=12/12=1\"
Final answer. The probabilities are 2/3, 1/4 and 1/12.

Question. Train a trigram model on “I like tea”, “I like coffee” and “you like tea”. Use two start markers and one end marker. Score “I like tea”.

Given

Three sentences. Padded first sentence is <s> <s> I like tea </s>; pad the other two in the same way.

To find

Four trigram prediction probabilities and the sentence probability.

Formula and why it applies

\"P=\\prod_i

Each prediction uses a two-token context; the two start markers supply the first history.

Solution

Step 1. Count the first-word event

The history (<s>,<s>) occurs three times, and two are followed by I.

\"P(I\\mid\\langle
Step 2. Count the second-word event

(<s>,I) occurs twice; both are followed by like.

\"P(like\\mid\\langle
Step 3. Count the third-word event

(I,like) occurs twice; one is followed by tea.

\"P(tea\\mid
Step 4. Count the end event

(like,tea) occurs twice; both are followed by end.

\"P(\\langle/s\\rangle\\mid
Step 5. Multiply all four
\"P=\\frac23\\times1\\times\\frac12\\times1=\\frac13\"
Final answer. Trigram P(I like tea) = 1/3. It differs from the bigram estimate because the context and denominator changed.

Question. A trigram model supplies P(we|<s>,<s>)=0.4, P(study|<s>,we)=0.5, P(NLP|we,study)=0.8 and P(</s>|study,NLP)=0.25. Score “we study NLP”.

Given

Four trigram conditional factors. The end marker is included.

To find

The complete sentence probability.

Formula and why it applies

\"P=P(we\\mid

Use each supplied two-word history exactly; do not swap its word order.

Solution

Step 1. Insert the values
\"P=0.4\\times0.5\\times0.8\\times0.25\"
Step 2. Multiply the first pair
\"0.4\\times0.5=0.2\"
Step 3. Include NLP
\"0.2\\times0.8=0.16\"
Step 4. Include the end
\"0.16\\times0.25=0.04\"
Final answer. P = 0.04.

Yaad rakho: Trigram: teen-word count upar, pehle do-word count neeche.

Why larger n is not automatically better. More context can capture local order more precisely, but each longer history occurs less often. Parameter count, storage and sparsity grow. A model trained on Shakespeare may assign poor probabilities to financial news because training and test distributions differ. Keep a held-out test set; do not evaluate only on memorized training sentences.

"},{"id":"class-count-table-questions","title":"Class-style count-table questions and partial-table traps","html":"

Question. Train a bigram model on “I am Sam”, “Sam I am”, and “I do not like green eggs and ham”. Include sentence boundaries. Find P(I|<s>), P(Sam|<s>), P(am|I), P(do|I), P(Sam|am) and P(</s>|Sam).

Given

Three sentences; one <s> and </s> per sentence; preserve capital I and Sam.

To find

Six bigram estimates.

Formula and why it applies

\"P(w\\mid

Count all matching histories, including repeated words in different sentence positions.

Solution

Step 1. Count the histories

START occurs 3 times; I occurs 3 times; am occurs 2 times; Sam occurs 2 times.

Step 2. Count the requested pairs
PairPair countHistory count
START I23
START Sam13
I am23
I do13
am Sam12
Sam END12
Step 3. Divide pair by history
Requested probabilityResult
P(I|START)2/3
P(Sam|START)1/3
P(am|I)2/3
P(do|I)1/3
P(Sam|am)1/2
P(END|Sam)1/2
Final answer. The six estimates are 2/3,1/3,2/3,1/3,1/2,1/2 in the requested order.

Question. A bigram table gives P(i|<s>)=0.25, P(want|i)=0.33, P(english|want)=0.0011, P(food|english)=0.5 and P(</s>|food)=0.68. Score “i want english food” using these supplied rounded values.

Given

All five transition probabilities are explicitly given. Include the end marker.

To find

The sentence probability from the supplied table.

Formula and why it applies

\"P=P(i\\mid

A table-based question uses the provided values; slight differences can arise if exact corpus fractions are used instead.

Solution

Step 1. First two events
\"0.25(0.33)=0.0825\"
Step 2. Add english
\"0.0825(0.0011)=0.00009075\"
Step 3. Add food
\"0.00009075(0.5)=0.000045375\"
Step 4. End the sentence
\"0.000045375(0.68)=0.000030855\"
Final answer. P≈0.000030855 using the supplied rounded probabilities.

Question. A corpus has vocabulary size 1446, C(i)=2533 and C(i,want)=827. Compute the unsmoothed and add-one-smoothed P(want|i). The displayed bigram table contains only eight vocabulary columns.

Given

Full history count C(i)=2533; pair count 827; complete vocabulary size V=1446. The displayed table is only a subset.

To find

Both conditional probabilities.

Formula and why it applies

\"P(want\\mid

Use the supplied full history total. Summing only the eight displayed columns would wrongly ignore all other possible continuations.

Solution

Step 1. Calculate the unsmoothed ratio
\"P=827/2533\\approx0.326490\"
Step 2. Add one to the pair count
\"827+1=828\"
Step 3. Add V to the history count
\"2533+1446=3979\"
Step 4. Divide
\"P_L=828/3979\\approx0.208092\"
Final answer. Unsmoothed≈0.326490; smoothed≈0.208092.

Yaad rakho: Partial table ka visible row-sum, full history total nahi hota.

"}]},{"id":"m2-evaluation-smoothing","title":"Perplexity, log probabilities and smoothing","unit":3,"scope":"Unit 3, pages 121–156","sections":[{"id":"evaluation-perplexity","title":"Perplexity: meaning, formula and four complete examples","html":"

Intrinsic evaluation evaluates a model directly on a held-out test corpus, for example using perplexity. Extrinsic evaluation places it inside a downstream application, such as a speech recognizer or translation system, and measures the application's performance. Extrinsic testing is closer to actual use but often slower and more expensive. A lower perplexity is not a guarantee of improvement on every downstream task.

Perplexity is the inverse probability of a test sequence, normalized by the number of prediction events. Intuitively, it is the model's effective average number of equally plausible choices. A model that always gives the correct next word probability 1/4 has perplexity 4. Lower is better when the models are tested on the same data, vocabulary, tokenization and boundary convention.

\"\\operatorname{PP}(W)=P(W)^{-1/N}=\\sqrt[N]{\\frac1{P(W)}}\"
SymbolMeaning
WThe complete test sequence.
P(W)Product of the probabilities assigned to the actual next tokens.
NNumber of scored prediction events under the stated convention.
Exponent −1/NTake the reciprocal, then the Nth root.

For calculator entry, use parentheses: P raised to (−1 ÷ N), or (1 ÷ P) raised to (1 ÷ N). Retain several digits in intermediate products. The ordinary start marker is usually given context, not a predicted event; the end marker counts if its probability is included.

Question. A model assigns probability 0.0625 to a test sequence containing four scored tokens. Compute its perplexity.

Given

P(W)=0.0625=1/16; N=4.

To find

PP(W).

Formula and why it applies

\"\\operatorname{PP}(W)=\\left(\\frac1{P(W)}\\right)^{1/N}\"

The root removes the effect of sequence length; the reciprocal makes lower surprise correspond to lower perplexity.

Solution

Step 1. Take the reciprocal
\"1/P(W)=1/0.0625=16\"
Step 2. Use the fourth root
\"\\operatorname{PP}=16^{1/4}\"
Step 3. Evaluate
\"2^4=16\\quad\\Rightarrow\\quad\\operatorname{PP}=2\"
Final answer. Perplexity = 2.

Question. Two language models score the same three-token test sequence. Model A gives next-token probabilities 0.5, 0.5, 0.5. Model B gives 0.5, 0.25, 0.25. Compute their perplexities and select the better intrinsic result.

Given

Same sequence and N=3 for both models. A factors: 0.5,0.5,0.5. B factors: 0.5,0.25,0.25.

To find

Both sequence probabilities and perplexities.

Formula and why it applies

\"P(W)=\\prod_i

A fair comparison uses identical test tokens and scoring conventions.

Solution

Step 1. Calculate A's probability
\"P_A=0.5\\times0.5\\times0.5=0.125=1/8\"
Step 2. Calculate A's perplexity
\"\\operatorname{PP}_A=8^{1/3}=2\"
Step 3. Calculate B's probability
\"P_B=0.5\\times0.25\\times0.25=0.03125=1/32\"
Step 4. Calculate B's perplexity
\"\\operatorname{PP}_B=32^{1/3}\\approx3.174802\"
Step 5. Compare

2 is lower than 3.174802; A places more probability on the observed sequence.

Final answer. Model A is better by perplexity on this test: 2 versus about 3.174802.

Question. The bigram training corpus is “This is a dog”, “This is a cat”, “I love my cat”, “This is my name”. Score “This is my cat”, including an end marker, and compute perplexity using five predicted events.

Given

Each training sentence has one start and end marker. Test predictions: This, is, my, cat, </s>. Thus N=5.

To find

Bigram sentence probability and perplexity.

Formula and why it applies

\"P(w\\mid

Start is given context. Four words plus the end marker produce five probability factors.

Solution

Step 1. Count the required transitions
TransitionPair count / history count
<s>→This3/4
This→is3/3
is→my1/3
my→cat1/2
cat→</s>2/2
Step 2. Form the product
\"P=\\frac34\\times1\\times\\frac13\\times\\frac12\\times1=\\frac18\"
Step 3. Take the reciprocal
\"1/P=8\"
Step 4. Take the fifth root
\"\\operatorname{PP}=8^{1/5}\\approx1.515717\"
Final answer. P(W)=1/8 and perplexity≈1.515717 using five predicted events.

If the question explicitly states N = 6: use that supplied convention and report PP = 81/6 ≈ 1.414214. The class slide counts both boundary markers in N. Write the chosen N beside the formula; do not silently switch between 5 and 6. Also, This occurs three times and is follows it three times, so P(is|This)=3/3, not 3/4.

Question. A uniform model assigns probability 1/10 to each actual next word in a five-token test. Find its perplexity.

Given

Five scored tokens, each with assigned probability 0.1. N=5.

To find

The sentence probability and perplexity.

Formula and why it applies

\"P(W)=(1/10)^5,\\qquad

Uniform equal probabilities make the effective number of choices especially clear.

Solution

Step 1. Multiply the five factors
\"P(W)=0.1^5=0.00001=10^{-5}\"
Step 2. Invert
\"1/P(W)=10^5\"
Step 3. Take the fifth root
\"\\operatorname{PP}=(10^5)^{1/5}=10\"
Final answer. Perplexity = 10.

Yaad rakho: Probability high → surprise low → perplexity low.

"},{"id":"log-probabilities","title":"Log probabilities: replace long products with sums","html":"

Probabilities are at most 1, so multiplying many of them produces very small numbers. Computers can round extremely small positive products to zero: numerical underflow. Log probabilities convert multiplication into addition and are more stable for long sequences. Natural logs use base e; log₂ uses base 2. Choose one base consistently.

\"\\ln
\"P(W)=e^{\\ln

The log of a probability below 1 is negative. A value closer to zero means a larger probability: ln(0.5) ≈ −0.693 is larger than ln(0.1) ≈ −2.303. A zero probability has log −∞; taking logs does not solve the unseen-event problem. Smoothing handles that separate problem.

Question. The next-word probabilities are 0.5, 0.25 and 0.5. Calculate the natural log probability, ordinary probability and perplexity.

Given

p₁=0.5; p₂=0.25; p₃=0.5; N=3.

To find

ln P(W), P(W), and PP.

Formula and why it applies

\"\\ln

Logarithms preserve products as sums and the exponent converts the result back.

Solution

Step 1. Compute individual logs
\"\\ln0.5\\approx-0.693147,\\quad\\ln0.25\\approx-1.386294\"
Step 2. Add all three terms
\"\\ln
Step 3. Recover the probability
\"P=e^{-2.772589}\\approx0.0625\"
Step 4. Find average negative log probability
\"-\\ln
Step 5. Exponentiate
\"\\operatorname{PP}=e^{0.924196}\\approx2.519842\"
Final answer. ln P≈−2.772589, P=0.0625, PP≈2.519842.

Question. A model has a total natural log probability −12 over six scored tokens. Calculate its perplexity.

Given

ln P(W)=−12; N=6.

To find

PP without forming the tiny ordinary probability.

Formula and why it applies

\"\\operatorname{PP}=\\exp(-\\ln

The logarithmic formula directly uses the supplied log score.

Solution

Step 1. Negate the log score
\"-\\ln
Step 2. Divide by token count
\"12/6=2\"
Step 3. Apply exponential
\"\\operatorname{PP}=e^2\\approx7.389056\"
Final answer. Perplexity≈7.389056.

Question. A language model's average negative log₂ probability is 3 bits per token. Find its perplexity.

Given

Cross-entropy H=3 bits/token. Log base is 2.

To find

PP.

Formula and why it applies

\"\\operatorname{PP}=2^H\"

The inverse operation of log₂ is raising 2 to that power; use e only for natural logs.

Solution

Step 1. Identify the log base

Bits mean base 2.

Step 2. Raise the base to H
\"\\operatorname{PP}=2^3=8\"
Final answer. Perplexity = 8.

Yaad rakho: ln use kiya toh e se wapas; log₂ use kiya toh 2 se wapas.

"},{"id":"laplace-basics","title":"Laplace smoothing: why both numerator and denominator change","html":"

Smoothing reallocates some probability mass to events that received zero or unreliable estimates from finite training data. Laplace, or add-one, smoothing adds one pseudo-count to every possible next outcome. If there are V possible outcomes, the total added count is V, so adding 1 only to the numerator would produce probabilities that do not sum to 1.

\"P_{\\mathrm{Laplace}}(w)=\\frac{C(w)+1}{N+V}\"
\"P_{\\mathrm{Laplace}}(w\\mid

For unigram smoothing, N is the total training token count. For bigram smoothing, C(h) is the count of the particular history. In both cases V is the number of outcomes being predicted, not the number of training sentences and not the number of distinct bigrams. When end is a possible prediction, include it in V; the start marker is usually context only.

Observed counts: 3, 1, 0
→
Add one to every outcome: 4, 2, 1
→
New total: 7
→
Smoothed probabilities: 4/7, 2/7, 1/7

Question. A fixed unigram vocabulary is {a,b,c}. The training counts are 3,1,0. Compute unsmoothed and Laplace-smoothed probabilities.

Given

C(a)=3, C(b)=1, C(c)=0; N=4; V=3.

To find

The complete distributions.

Formula and why it applies

\"P(w)=C(w)/N,\\qquad

The zero-count c is still in the fixed vocabulary, so it must receive one pseudo-count and contribute to V.

Solution

Step 1. Compute unsmoothed values
WordUnsmoothed
a3/4 = 0.75
b1/4 = 0.25
c0/4 = 0
Step 2. Calculate the new denominator
\"N+V=4+3=7\"
Step 3. Add one to each numerator
\"C(a)+1=4,\\quad
Step 4. Divide
WordSmoothed
a4/7 ≈ 0.571429
b2/7 ≈ 0.285714
c1/7 ≈ 0.142857
Step 5. Confirm the total
\"4/7+2/7+1/7=1\"
Final answer. Smoothed probabilities: a=4/7, b=2/7, c=1/7. Common a loses mass; unseen c gains mass.

Question. After history I, a corpus records like three times, study once and sleep zero times. The allowed next vocabulary is {like,study,sleep}. Compute all Laplace-smoothed bigram probabilities.

Given

C(I,like)=3; C(I,study)=1; C(I,sleep)=0. History count C(I)=4. V=3.

To find

P_L(like|I), P_L(study|I), P_L(sleep|I).

Formula and why it applies

\"P_L(w\\mid

This is smoothing within the I row; use its total 4, not the global corpus token count.

Solution

Step 1. Form the common denominator
\"C(I)+V=4+3=7\"
Step 2. Calculate like
\"P_L(like\\mid
Step 3. Calculate study
\"P_L(study\\mid
Step 4. Calculate sleep
\"P_L(sleep\\mid
Final answer. The row is (4/7, 2/7, 1/7). An unseen bigram is no longer forced to zero.

Question. A fixed vocabulary has five possible next tokens. A history h is unseen in training. What is its add-one probability for each next token?

Given

C(h)=0; C(h,w)=0 for each outcome; V=5.

To find

P_L(w|h) for each token.

Formula and why it applies

\"P_L(w\\mid

All outcomes receive equal pseudo-counts because the training data gives no preference for this history.

Solution

Step 1. Add the pseudo-count
\"0+1=1\"
Step 2. Form the denominator
\"0+5=5\"
Step 3. Divide and sum
\"P_L(w\\mid
Final answer. Every outcome gets 0.2 under this fixed-vocabulary model.

Vocabulary warning. Add-one does not automatically define probabilities for every word in the world. Unknown words require a declared vocabulary/UNK convention. If a question supplies V, use that stated V rather than inventing additional tokens.

Yaad rakho: Add-one: upar +1, neeche +V; bigram mein neeche history ka count.

"},{"id":"smoothed-sentence","title":"A complete unseen-bigram question from corpus to final answer","html":"

Question. Consider the corpus “I love to code in Python. I love to learn new things. I code every day.” Obtain the bigram probabilities needed for “I love Python”. Explain the zero-probability issue, then use Laplace smoothing to score it. Treat each sentence separately, preserve case, remove punctuation, and include an end token among predicted outcomes.

Given

Three sentences padded with one start/end marker. Distinct words: I, love, to, code, in, Python, learn, new, things, every, day. With </s>, V=12. Start is context only.

To find

Unsmoothed and smoothed P(I love Python </s> | <s>).

Formula and why it applies

\"P(w\\mid

The test contains an unseen transition love→Python. Smoothing assigns probability to that transition and renormalizes every required history row.

Solution

Step 1. Count the contexts
HistoryCount
<s>3
I3
love2
Python1
Step 2. Count each required pair
PairCount
<s> I3
I love2
love Python0
Python </s>1
Step 3. Compute the unsmoothed product
\"P=\\frac33\\times\\frac23\\times\\frac02\\times\\frac11=0\"

Both words love and Python were seen, but their adjacent pair was not. This is an unseen bigram, not an unknown-word case.

Step 4. Smooth the first event
\"P_L(I\\mid\\langle
Step 5. Smooth the second event
\"P_L(love\\mid
Step 6. Smooth the unseen event
\"P_L(Python\\mid
Step 7. Smooth the end event
\"P_L(\\langle/s\\rangle\\mid
Step 8. Multiply the smoothed factors
\"P_L=\\frac4{15}\\times\\frac3{15}\\times\\frac1{14}\\times\\frac2{13}\"
\"P_L=\\frac{24}{40950}=\\frac4{6825}\\approx0.000586081\"
Final answer. Unsmoothed P=0. With the stated V=12 and boundaries, Laplace-smoothed P≈0.000586081.

If an exam states a different convention—for example excluding boundary tokens from the entire calculation—recompute both V and the set of probability factors consistently. Do not combine a boundary-aware numerator with a boundary-free denominator.

"},{"id":"adjusted-count-discount","title":"Adjusted counts and relative discounting","html":"

The slides also express smoothing through an adjusted count c*. Instead of writing the smoothed probability with denominator N+V, they rescale its numerator so the denominator can remain N. Algebraically these are equivalent:

\"c^*=(c+1)\\frac{N}{N+V},\\qquad
\"d=\\frac{c^*}{c}\\quad(c>0)\"

The relative discount d compares adjusted and original counts. It is undefined when c=0 because division by zero is undefined. Add-one smoothing moves the distribution toward uniformity: frequent events are reduced, but some rare seen events may increase. It is not correct to claim that every nonzero count must decrease under Laplace smoothing.

Question. For N=20 and V=5, a word has original count c=10. Find its adjusted count, smoothed probability and relative discount.

Given

N=20; V=5; c=10.

To find

c*, P_L and d.

Formula and why it applies

\"c^*=(c+1)N/(N+V),\\quad

These expressions rewrite add-one smoothing without changing its probability.

Solution

Step 1. Form the new total
\"N+V=20+5=25\"
Step 2. Find the adjusted count
\"c^*=(10+1)\\frac{20}{25}=11(0.8)=8.8\"
Step 3. Find the probability
\"P_L=8.8/20=0.44\"
Step 4. Find the relative discount
\"d=8.8/10=0.88\"
Final answer. c*=8.8, P_L=0.44, d=0.88. The original probability 0.5 is reduced.

Question. Under N=20, V=5, a rare word occurs once. Compute its adjusted count and compare the original and smoothed probabilities.

Given

N=20; V=5; c=1.

To find

c*, original P and smoothed P_L.

Formula and why it applies

\"c^*=(c+1)N/(N+V)\"

The same smoothing rule may increase a rare event's probability.

Solution

Step 1. Original probability
\"P=1/20=0.05\"
Step 2. Adjusted count
\"c^*=(1+1)(20/25)=1.6\"
Step 3. Smoothed probability
\"P_L=1.6/20=0.08\"
Final answer. c*=1.6 and P_L=0.08, which exceeds the original 0.05.

Question. For N=20 and V=5, an in-vocabulary word has count zero. Find its adjusted count and smoothed probability. Can a relative discount be calculated?

Given

c=0; N=20; V=5.

To find

c*, P_L and whether d exists.

Formula and why it applies

\"c^*=(c+1)N/(N+V),\\qquad

The pseudo-count creates positive mass for the unseen vocabulary item.

Solution

Step 1. Adjusted count
\"c^*=(0+1)(20/25)=0.8\"
Step 2. Smoothed probability
\"P_L=0.8/20=0.04\"
Step 3. Relative discount

d would be 0.8/0. That is undefined, so do not report a finite discount.

Final answer. c*=0.8 and P_L=0.04; relative discount is undefined for c=0.
"},{"id":"class-perplexity-characters","title":"Character-model perplexity: keep exact probabilities","html":"

Question. A unigram character model gives P(A)=0.25, P(B)=0.50, P(C)=0.25. Find the perplexity of AAA and ABC. Then find both perplexities when the three characters are equally likely.

Given

Each test string contains 3 scored characters, so N=3. Initial distribution 0.25,0.50,0.25; comparison distribution 1/3,1/3,1/3.

To find

Both sequence probabilities and perplexities under each distribution.

Formula and why it applies

\"P(c_1c_2c_3)=P(c_1)P(c_2)P(c_3),\\qquad

Characters, not words, are the model's prediction units in this question. Use exact 1/3 for uniform probabilities, rather than rounded 0.33.

Solution

Step 1. Score AAA
\"P(AAA)=0.25^3=\\frac1{64}=0.015625\"
Step 2. Find AAA perplexity
\"PP(AAA)=64^{1/3}=4\"
Step 3. Score ABC
\"P(ABC)=0.25(0.5)(0.25)=\\frac1{32}=0.03125\"
Step 4. Find ABC perplexity
\"PP(ABC)=32^{1/3}\\approx3.174802\"
Step 5. Change to an exactly uniform model
\"P(AAA)=P(ABC)=(1/3)^3=1/27\"
Step 6. Find both uniform-model perplexities
\"PP=27^{1/3}=3\"
Final answer. Original model: AAA→4, ABC→3.174802. Uniform model: both→3. These values follow the stated formulas without intermediate rounding.
"}]},{"id":"m2-pos-tagging","title":"POS tagging and its learning approaches","unit":4,"scope":"Unit 4, pages 3–46","sections":[{"id":"word-classes-pos","title":"Parts of speech and contextual ambiguity","html":"

Part-of-speech (POS) tagging assigns a grammatical category to every word token in context, such as noun, verb, adjective or determiner. It is a sequence-labeling task: the input is a sequence of words, and the output is a tag for each word. A tag describes the word's role in this sentence, not its meaning in every possible sentence.

TagRoleExample in context
NN / NNSSingular / plural common nouna book / two books
NNPProper nounMary, Mumbai
VB / VBD / VBZBase / past / third-person singular verbread / read / reads
JJAdjectivethe red book
RBAdverbruns quickly
DTDeterminerthe, a, this
INPreposition or subordinating conjunctionon the table
PRPPersonal pronounshe, they
MDModal auxiliarycan, will, should

Open classes readily accept new content words: nouns, verbs, adjectives and adverbs. Examples include chatbot and livestream. Closed classes contain comparatively stable grammatical/function words: determiners, pronouns, prepositions and conjunctions. These are tendencies, not a claim that language never changes.

Input: Book that flight
→
Context: an imperative request
→
Output: Book/VB that/DT flight/NN

Compare “hand me that book”: book is a noun here. Similarly, “I can swim” has can/MD, while “the can is empty” has can/NN. The spelling stays the same; neighboring words and sentence structure change the correct tag. This is POS ambiguity.

SentenceTagged interpretationReason
They watch birds.They/PRP watch/VBP birds/NNSwatch is the action.
The watch stopped.The/DT watch/NN stopped/VBDwatch is the object that stopped.
She has a light bag.She/PRP has/VBZ a/DT light/JJ bag/NNlight describes bag.
Light the lamp.Light/VB the/DT lamp/NNLight is an imperative verb.

POS tags support parsing, named entity recognition, question answering, sentiment analysis and word-sense disambiguation. They do not by themselves identify every entity or fully resolve meaning.

Yaad rakho: Word ka naam same ho sakta hai; sentence mein uska kaam tag decide karta hai.

"},{"id":"tagger-methods","title":"Lexical, rule-based, stochastic, Brill and neural taggers","html":"

A lexical baseline gives each known word its most frequent training tag. It is simple and often useful, but ignores context. A rule-based tagger uses a lexicon of possible tags plus manually designed contextual or morphological rules to resolve ambiguity. For example, after a determiner, a noun reading of “can” may be preferred in “the can”.

Rules are usually heuristics with exceptions. “A red car” shows that the word immediately after an article is not always a noun. The suffix -ed does not make “red” a past-tense verb. A robust rule system uses multiple linguistic constraints instead of treating such shortcuts as universal laws.

MethodHow it chooses tagsStrengthLimitation
Most frequent / lexicalMost common training tag for the wordSimple baselineMisses contextual changes and unknown words
Rule-basedLexicon plus handwritten rulesReadable, controllable linguistic decisionsExpert effort; rules may conflict or need updates
StochasticScores tag sequences using estimated probabilitiesUses corpus evidence and contextSparse/unseen events; depends on training domain
Transformation-based / BrillStart with baseline, then apply learned correction rulesReadable learned correctionsNeeds tagged training data and suitable rule templates
NeuralLearns contextual representations and sequence predictionsCan exploit richer contextNeeds training resources and data; less directly interpretable

Transformation-based error-driven learning (Brill tagging) combines a simple starting assignment with learned correction rules. First tag the training corpus with a baseline. Compare against the gold tags. Generate candidate correction rules from templates, such as “change MD to NN after DT”. Apply the rule that fixes the most errors net of newly introduced errors; repeat until further improvement is too small. At test time apply the learned ordered rule list.

Baseline: The/DT can/MD was/VBD crushed/VBN
→
Context rule: change can from MD to NN after DT
→
Result: The/DT can/NN was/VBD crushed/VBN

Rule-based versus Brill: a purely rule-based system depends on linguists specifying its rules; a Brill tagger learns which transformations improve a labeled corpus. A stochastic tagger such as an HMM finds a high-probability tag sequence using transition and emission probabilities. CRFs and neural sequence taggers are other learning approaches; they do not have to make the same HMM independence assumptions.

Exam-ready answer. POS tagging assigns a grammatical tag to each word in context. A rule-based tagger uses lexicons and linguistic rules, whereas a stochastic tagger chooses tags through probabilities estimated from a corpus. For example, can is MD in “I can swim” but NN in “the can”; context resolves the ambiguity.

"},{"id":"generative-discriminative","title":"Generative versus discriminative models and Bayes' rule","html":"

Let x be the observed words and y be the desired tag sequence. A generative model describes their joint distribution P(x,y), often as P(y)P(x|y). A discriminative model learns P(y|x) directly or learns a decision boundary that separates possible labels. Both kinds can be trained with labeled data; “generative” does not mean “always unsupervised”.

\"P(y\\mid
QuantityMeaningPOS interpretation
P(y)PriorHow plausible is this tag or tag sequence before seeing the words?
P(x|y)LikelihoodHow plausible are these words under that tag sequence?
P(x)EvidenceTotal probability of the observed words across possible labels.
P(y|x)PosteriorProbability of the tags after seeing the words.
AspectGenerativeDiscriminative
ModelsJoint P(x,y), or prior × likelihoodConditional P(y|x) or decision boundary
Classification examplesNaive BayesLogistic regression, SVM
Sequence-labeling examplesHMMLinear-chain CRF
IntuitionModel how a label could generate the observationModel which label fits the observation

Question. A token has two possible tags, noun N and verb V. P(N)=0.6, P(V)=0.4, P(book|N)=0.2 and P(book|V)=0.5. Calculate P(N|book) and P(V|book).

Given

Two tags are exhaustive. Priors: 0.6,0.4. Word likelihoods: 0.2,0.5.

To find

Normalized posterior probabilities.

Formula and why it applies

\"P(t\\mid

Bayes' rule combines prior and likelihood. Normalization is needed to convert scores into posterior probabilities.

Solution

Step 1. Calculate the noun score
\"0.2\\times0.6=0.12\"
Step 2. Calculate the verb score
\"0.5\\times0.4=0.20\"
Step 3. Find the evidence
\"P(book)=0.12+0.20=0.32\"
Step 4. Normalize the noun score
\"P(N\\mid
Step 5. Normalize the verb score
\"P(V\\mid
Final answer. Verb is more probable in this simplified model: 0.625 versus 0.375.

Question. Two candidate tags have prior-likelihood products 0.06 and 0.14. Find their normalized probabilities.

Given

Unnormalized scores s₁=0.06 and s₂=0.14; these are the only candidates.

To find

The two posterior probabilities.

Formula and why it applies

\"P_i=s_i/(s_1+s_2)\"

A common denominator converts the candidate scores to a distribution summing to one.

Solution

Step 1. Add scores
\"0.06+0.14=0.20\"
Step 2. Normalize first
\"0.06/0.20=0.3\"
Step 3. Normalize second
\"0.14/0.20=0.7\"
Final answer. Posterior probabilities are 0.3 and 0.7.

Yaad rakho: Generative: tag se word ka model. Discriminative: word se tag ka decision.

"}]},{"id":"m2-hmm","title":"Markov chains and Hidden Markov Models","unit":4,"scope":"Unit 4, pages 47–73","sections":[{"id":"markov-chain","title":"Markov chains: read a transition table and multiply a path","html":"

A Markov chain is a probabilistic model of a sequence of states. Under the first-order Markov assumption, the next state's probability depends on the current state rather than the complete previous history. The initial probability tells us how likely each first state is. A transition probability tells us how likely a move from one state to another is.

\"P(s_t\\mid
\"P(s_1,\\ldots,s_T)=P(s_1)\\prod_{t=2}^{T}P(s_t\\mid

For T states in a path, use one initial factor and T−1 transition factors. A row-oriented transition matrix must have each complete row sum to 1. Always read the row/column labels; some exam diagrams use a transposed arrangement.

Current → nextRainDry
Rain0.30.7
Dry0.20.8

Initial probabilities: P(Rain)=0.4, P(Dry)=0.6. Rows describe the current state; columns describe the next state.

Question. For the Rain/Dry Markov chain with initial P(Rain)=0.4 and P(Dry)=0.6, and transitions Rain→Rain=0.3, Rain→Dry=0.7, Dry→Rain=0.2, Dry→Dry=0.8, find the probability of Dry, Dry, Rain, Rain.

Given

Current → nextRainDry
Rain0.30.7
Dry0.20.8

Initial probabilities: P(Rain)=0.4, P(Dry)=0.6. Rows describe the current state; columns describe the next state.

To find

P(Dry,Dry,Rain,Rain).

Formula and why it applies

\"P(D,D,R,R)=P(D)P(D\\mid

Each step conditions on the immediately preceding state. The sequence contains four states, so it needs one initial plus three transition factors.

Solution

Step 1. Start in Dry
\"P(D)=0.6\"
Step 2. Move Dry to Dry
\"0.6\\times0.8=0.48\"
Step 3. Move Dry to Rain
\"0.48\\times0.2=0.096\"
Step 4. Move Rain to Rain
\"0.096\\times0.3=0.0288\"
Final answer. Path probability = 0.0288.

Question. For initial probabilities Rain=0.4, Dry=0.6 and transition rows Rain=(0.3,0.7), Dry=(0.2,0.8) in column order Rain,Dry, find the probability of Rain, Dry, Rain.

Given

Current → nextRainDry
Rain0.30.7
Dry0.20.8

Initial probabilities: P(Rain)=0.4, P(Dry)=0.6. Rows describe the current state; columns describe the next state.

To find

P(Rain,Dry,Rain).

Formula and why it applies

\"P(R,D,R)=P(R)P(D\\mid

Do not replace Dry→Rain with Rain→Dry: the two directional probabilities differ.

Solution

Step 1. Initial Rain
\"P(R)=0.4\"
Step 2. First transition
\"P(D\\mid
Step 3. Second transition
\"P(R\\mid
Final answer. Path probability = 0.056.

Question. For the same fully specified chain—initial Rain=0.4, Dry=0.6; Rain→Rain=0.3, Rain→Dry=0.7, Dry→Rain=0.2, Dry→Dry=0.8—find P(Dry,Dry). Also find the conditional probability of next Dry given current Dry.

Given

Current → nextRainDry
Rain0.30.7
Dry0.20.8

Initial probabilities: P(Rain)=0.4, P(Dry)=0.6. Rows describe the current state; columns describe the next state.

To find

The two-state joint probability and the one-step conditional probability.

Formula and why it applies

\"P(D,D)=P(D)P(D\\mid

The joint path includes an uncertain initial state. The conditional transition assumes the current state is already Dry.

Solution

Step 1. Read the conditional transition
\"P(D\\mid
Step 2. Include the initial state for the joint path
\"P(D,D)=0.6\\times0.8=0.48\"
Final answer. P(Dry,Dry)=0.48; P(next Dry|current Dry)=0.8. These answer different questions.

Yaad rakho: Path: pehle start probability, phir har arrow ki probability.

"},{"id":"hmm-components","title":"Hidden states, observations, transitions and emissions","html":"

A Hidden Markov Model (HMM) is a probabilistic sequence model in which the underlying state sequence is unobserved, while each state emits an observed item. For POS tagging, words are observed and their grammatical tags are hidden. The task is to infer a suitable tag sequence from the words.

Hidden: Noun
→
Hidden: Verb
→
Hidden: Noun
Observed: birds
→
Observed: eat
→
Observed: seeds

Horizontal state-to-state links are transitions. The relation from each hidden state to its observed word is an emission. A transition P(V|N) asks how often a verb tag follows a noun tag. An emission P(eat|V) asks how often the observed word eat appears among verb-tagged tokens.

ComponentNotationMeaning
State setS={N,V,…}Possible hidden tags.
Observation vocabularyO={birds,eat,seeds,…}Visible word tokens.
Initial distributionπⱼ=P(t₁=j)Probability of starting in tag j.
Transition matrixaᵢⱼ=P(tₖ=j|tₖ₋₁=i)Probability of next tag j after i.
Emission matrixbⱼ(w)=P(w|tag j)Probability that state j emits word w.

Two basic assumptions are used: the current hidden state depends only on the previous hidden state, and the current observation depends only on the current hidden state. A standard first-order HMM therefore does not explicitly condition an emitted word on the whole sentence.

\"P(t_{1:T},w_{1:T})=\\pi_{t_1}b_{t_1}(w_1)\\prod_{k=2}^{T}a_{t_{k-1},t_k}b_{t_k}(w_k)\"

Read the formula as start × first emission × transition × emission × transition × emission…. There are T emission factors and T−1 transitions, plus one initial factor. If the question supplies an explicit end-state probability, multiply that too. Do not invent an end factor when the model has none.

Three different questions. (1) Scoring a specified tag path uses the joint product above. (2) Finding the probability of the observed words sums scores over all possible tag paths. (3) Finding the best tag path maximizes the path score, typically using Viterbi. A joint path score is not a normalized posterior P(tags|words).

Yaad rakho: Transition = tag se agla tag. Emission = tag se visible word.

"},{"id":"transition-emission-estimation","title":"Estimate transition and emission probabilities from tagged data","html":"
\"P(t'\\mid
\"P(w\\mid

In a transition denominator, count the previous tag's outgoing events. In an emission denominator, count all tokens bearing that tag. With explicit end transitions the totals align conveniently. Keep the two numerator types separate: tag→tag counts are not word→tag counts.

Question. A tagged corpus contains 116454 determiner tags DT, each counted with an outgoing event. DT is followed by NN 56509 times. Estimate P(NN|DT).

Given

C(DT)=116454; C(DT,NN)=56509.

To find

The DT→NN transition probability.

Formula and why it applies

\"P(NN\\mid

The denominator is the number of previous DT contexts because DT is the condition.

Solution

Step 1. Identify the transition count

DT followed by NN occurs 56509 times.

Step 2. Identify the conditional total

There are 116454 DT contexts.

Step 3. Divide
\"P(NN\\mid
Final answer. P(NN|DT)≈0.485247.

Question. The tag VBZ occurs 21627 times, and 10073 of those tokens are the word “is”. Estimate P(is|VBZ).

Given

C(VBZ)=21627; C(is,VBZ)=10073.

To find

The emission probability of is from VBZ.

Formula and why it applies

\"P(is\\mid

The condition is the tag VBZ; we ask which word it emits.

Solution

Step 1. Choose the word-tag count

The required word is is with tag VBZ: count 10073.

Step 2. Choose the tag total

All VBZ-tagged word tokens: count 21627.

Step 3. Divide
\"P(is\\mid
Final answer. P(is|VBZ)≈0.465760. This is different from P(VBZ|is).

Question. Use the tagged sentences “dogs/N chase/V cats/N” and “cats/N sleep/V”. Estimate the initial tag probabilities, all nonzero transitions including END, and emissions.

Given

Corpus: START dogs/N chase/V cats/N END; START cats/N sleep/V END.

To find

Initial distribution, transition rows and emission rows.

Formula and why it applies

\"\\pi_t=C(START,t)/2,\\quad

The corpus explicitly tells us the tags, so we can count training frequencies directly.

Solution

Step 1. Count tag tokens

N occurs 3 times: dogs, cats, cats. V occurs 2 times: chase, sleep.

Step 2. Count sentence starts

Both sentences start with N.

\"\\pi_N=2/2=1,\\quad
Step 3. Count transitions
PairCount
N→V2
N→END1
V→N1
V→END1
Step 4. Normalize transition rows
CurrentTo NTo VTo END
N02/31/3
V1/201/2
Step 5. Count and normalize emissions
Tagdogscatschasesleep
N1/32/300
V001/21/2
Final answer. π=(1,0); transitions and emissions are given in the completed tables. Each complete probability row sums to 1.

Yaad rakho: Tag→tag table aur tag→word table ko mix mat karna.

"},{"id":"hot-cold-paths","title":"HMM path probabilities: three complete numerical examples","html":"

The following model uses hidden weather states Hot (H) and Cold (C). Observations are numbers 1, 2 and 3. These numbers are observation labels, not quantities to multiply directly. Their emission probabilities come from the table.

Initial stateProbability
H0.8
C0.2
From → toHC
H0.70.3
C0.40.6
Emission stateObservation 1Observation 2Observation 3
H0.20.40.4
C0.50.40.1

Question. An HMM has π(H)=0.8, π(C)=0.2; H→H=0.7, H→C=0.3, C→H=0.4, C→C=0.6. Emissions for observations (1,2,3) are H:(0.2,0.4,0.4) and C:(0.5,0.4,0.1). Find the joint probability of hidden path H,C,H and observations 1,3,1.

Given

Hidden path H→C→H. Observations 1→3→1. Model probabilities are specified in the question. There is no END factor.

To find

P(H,C,H; 1,3,1).

Formula and why it applies

\"P=\\pi_H

There are three hidden states and observations: three emissions, two transitions, and one initial probability.

Solution

Step 1. Start Hot and emit 1
\"0.8\\times0.2=0.16\"
Step 2. Transition Hot to Cold
\"0.16\\times0.3=0.048\"
Step 3. Cold emits 3
\"0.048\\times0.1=0.0048\"
Step 4. Transition Cold to Hot
\"0.0048\\times0.4=0.00192\"
Step 5. Hot emits the last 1
\"0.00192\\times0.2=0.000384\"
Final answer. Joint probability = 0.000384 = 3.84×10⁻⁴.

Question. For π(H)=0.8, π(C)=0.2; H→H=0.7, H→C=0.3, C→H=0.4, C→C=0.6; emissions H:(0.2,0.4,0.4), C:(0.5,0.4,0.1) for labels (1,2,3), find the joint probability of path H,H,H and observations 1,3,1.

Given

State path H→H→H, observations 1→3→1; no END factor.

To find

P(H,H,H;1,3,1).

Formula and why it applies

\"P=\\pi_H

The state remains Hot; each transition still contributes its probability 0.7.

Solution

Step 1. Initial state and first emission
\"0.8(0.2)=0.16\"
Step 2. First self-transition
\"0.16(0.7)=0.112\"
Step 3. Emit observation 3
\"0.112(0.4)=0.0448\"
Step 4. Second self-transition
\"0.0448(0.7)=0.03136\"
Step 5. Emit final observation 1
\"0.03136(0.2)=0.006272\"
Final answer. Joint probability = 0.006272. A self-transition is not automatically probability 1.

Question. For π(H)=0.8, π(C)=0.2; transitions H:(0.7,0.3), C:(0.4,0.6) in next-state order H,C; emissions H:(0.2,0.4,0.4), C:(0.5,0.4,0.1) for observations 1,2,3, find the score of path C,C,H with observations 1,3,1.

Given

Hidden path C→C→H. Observation sequence 1→3→1. No end-state factor.

To find

The joint path-and-observation probability.

Formula and why it applies

\"P=\\pi_C

The initial state is Cold, so use π(C), not π(H). Each emission is selected from the state at the same position.

Solution

Step 1. Start Cold and emit 1
\"0.2(0.5)=0.1\"
Step 2. Stay Cold
\"0.1(0.6)=0.06\"
Step 3. Emit 3 from Cold
\"0.06(0.1)=0.006\"
Step 4. Move Cold to Hot
\"0.006(0.4)=0.0024\"
Step 5. Emit 1 from Hot
\"0.0024(0.2)=0.00048\"
Final answer. Joint probability = 0.00048.

Yaad rakho: Har column par: state ka emission. Columns ke beech: transition.

"},{"id":"tagging-corpus-case","title":"POS HMM worked corpus: Mary, Jane, can, will and spot","html":"

The class example contains words that can be either names or ordinary verbs/modal auxiliaries. The term is modal (M), not “model”. To avoid guessing tags from spelling, the training corpus below supplies them explicitly and lowercases words for the emission vocabulary.

Training sentenceGold tags
mary jane can see willN N M V N
spot will see maryN M V N
will jane spot maryM N V N
mary will pat spotN M V N

There are 9 noun tokens, 4 modal tokens and 4 verb tokens. Each sentence includes a start and end transition. For nouns, the next event can be another noun, modal, verb or END.

Current → nextNMVEND
START3/41/400
N1/93/91/94/9
M1/403/40
V4/4000
Tag → wordmaryjanecanseewillspotpat
N4/92/9001/92/90
M001/403/400
V0002/401/41/4

Question. Using the fully tagged four-sentence corpus above, calculate the joint score of “will jane spot mary” assigned tags M,N,V,N, including END.

Given

Required factors from the tables: START→M=1/4; emit will|M=3/4; M→N=1/4; jane|N=2/9; N→V=1/9; spot|V=1/4; V→N=1; mary|N=4/9; N→END=4/9.

To find

The specified tag-path score, including words and sentence end.

Formula and why it applies

\"P=\\pi_M

Both transitions and emissions are needed. A tag transition alone cannot tell us whether a particular word is plausible.

Solution

Step 1. Start and emit will
\"\\frac14\\times\\frac34=\\frac3{16}\"
Step 2. Move to N and emit jane
\"\\frac3{16}\\times\\frac14\\times\\frac29=\\frac1{96}\"
Step 3. Move to V and emit spot
\"\\frac1{96}\\times\\frac19\\times\\frac14=\\frac1{3456}\"
Step 4. Move to N and emit mary
\"\\frac1{3456}\\times1\\times\\frac49=\\frac1{7776}\"
Step 5. End the sentence
\"\\frac1{7776}\\times\\frac49=\\frac1{17496}\\approx0.0000571559\"
Final answer. The joint score is 1/17496 ≈ 0.0000571559.

Question. For the same tagged corpus and tables, explain the unsmoothed score of “will can spot mary” assigned tags M,V,N,N.

Given

The proposed second word is can with hidden tag V. In the training corpus, can occurs only with M, so b_V(can)=0.

To find

Whether the complete path score is zero and what this means.

Formula and why it applies

\"P(\\text{path,words})=(\\text{other

A product containing one zero factor is zero even if every other factor is positive.

Solution

Step 1. Find the critical emission
\"b_V(can)=C(can,V)/C(V)=0/4=0\"
Step 2. Use the full-product consequence
\"P(\\text{path,words})=0\"
Step 3. Interpret correctly

This unsmoothed corpus model has no evidence for can as a verb. It does not prove that a verb usage of can is impossible in English.

Final answer. The unsmoothed path score is zero.
"}]},{"id":"m2-hmm-decoding","title":"Choosing the most likely tag sequence","unit":4,"scope":"Unit 4 page 73: Viterbi motivation, explained with small worked extensions","sections":[{"id":"sequence-count","title":"Why exhaustive search grows exponentially","html":"

If each of T observed words can receive any of N tags, there are N choices at each position. Thus the total number of complete tag sequences is N multiplied by itself T times. This counts possible assignments before grammar rules or zero probabilities eliminate any.

\"\\text{Number

Question. A four-word sentence is tagged using three possible POS tags per word. How many complete tag sequences are possible?

Given

N=3 possible tags per position; T=4 word positions.

To find

The number of tag sequences.

Formula and why it applies

\"M=N^T\"

Every independent position contributes a factor of three choices.

Solution

Step 1. Write one factor per position
\"M=3\\times3\\times3\\times3\"
Step 2. Multiply
\"3\\times3=9,\\quad9\\times3=27,\\quad27\\times3=81\"
Final answer. 81 possible tag sequences.

Question. A five-word sentence has two candidate tags per word. Count its tag sequences.

Given

N=2; T=5.

To find

M.

Formula and why it applies

\"M=N^T\"

The length is the exponent, not a multiplier.

Solution

Step 1. Substitute
\"M=2^5\"
Step 2. Evaluate
\"2\\times2\\times2\\times2\\times2=32\"
Final answer. 32 possible sequences.

Question. Using 45 possible tags, compare the number of tag sequences for sentences of length 2 and 3.

Given

N=45. First T=2, then T=3.

To find

Both counts and the growth factor.

Formula and why it applies

\"M_2=45^2,\\quad

Adding one position multiplies the possibilities by the number of tag choices.

Solution

Step 1. Two words
\"45^2=2025\"
Step 2. Three words
\"45^3=2025\\times45=91125\"
Step 3. Compare
\"91125/2025=45\"
Final answer. 2 words: 2025 paths. 3 words: 91125 paths. One additional word multiplies the count by 45.

The slides introduce Viterbi to avoid enumerating every path. It reuses the best prefix score for each ending state. For a dense first-order HMM, it needs O(TN²) recurrence work instead of enumerating NT complete paths.

Yaad rakho: Choices per word ki power = sentence length.

"},{"id":"viterbi-explained","title":"Viterbi decoding: compare incoming paths, keep the best, trace back","html":"

The Viterbi algorithm finds the most likely hidden-state sequence for a fixed observation sequence. At each position, it keeps the best path ending in each state. This is safe because, under the HMM assumptions, future extensions depend on that ending state, not the discarded details of the prefix.

\"\\delta_1(j)=\\pi_jb_j(o_1)\"
\"\\delta_t(j)=b_j(o_t)\\max_i[\\delta_{t-1}(i)a_{ij}]\"
\"\\psi_t(j)=\\arg\\max_i[\\delta_{t-1}(i)a_{ij}]\"
SymbolMeaning in ordinary words
δₜ(j)Score of the best prefix that ends in state j at position t.
ψₜ(j)Previous state that supplied that best prefix; keep it as a backpointer.
maxChoose the larger candidate score; do not add.
argmaxRemember which previous state won, rather than only its value.
Initialize: start × first emission
→
For each next state: compare incoming scores
→
Multiply winner by this state's emission
→
Keep score + winning predecessor
→
Choose best final state and trace arrows backward

Do not choose the best state independently at each word. The best last state can come from a different best predecessor than a local guess. Backpointers recover one consistent complete path. If scores tie, either tied path is optimal under the model; state the tie.

Question. Use an HMM with π(H)=0.8, π(C)=0.2, transitions H:(0.7,0.3), C:(0.4,0.6) in H,C order, and emissions H:(0.2,0.4,0.4), C:(0.5,0.4,0.1) for observations 1,2,3. Find the most likely hidden path for observations 1,3,1 using Viterbi.

Given

Initial probabilities: π(H)=0.8, π(C)=0.2. Transition matrix: H→H=0.7, H→C=0.3, C→H=0.4, C→C=0.6. Emissions for labels (1,2,3): H=(0.2,0.4,0.4), C=(0.5,0.4,0.1). There is no END-state factor.

To find

The best hidden-state sequence and its joint score.

Formula and why it applies

\"\\delta_1(j)=\\pi_jb_j(o_1),\\quad\\delta_t(j)=b_j(o_t)\\max_i[\\delta_{t-1}(i)a_{ij}]\"

Viterbi keeps two best prefix scores per column, one ending H and one ending C.

Solution

Step 1. Initialize at observation 1
\"\\delta_1(H)=0.8(0.2)=0.16\"
\"\\delta_1(C)=0.2(0.5)=0.10\"
Step 2. At observation 3, calculate the H cell
\"H\\to
\"C\\to
\"\\delta_2(H)=\\max(0.112,0.040)(0.4)=0.0448\"

Backpointer for this H cell: H.

Step 3. At observation 3, calculate the C cell
\"H\\to
\"C\\to
\"\\delta_2(C)=\\max(0.048,0.060)(0.1)=0.006\"

Backpointer for this C cell: C.

Step 4. At final observation 1, calculate H
\"H\\to
\"C\\to
\"\\delta_3(H)=0.03136(0.2)=0.006272\"

Backpointer: H.

Step 5. At final observation 1, calculate C
\"H\\to
\"C\\to
\"\\delta_3(C)=0.01344(0.5)=0.00672\"

Backpointer: H.

Step 6. Select the final state

0.00672 > 0.006272, so the best final state is C.

Step 7. Trace back

Final C came from H at position2. That H came from H at position1. Reading forward gives H,H,C.

Step 8. Verify the recovered complete path
\"0.8(0.2)(0.7)(0.4)(0.3)(0.5)=0.00672\"
Final answer. Best path H→H→C with joint score 0.00672.

Question. For π(H)=0.8, π(C)=0.2; transitions H:(0.7,0.3), C:(0.4,0.6); emissions H:(0.2,0.4,0.4), C:(0.5,0.4,0.1) for labels1,2,3, decode observations 3,1.

Given

Initial probabilities: π(H)=0.8, π(C)=0.2. Transition matrix: H→H=0.7, H→C=0.3, C→H=0.4, C→C=0.6. Emissions for labels (1,2,3): H=(0.2,0.4,0.4), C=(0.5,0.4,0.1). There is no END-state factor.

To find

Best two-state hidden path and score.

Formula and why it applies

\"\\delta_1(j)=\\pi_jb_j(3),\\quad\\delta_2(j)=b_j(1)\\max_i[\\delta_1(i)a_{ij}]\"

The same recurrence applies even though the observation order changed.

Solution

Step 1. Initialize observation 3
\"\\delta_1(H)=0.8(0.4)=0.32,\\quad\\delta_1(C)=0.2(0.1)=0.02\"
Step 2. Calculate the final H
\"\\max(0.32(0.7),0.02(0.4))(0.2)\"
\"=\\max(0.224,0.008)(0.2)=0.0448\"

Winning predecessor H.

Step 3. Calculate the final C
\"\\max(0.32(0.3),0.02(0.6))(0.5)\"
\"=\\max(0.096,0.012)(0.5)=0.048\"

Winning predecessor H.

Step 4. Select and trace

Final C wins with 0.048 and came from H. The path is H,C.

Final answer. Best path H→C, score 0.048.

Question. For π(H)=0.8, π(C)=0.2; transitions H:(0.7,0.3), C:(0.4,0.6); emissions H:(0.2,0.4,0.4), C:(0.5,0.4,0.1) for labels1,2,3, decode observations 2,2.

Given

Initial probabilities: π(H)=0.8, π(C)=0.2. Transition matrix: H→H=0.7, H→C=0.3, C→H=0.4, C→C=0.6. Emissions for labels (1,2,3): H=(0.2,0.4,0.4), C=(0.5,0.4,0.1). There is no END-state factor.

To find

Best path and score.

Formula and why it applies

\"\\delta_1(j)=\\pi_jb_j(2),\\quad\\delta_2(j)=b_j(2)\\max_i[\\delta_1(i)a_{ij}]\"

Both states emit2 with probability 0.4, but their initial and transition probabilities still differ.

Solution

Step 1. Initialize
\"\\delta_1(H)=0.8(0.4)=0.32,\\quad\\delta_1(C)=0.2(0.4)=0.08\"
Step 2. Final H
\"\\max(0.32(0.7),0.08(0.4))(0.4)\"
\"=\\max(0.224,0.032)(0.4)=0.0896\"

Predecessor H.

Step 3. Final C
\"\\max(0.32(0.3),0.08(0.6))(0.4)\"
\"=\\max(0.096,0.048)(0.4)=0.0384\"

Predecessor H.

Step 4. Select and trace

The final H score 0.0896 is larger. Its predecessor is H.

Final answer. Best path H→H with score 0.0896.

Yaad rakho: Har destination ke liye incoming paths compare; winner ka arrow yaad.

"},{"id":"joint-vs-posterior","title":"A path score is not the probability of the observed sentence","html":"

Question. An HMM has π(H)=0.8, π(C)=0.2; transitions H:(0.7,0.3), C:(0.4,0.6); emissions H:(0.2,0.4,0.4), C:(0.5,0.4,0.1) for labels1,2,3. For observations 1,3, calculate the probability of the observations by summing all four two-state paths.

Given

Initial probabilities: π(H)=0.8, π(C)=0.2. Transition matrix: H→H=0.7, H→C=0.3, C→H=0.4, C→C=0.6. Emissions for labels (1,2,3): H=(0.2,0.4,0.4), C=(0.5,0.4,0.1). There is no END-state factor.

To find

P(observations 1,3), not just the best-path score.

Formula and why it applies

\"P(o_1,o_2)=\\sum_{s_1,s_2}\\pi_{s_1}b_{s_1}(o_1)a_{s_1s_2}b_{s_2}(o_2)\"

The hidden path is unspecified, so its mutually exclusive alternatives must be summed.

Solution

Step 1. Score H,H
\"0.8(0.2)(0.7)(0.4)=0.0448\"
Step 2. Score H,C
\"0.8(0.2)(0.3)(0.1)=0.0048\"
Step 3. Score C,H
\"0.2(0.5)(0.4)(0.4)=0.016\"
Step 4. Score C,C
\"0.2(0.5)(0.6)(0.1)=0.006\"
Step 5. Add all paths
\"P(1,3)=0.0448+0.0048+0.016+0.006=0.0716\"
Final answer. Observation probability =0.0716. The best single-path score 0.0448 is a different quantity.

If asked for the posterior of path H,H given observations 1,3, normalize: 0.0448/0.0716 ≈0.625698. Viterbi chooses the best path using joint scores because the observation denominator is common to every candidate. It does not compute that denominator by taking a maximum.

"}]},{"id":"m2-chunks-grammar","title":"Chunking, chinking and context-free grammar","unit":4,"scope":"Unit 4, pages 14–19, 74–97","sections":[{"id":"chunking-chinking","title":"Chunking groups words; chinking removes words from a candidate chunk","html":"

Chunking, also called shallow parsing, groups adjacent tagged words into meaningful non-overlapping units, commonly noun phrases. It usually identifies local phrase boundaries without producing the complete hierarchical structure of a sentence. Chinking removes specified tokens from an existing candidate chunk, splitting or trimming it.

The/DT small/JJ cat/NN sat/VBD on/IN the/DT mat/NN
→
NP: [The small cat]
→
Outside: sat on
→
NP: [the mat]

A POS tagger labels individual words. A chunker groups words into phrases. A full parser shows how phrases relate recursively to form the entire sentence. These are related tasks with different outputs.

OperationInputOutput
POS taggingThe small catThe/DT small/JJ cat/NN
NP chunkingTagged tokens[NP The small cat]
ChinkingA broad candidate chunkRemove verbs/prepositions, keep noun-phrase segments
Full parsingSentence + grammarA nested tree with sentence, noun phrase and verb phrase nodes

Rule-based chunking uses tag patterns. Statistical or learned chunkers infer boundaries from labeled examples. A useful NP rule is an optional determiner, followed by zero or more adjectives, followed by a noun. In NLTK-style notation: NP: {<DT>?<JJ>*<NN>}. Here ? means zero or one occurrence; * means zero or more; + means one or more when used. The braces mean “form a chunk around the matched sequence”.

Question. Apply the NP rule optional DT, zero or more JJ, then one NN to “The/DT quick/JJ brown/JJ fox/NN jumps/VBZ over/IN the/DT lazy/JJ dog/NN”.

Given

Tagged input is supplied. Rule: DT? JJ* NN.

To find

All noun-phrase chunks.

Formula and why it applies

NP → DT? JJ* NN

The rule groups a noun with its preceding determiner and adjectives; a verb or preposition does not match this NP pattern.

Solution

Step 1. Start at The

The matches optional DT; quick and brown both match JJ*; fox matches NN.

Step 2. Form the first NP

[NP The quick brown fox]

Step 3. Skip nonmatching tokens

jumps/VBZ and over/IN do not satisfy the NP pattern.

Step 4. Match the second NP

the is DT, lazy is JJ, dog is NN: [NP the lazy dog].

Final answer. [NP The quick brown fox] jumps over [NP the lazy dog].

Question. Using NP → DT? JJ* NN, identify the chunks in “bright/JJ stars/NN shine/VBP and/CC a/DT moon/NN glows/VBZ”. Use the supplied tags exactly.

Given

Input tags label stars as NN for this exercise. Rule requires optional DT, zero or more JJ, then NN.

To find

The matched NP spans.

Formula and why it applies

NP → DT? JJ* NN

The optional determiner may be absent and the adjective repetition may be empty.

Solution

Step 1. Match bright stars

No determiner is present, which DT? permits. bright matches JJ; stars matches NN.

Step 2. Inspect shine and and

VBP and CC fall outside the NP rule.

Step 3. Match a moon

a matches DT; no adjective is present; moon matches NN.

Final answer. Chunks are [NP bright stars] and [NP a moon]. With ordinary Penn tagging, stars is usually NNS; broaden the noun pattern to include NNS when that tag is used.

Question. Begin with one candidate chunk containing “The/DT cat/NN sat/VBD on/IN the/DT mat/NN”. Chink out verbs and prepositions. Show the remaining chunks.

Given

Initial candidate: [The cat sat on the mat]. Remove tags VBD and IN.

To find

The chunk segments after exclusion.

Formula and why it applies

Chinking removes matched tokens from the existing chunk.

Removing the middle verb/preposition region divides the original candidate into two remaining contiguous spans.

Solution

Step 1. Locate the tokens to remove

sat is VBD and on is IN.

Step 2. Remove those tokens from chunk membership

The remaining token groups are “The cat” and “the mat”. sat and on still exist in the sentence; they are simply outside these chunks.

Step 3. Write the final chunked sentence

[The cat] sat on [the mat].

Final answer. Two chunks remain: The cat; the mat.

Yaad rakho: Chunking = andar group karo; chinking = unwanted tokens ko group se bahar karo.

"},{"id":"cfg-foundation","title":"CFG: terminals, nonterminals, production rules and parse trees","html":"

A context-free grammar (CFG) specifies a language using production rules whose left side is a single nonterminal. Formally, G=(N,Σ,R,S): N is the set of nonterminals, Σ the terminal vocabulary, R the production rules, and S the start symbol. A rule can expand its nonterminal regardless of the neighboring symbols; that is the meaning of “context-free”.

ConceptPlain explanationExample
TerminalA word at a leaf; it is not expanded furtherthe, girl, read
NonterminalA grammatical category that can be expandedS, NP, VP, Det, N, V
ProductionA permitted replacementS → NP VP
Start symbolCategory for a complete sentenceS
LexiconWord-to-category rulesN → girl | paper
DerivationSequence of rule applicationsS ⇒ NP VP ⇒ Det N VP ⇒ …
Parse treeHierarchical picture of those expansionsRoot S, internal phrases, word leaves
\"S\\to

Read S→NP VP as “a sentence can contain a noun phrase followed by a verb phrase”. It does not mean subtraction or multiplication. A vertical bar in a grammar rule means alternatives: Det→the | a allows either the or a. This is different from the probability bar meaning “given”.

S: the whole sentence
→
NP: the girl + VP: read the paper
→
NP→Det N; VP→V NP
→
Leaves in order: the / girl / read / the / paper

A constituent is a group of words that acts as a grammatical unit, such as “the girl”. Syntactic parsing assigns a grammatical structure to an input sentence using a grammar. It can support grammar checking, information extraction, translation and later semantic analysis. Failure under one restricted grammar means that grammar could not derive the sentence; it does not necessarily prove the sentence is invalid English.

A treebank is a corpus annotated with parse trees. It supplies training/evaluation structures for parsers and lets linguists study recurring syntactic patterns. A tagged corpus gives word tags; a treebank additionally records relationships among phrases.

RepresentationExample information retained
Bag of wordsWords occur, but word order/grammar is lost
POS tagsEach word's local grammatical category
ChunksShort phrase boundaries
Parse treeNested constituent structure of the whole sentence

Yaad rakho: Grammar rule ek expansion instruction hai: S ko NP aur VP mein kholo.

"},{"id":"structural-ambiguity","title":"Ambiguity and grammar coverage","html":"

Structural ambiguity occurs when a sentence has more than one valid parse tree. In “I saw the man with a telescope”, the prepositional phrase “with a telescope” can attach to saw (the instrument I used) or to the man (the man possessing the telescope). The words and their order stay the same, but the grouping changes.

ReadingAttachmentMeaning
I [saw the man] [with a telescope]PP modifies VPI used a telescope to see the man.
I saw [the man with a telescope]PP modifies NPThe man had a telescope.

A parser may need context, probabilities or semantic constraints to choose between trees. A plain CFG describes permitted structures but does not itself assign a probability to each tree. Do not invent rule probabilities for an ordinary CFG question.

Lexical ambiguity also matters: book can be a noun or verb. In “Book the flight”, the imperative analysis begins S→VP, followed by VP→V NP. A parser that initially chooses book as a noun may reach a dead end and must try another permitted analysis.

Yaad rakho: Same words, alag grouping → alag meaning.

"}]},{"id":"m2-parsing","title":"Top-down and bottom-up parsing: complete traces","unit":4,"scope":"Unit 4, pages 98–116","sections":[{"id":"top-down-theory","title":"Recursive-descent top-down parsing","html":"

Top-down parsing starts at the grammar's start symbol S and expands nonterminals until the leaves match the input words in order. Recursive descent commonly implements each nonterminal as a procedure that tries its productions. It builds from the root toward the leaves.

Start with S
→
Choose an allowed S production
→
Expand its next nonterminal
→
Match the resulting terminal to the input
→
Continue, or backtrack if the choice fails

Algorithm. Start with S as the current goal. Choose a production for the leftmost unexpanded nonterminal. Replace that nonterminal by the production's right side. If a terminal appears, compare it with the next input token. If it matches, consume the token; if it does not, reject that branch and try another production. Accept only when the complete input is consumed and all grammar goals are satisfied.

A leftmost derivation always expands the leftmost remaining nonterminal first. Top-down parsing can make guesses that the actual words cannot support, so it may need backtracking. A naive recursive-descent parser can loop on left-recursive rules such as NP→NP PP unless the grammar or algorithm is changed. Predictive LL(1) parsing chooses productions using one lookahead token when the grammar permits; not every English CFG is LL(1).

ApproachBegins withDirectionCommon difficulty
Top-downStart symbol SRoot to word leavesPredictions that do not match input; left recursion
Bottom-upActual input tokensLeaves to root SReductions that cannot contribute to a complete sentence

Yaad rakho: Top-down: sentence ka structure pehle, words tak neeche aao.

"},{"id":"top-down-examples","title":"Three full top-down derivations with parse trees","html":"

Question. Using S→NP VP, NP→Det N, VP→V NP, Det→the, N→girl|paper and V→read, give a leftmost top-down derivation and parse tree for “the girl read the paper”.

Given

Rule typeRules
StructureS → NP VP; NP → Det N; VP → V NP
LexiconDet → the; N → girl | paper; V → read

To find

A successful derivation and tree whose leaves equal the input.

Formula and why it applies

At every stage expand the leftmost nonterminal using a supplied production.

A top-down derivation begins at S; the input determines which terminal alternatives can succeed.

Solution

Step 1. Begin at the sentence symbol

S

Step 2. Expand the sentence

S ⇒ NP VP

Step 3. Expand its first noun phrase

⇒ Det N VP

Step 4. Match the first determiner

⇒ the N VP

Step 5. Match the subject noun

⇒ the girl VP

Step 6. Expand the verb phrase

⇒ the girl V NP

Step 7. Match the verb

⇒ the girl read NP

Step 8. Expand the object noun phrase

⇒ the girl read Det N

Step 9. Match its determiner

⇒ the girl read the N

Step 10. Match the last noun

⇒ the girl read the paper

Step 11. Draw the tree
Parse tree: the girl read the paperSNPDettheNgirlVPVreadNPDettheNpaper
Parse tree: the girl read the paper
Final answer. The grammar derives the complete sentence. The subject NP is “the girl”; the object NP is “the paper”.

Question. Use S→NP VP, NP→Det N|N, VP→V NP, Det→A, N→restaurant|Dosas and V→serves to parse “A restaurant serves Dosas” by a leftmost top-down derivation.

Given

Rule typeRules
StructureS → NP VP; NP → Det N | N; VP → V NP
LexiconDet → A; N → restaurant | Dosas; V → serves

To find

A derivation and parse tree.

Formula and why it applies

Expand the leftmost nonterminal; choose NP→Det N for the subject and NP→N for the object.

The object has no determiner, so forcing NP→Det N there would fail.

Solution

Step 1. Start and expand S

S ⇒ NP VP

Step 2. Choose the subject NP

⇒ Det N VP

Step 3. Match A

⇒ A N VP

Step 4. Match restaurant

⇒ A restaurant VP

Step 5. Expand the VP

⇒ A restaurant V NP

Step 6. Match serves

⇒ A restaurant serves NP

Step 7. Choose the object NP rule

⇒ A restaurant serves N

Step 8. Match Dosas

⇒ A restaurant serves Dosas

Step 9. Draw the successful tree
Parse tree: A restaurant serves DosasSNPDetANrestaurantVPVservesNPNDosas
Parse tree: A restaurant serves Dosas
Final answer. The parse succeeds with subject NP→Det N and object NP→N.

Question. Using S→NP VP|VP, NP→Det N, VP→V NP, Det→that, N→book|flight and V→book, derive the imperative “book that flight” and explain the role of book.

Given

Rule typeRules
StructureS → NP VP | VP; NP → Det N; VP → V NP
LexiconDet → that; N → book | flight; V → book

To find

The successful derivation, tree and POS of book.

Formula and why it applies

Choose a production for S that can derive the actual input; an imperative may consist of VP.

The grammar allows book as both noun and verb, but this imperative needs the verb reading.

Solution

Step 1. Choose the imperative sentence rule

S ⇒ VP

Step 2. Expand the verb phrase

⇒ V NP

Step 3. Choose the verb book

⇒ book NP

Step 4. Expand the object

⇒ book Det N

Step 5. Match that

⇒ book that N

Step 6. Match flight

⇒ book that flight

Step 7. Draw the tree
Imperative parse: book that flightSVPVbookNPDetthatNflight
Imperative parse: book that flight
Final answer. book is V; that flight is NP; S→VP derives the imperative.
"},{"id":"shift-reduce-theory","title":"Shift-reduce bottom-up parsing: stack, input and actions","html":"

Bottom-up parsing starts from the input tokens and combines them into larger constituents until the start symbol is reached. A shift-reduce parser maintains a stack of recognized material and a list of remaining input tokens.

ActionExact operationExample
ShiftMove the next input token onto the stackStack: the; remaining input: dog runs
ReduceReplace a suffix of the stack matching a rule's RHS with its LHSDet N becomes NP using NP→Det N
AcceptStack is S and input is exhaustedStack: S; input: empty
Fail/backtrackChosen reductions cannot complete a parseA noun reading of book blocks an imperative parse

Reduction uses a production in reverse: the grammar NP→Det N expands an NP top-down, but reduces Det N into NP bottom-up. The stack's rightmost item is its top in the traces below. Each row shows the state after the listed action, avoiding an off-by-one mismatch between actions and stack contents.

A lexical reduction such as N→dog still counts as a reduction. Do not skip the sequence from word to category to phrase when the question asks for a complete trace. Backtracking means returning to an earlier choice and trying another applicable rule, not changing the original input.

Yaad rakho: Shift = input se stack. Reduce = chhote pieces se bada grammatical unit.

"},{"id":"shift-reduce-examples","title":"Three full shift-reduce traces","html":"

Question. Show a complete shift-reduce parse of “the big dog sees the cat” using S→NP VP, NP→Det N|Det Adj N, VP→V NP, Det→the, Adj→big, N→dog|cat and V→sees.

Given

Grammar is as specified. Stack starts empty; input is the big dog sees the cat. Top of stack is on the right.

To find

The stack, remaining input and action at every step.

Formula and why it applies

Shift tokens; reduce matched right-hand sides to their left-hand nonterminals.

The first noun phrase includes an adjective and must use NP→Det Adj N.

Solution

Step 1. Trace each action
Step / actionStack after actionRemaining input
0. Start∅the big dog sees the cat
1. Shift thethebig dog sees the cat
2. Reduce Det→theDetbig dog sees the cat
3. Shift bigDet bigdog sees the cat
4. Reduce Adj→bigDet Adjdog sees the cat
5. Shift dogDet Adj dogsees the cat
6. Reduce N→dogDet Adj Nsees the cat
7. Reduce NP→Det Adj NNPsees the cat
8. Shift seesNP seesthe cat
9. Reduce V→seesNP Vthe cat
10. Shift theNP V thecat
11. Reduce Det→theNP V Detcat
12. Shift catNP V Det cat∅
13. Reduce N→catNP V Det N∅
14. Reduce NP→Det NNP V NP∅
15. Reduce VP→V NPNP VP∅
16. Reduce S→NP VPS∅
Step 2. Check acceptance

Input is empty and stack is exactly S; therefore accept.

Step 3. Draw the resulting structure
Parse tree: the big dog sees the catSNPDettheAdjbigNdogVPVseesNPDettheNcat
Parse tree: the big dog sees the cat
Final answer. Accepted. The parser builds both noun phrases, then VP, then S.

Question. Use S→NP VP, NP→Det N, VP→V NP, Det→the|this, N→man|book and V→read to give a shift-reduce trace for “the man read this book”.

Given

The input has five tokens. Stack is empty initially. The lexical verb read is supplied by V→read.

To find

A complete bottom-up trace.

Formula and why it applies

Reduce only when the top stack sequence matches a supplied production's right side.

Both noun phrases follow Det N, and the verb phrase follows V NP.

Solution

Step 1. Trace all operations
Step / actionStack after actionRemaining input
0. Start∅the man read this book
1. Shift thetheman read this book
2. Reduce Det→theDetman read this book
3. Shift manDet manread this book
4. Reduce N→manDet Nread this book
5. Reduce NP→Det NNPread this book
6. Shift readNP readthis book
7. Reduce V→readNP Vthis book
8. Shift thisNP V thisbook
9. Reduce Det→thisNP V Detbook
10. Shift bookNP V Det book∅
11. Reduce N→bookNP V Det N∅
12. Reduce NP→Det NNP V NP∅
13. Reduce VP→V NPNP VP∅
14. Reduce S→NP VPS∅
Step 2. Verify the final state

The complete sentence has reduced to S with no input left.

Final answer. Accepted. Subject: the man. Predicate: read this book. Object: this book.

Question. Parse “book the flight” bottom-up using S→NP VP|VP, NP→Det N, VP→V NP, Det→the, N→book|flight, V→book. Explain why choosing the noun reading first can fail and show the successful alternative.

Given

book has two lexical categories N and V. The input is an imperative. Stack starts empty.

To find

The failed choice and a complete successful trace with backtracking.

Formula and why it applies

Try lexical reductions allowed by the grammar; on a dead end, return to the ambiguous choice and try another.

Grammar ambiguity requires a search choice. The noun reading cannot supply the required imperative verb.

Solution

Step 1. Show the noun-first dead end

Shift book and reduce it to N. Continue shifting/reducing the flight to Det N. Reducing Det N→NP leaves stack N NP and empty input. No supplied rule reduces N NP into S, so this branch fails.

Step 2. Return to the lexical choice

Restore the state just after shifting book, then choose V→book instead of N→book.

Step 3. Trace the successful branch
Step / actionStack after actionRemaining input
0. Restored after shifting bookbookthe flight
1. Reduce V→bookVthe flight
2. Shift theV theflight
3. Reduce Det→theV Detflight
4. Shift flightV Det flight∅
5. Reduce N→flightV Det N∅
6. Reduce NP→Det NV NP∅
7. Reduce VP→V NPVP∅
8. Reduce S→VPS∅
Step 4. Accept the alternative

The verb reading produces S and consumes all tokens.

Final answer. The noun-first branch fails; backtracking to book/V yields the successful imperative parse.

Yaad rakho: Wrong branch fail hua toh sentence wrong nahi; next legal rule try karo.

"}]},{"id":"m2-ner","title":"Named Entity Recognition","unit":4,"scope":"Unit 4, pages 117–128","sections":[{"id":"ner-definition","title":"Entity spans, categories and context","html":"

Named Entity Recognition (NER) identifies text spans referring to entities and assigns their types, such as PERSON, ORGANIZATION and LOCATION. Many practical schemas also include DATE, TIME, MONEY or domain-specific types. The exact allowed labels depend on the application's annotation scheme.

NER has two parts: boundary detection finds where the entity starts and ends; classification assigns the appropriate type. A multiword name should normally be one entity span: “New York” is a location span, not two unrelated location decisions.

Text: Mary joined Google in New York on Monday.
→
Mary → PERSON
→
Google → ORGANIZATION
→
New York → LOCATION
→
Monday → DATE

Not every noun is a named entity. In “a developer works at Google”, developer is an ordinary job description while Google is a named organization. A specialized HR system may define a JOB_TITLE category, but it must state that schema explicitly. NER identifies mentions; entity linking is the separate task of connecting a mention to a unique database identity.

TaskQuestion it answersExample
POSWhat grammatical role does this token have?Google/NNP
ChunkingWhich adjacent words form a phrase?[NP a skilled developer]
NERWhich span names an entity, and of what type?[Google] ORGANIZATION
Entity linkingWhich real entity does this mention denote?Connect Paris to the intended city or person record

Applications. NER structures search results and documents, extracts organizations/people from news, routes support tickets by product or organization, supports recommendation, and extracts specialized entities from healthcare or legal text. Context is essential: “Apple released a laptop” names an organization, while “she ate an apple” describes fruit.

Yaad rakho: NER mein do kaam: naam kahan se kahan tak, aur kis category ka.

"},{"id":"ner-approaches","title":"Dictionary, rules, machine learning and deep learning","html":"
ApproachHow it worksExampleMain limitation
Dictionary / gazetteerMatch text against lists of known namesA city list includes Mumbai and PuneNew names, spelling variants and ambiguous names are missed
Rule-basedUse word shapes, regular expressions and contextA title such as Dr. followed by a name-like span suggests PERSONManual maintenance; exceptions and domain/language dependence
Machine learningLearn a labeling rule from annotated examples using featuresCapitalization, suffixes, POS and surrounding words help a classifier/CRFRequires labeled data and useful features
Deep learningLearn contextual word/character representations and predict labelsA contextual sequence encoder labels multiword entity spansTraining/domain dependence and resource demands

A pattern-based rule examines the form of text, such as an email pattern or a date format. A context-based rule examines neighboring words: “Dr.” can support a following PERSON span, while “headquartered in” can support a following LOCATION span. Capitalization alone is insufficient because sentence-initial common nouns are also capitalized.

A dictionary may find “May” as a person's name, but “in May” may refer to a month. A learned model uses surrounding evidence to disambiguate. Hybrid systems can combine dictionaries, rules and learned models; these categories are practical tools rather than mutually exclusive restrictions.

Preprocessing for NER. Tokenize while preserving token offsets and useful capitalization. Blindly removing capital letters or stopwords before recognition can damage names such as “Bank of India”. NER output should preserve the original span so the reader can locate it in the input.

Exam-ready 3-mark answer. NER locates named-entity spans and classifies them into categories such as person, organization and location. A dictionary-based approach matches known name lists; a rule-based approach uses patterns or context; machine-learning and deep-learning approaches learn entity decisions from labeled data and representations. For example, in “Mary works at Google in Mumbai”, Mary is PERSON, Google is ORGANIZATION and Mumbai is LOCATION.

"},{"id":"ner-worked-examples","title":"Three complete entity-recognition examples","html":"

Question. Identify PERSON, ORGANIZATION and LOCATION entities in “Asha Rao joined Infosys in Bengaluru.”

Given

Sentence: Asha Rao joined Infosys in Bengaluru. Allowed entity types: PERSON, ORGANIZATION, LOCATION.

To find

Complete entity spans and labels.

Formula and why it applies

Find the span first, then assign its type from context.

A multiword personal name is a single entity; the verb and preposition remain outside entities.

Solution

Step 1. Identify the person span

Asha Rao names the person who joined, so take both words together.

Step 2. Identify the organization

Infosys is the organization joined.

Step 3. Identify the location

Bengaluru names the city.

Step 4. Write the output
SpanLabel
Asha RaoPERSON
InfosysORGANIZATION
BengaluruLOCATION
Final answer. [Asha Rao]PERSON joined [Infosys]ORGANIZATION in [Bengaluru]LOCATION.

Question. Using PERSON, ORGANIZATION, LOCATION and DATE, identify entities in “Dr. Meera Shah visited New Delhi on 5 October 2026.” Treat professional titles as outside the PERSON span for this exercise.

Given

Allowed types and title-boundary convention are specified. Tokens include a multiword name, city and date.

To find

Entity boundaries and types.

Formula and why it applies

Boundary rules come from the annotation schema; use the stated title convention.

Dr. supplies context for a person but is excluded from that span here.

Solution

Step 1. Locate the person

Meera Shah is the PERSON span; Dr. is outside by instruction.

Step 2. Locate the location

New Delhi is one LOCATION span.

Step 3. Locate the date

5 October 2026 is one DATE span.

Step 4. Check organization mentions

No organization is named in this sentence.

Final answer. Meera Shah→PERSON; New Delhi→LOCATION; 5 October 2026→DATE.

Question. Explain the entity label of Apple in “Apple opened an office in Mumbai” and apple in “Ravi ate an apple”. Use PERSON, ORGANIZATION and LOCATION only.

Given

Two contexts contain the same surface word apart from capitalization. The schema contains no FOOD category.

To find

Contextual labels and the ordinary noun case.

Formula and why it applies

NER labels a mention in context, not every occurrence of a dictionary string identically.

A company can open an office; the object eaten in the second sentence is a fruit.

Solution

Step 1. Analyze the first sentence

Apple refers to the company, so label it ORGANIZATION. Mumbai is LOCATION.

Step 2. Analyze the second sentence

Ravi is PERSON. apple is an ordinary fruit noun and receives no entity label under this schema.

Step 3. State the dictionary limitation

Blindly labeling every apple as an organization would produce a false positive.

Final answer. First: Apple→ORGANIZATION, Mumbai→LOCATION. Second: Ravi→PERSON; apple is outside the entity types.
"}]},{"id":"u5-meaning","title":"Word meaning, lexical relations and WordNet","unit":5,"mode":"both","scope":"Unit 5 · PDF pages 1–22","sections":[{"id":"u5-words","title":"Lexeme, lemma, sense and context","html":"

Semantics is the study of meaning in language. Lexical semantics studies meanings of words and relationships between their meanings. Sentence semantics studies how word meanings and sentence structure combine.

A word form is what appears in the sentence. A lexeme is the underlying vocabulary item whose grammatical forms belong together. A lemma is the dictionary form used to name it. A sense is one particular meaning.

TermExampleWhat to understand
Word formsrun, runs, ran, runningDifferent grammatical forms
LexemeRUNThe shared underlying vocabulary item
LemmarunDictionary heading
Senserun a race / run a businessDifferent meanings need context
LexiconA language's vocabulary resourceStores lexical items and information about them

The four forms run, ran, runs, running belong to the same lexeme. They are not four unrelated lexemes. “I deposited cash in the bank” and “We sat by the bank of the river” contain the same written form but different senses.

word: bank
→
context: cash, deposit
→
sense: financial institution

Syntax asks how words are arranged. Semantics asks what the expression means. Pragmatics uses situation and speaker intention: “Can you open the window?” is grammatically a question about ability but is normally a request.

Form = jo likha; lemma = dictionary naam; sense = iss sentence mein matlab.

"},{"id":"u5-relations","title":"Every lexical relation with examples","html":"

A lexical relation connects words or senses by a meaning relationship. State the relation and its direction before giving the example.

RelationExam definitionExamples and direction
HomonymyThe same form has unrelated meanings.bank: financial institution / river edge; bat: animal / sports implement
PolysemyOne lexical item has multiple related senses.head: body part / head of a department; paper: material / article
SynonymyDifferent words have identical or closely similar meanings in a specified sense.sofa / couch; automobile / car
AntonymyWords have opposite meanings.hot / cold; alive / dead; buy / sell
HyponymyA more specific concept is a type of a more general one.rose is a flower; car is a vehicle
HypernymyA broader concept includes a more specific one.flower is a hypernym of rose
MeronymyA concept is a part of another concept.wheel is part of car; finger is part of hand
HolonymyA concept is the whole containing a part.car is a holonym of wheel

How to distinguish the commonly confused pairs

For homonymy versus polysemy, ask whether the senses share an identifiable meaning. A river bank and a financial bank are unrelated. A financial bank and a blood bank share a repository/storage idea, illustrating polysemy in the supplied notes.

For hyponymy versus meronymy, use the sentence test: “X is a kind of Y” indicates hyponymy; “X is part of Y” indicates meronymy. A wheel is not a type of car.

ExpressionCorrect relationReason
sparrow → birdHyponymyA sparrow is a kind of bird
bird → sparrowHypernymyBird is the broader category
keyboard → computerMeronymyKeyboard is a component
computer → keyboardHolonymyComputer is the containing whole
mouth of a person / mouth of a riverPolysemyBoth senses share the idea of an opening; the river use is a related extension

Synonyms are not always interchangeable in every context or register. “Big mistake” and “large mistake” can sound different even though big and large overlap in meaning. Antonyms can be gradable (hot/cold), complementary (alive/dead) or relational converses (buy/sell).

HYPO = specific type; HYPER = bigger category. MERO = part; HOLO = whole.

"},{"id":"u5-wordnet","title":"WordNet: synsets, glosses and hierarchy","html":"

WordNet is a lexical database that groups word senses into synonym sets called synsets and links them through semantic and lexical relations. It is a network of meanings, not merely a list of similar spellings.

A synset represents one concept. Its lemmas are the words that can express that concept. A gloss is a short definition, often accompanied by an example. An ambiguous word can occur in several synsets.

vehicle (general)
→
car / automobile (same concept)
→
sports car (specific)

This is a simplified teaching hierarchy, not a claim about every intermediate WordNet node. Going upward reaches hypernyms; going downward reaches hyponyms. Multiple parents can exist, so the structure need not be a single simple tree.

OperationMeaningIllustration
Get senses of a wordFind candidate synsetsbank → financial sense, river-edge sense, …
Get lemmasFind synonyms within one synsetcar, automobile, motorcar
Get definition/examplesRead the gloss and usesChoose the intended concept
Get hypernymsMove to broader categoriescar → vehicle
Get hyponymsMove to narrower categoriesvehicle → particular vehicle categories
Get meronyms/holonymsFind part/whole linkscar ↔ wheel

WordNet versus a thesaurus

A thesaurus mainly groups related expressions for vocabulary lookup. WordNet explicitly distinguishes senses and labels relations such as hypernymy, hyponymy and meronymy. This supports WSD, query expansion, lexical similarity and semantic search.

Three uses explained

  1. Search expansion: a query for “automobile” can retrieve relevant documents using “car”. Expansion should use the right sense.
  2. WSD: definitions of the senses of “bank” provide words that can be compared with the sentence context.
  3. Similarity: concepts near one another in an “is-a” hierarchy tend to be semantically related. Path-based measures depend on which senses were selected.

WordNet mein spelling ka group nahi—meaning ka synset, aur meanings ke beech labelled links.

"}]},{"id":"u5-wsd","title":"Word sense disambiguation: Lesk, Walker and graph methods","unit":5,"mode":"both","scope":"Unit 5 · PDF pages 23–48","sections":[{"id":"u5-wsd-basics","title":"What WSD solves and how approaches differ","html":"

Word sense disambiguation (WSD) is the task of selecting the intended sense of an ambiguous word in its context from a set of possible senses. The output is a sense label, not a different replacement word.

Input: “He grilled the bass.”
→
Candidate senses: fish / low sound
→
Context: grilled
→
Output: fish sense

The target word is the ambiguous word. The context is the surrounding evidence. The sense inventory lists candidate meanings, often using WordNet. A feature is a usable clue: nearby word, position, part-of-speech tag or a phrase.

ApproachWhat it needsHow it decidesLimitation
Knowledge basedDictionary, thesaurus or semantic graphDefinition overlap, category evidence or graph scoresShort definitions and missing knowledge
SupervisedExamples already labelled with sensesLearns context → sense mappingLabelling cost, limited rare-sense examples
Semi-supervisedSmall labelled set plus unlabelled dataBootstraps from reliable seed examplesEarly errors can spread
Unsupervised sense inductionUnlabelled contextual occurrencesClusters similar contextsClusters are not automatically named dictionary senses
Neural/contextualLearned word/context representationsPredicts or compares a context representationNeeds suitable data/model and evaluation

Bag-of-words features record nearby words while ignoring exact positions. Collocational features preserve a positional phrase such as “river bank”, “bank account”, “chocolate bar” or “bar exam”.

Why WSD matters

Machine translation needs the correct target-language meaning; information retrieval should distinguish river-bank travel from banking services; text-to-speech may need different pronunciations for “bass”; information extraction should avoid connecting entities through the wrong relation.

WSD: same spelling ke options mein context dekh ke sahi meaning select.

"},{"id":"u5-lesk","title":"Lesk algorithm, from words to overlap scores","html":"

Lesk is a knowledge-based WSD method that chooses senses whose dictionary descriptions have the greatest overlap with contextual evidence. Original Lesk compares definitions of neighbouring words' senses. A common simplified version compares each target-sense gloss directly with the sentence context. State which version you use.

Simplified Lesk procedure

  1. Remove the target word from the context; tokenize and normalize the remaining content words.
  2. Obtain the gloss for each candidate sense and apply the same normalization.
  3. Count shared words between the context and each gloss.
  4. Select the sense with maximum overlap. If scores tie, report the tie or use a specified tie-break rule.
\"\\operatorname{score}(s)=|C\\cap

\"C\" is the set of context words, \"G_s\" the set of words in sense \"s\"'s gloss, \"\\cap\" means common elements, \"|\\cdot|\" means count, and \"\\arg\\max\" means “the sense giving the largest score”. This version counts unique matching words once.

Question. Use simplified Lesk to disambiguate “bank” in “The river water rose near the bank.” Use C = {river, water, rose, near}; financial gloss = {institution, money, deposit}; river gloss = {land, river, water, edge}.

Given

The normalized context and two gloss sets are given. Count exact shared words.

To find

The score of each sense and the selected meaning.

Formula and why it applies

Use \"\\operatorname{score}(s)=|C\\cap because the task specifies context–gloss overlap.

Step 1. Compare the financial sense

No word occurs in both {river, water, rose, near} and {institution, money, deposit}. Overlap = {}.

Step 2. Count the financial score

\"\\operatorname{score}(F)=0\".

Step 3. Compare the river sense

Shared words are {river, water}. “Rose” and “near” are not in the gloss.

Step 4. Count and select

\"\\operatorname{score}(R)=2\". Since \"2>0\", choose the river-edge sense.

Answer. Bank means land beside a river.

Question. Use simplified Lesk for “The dog barked loudly.” Treat barked as bark. Context C = {dog, loudly}; sound gloss = {sharp, cry, dog}; tree gloss = {outer, covering, tree}; ship gloss = {sailing, ship}.

Given

Three glosses and normalized context {dog, loudly}.

To find

Select the sense of bark.

Formula and why it applies

\"\\operatorname{score}(s)=|C\\cap. Lemmatization lets barked match the target lemma bark; it does not create additional overlap words.

Step 1. Sound sense

Intersection = {dog}; score = 1.

Step 2. Tree sense

Intersection = {}; score = 0.

Step 3. Ship sense

Intersection = {}; score = 0.

Step 4. Compare

The sound sense has the unique maximum: 1 > 0.

Answer. Bark is the dog's cry.

Question. Apply definition–definition Lesk to “pine cone”. Pine senses: P1 = {evergreen, tree, needle, leaf}; P2 = {waste, sorrow, illness}. Cone senses: C1 = {solid, shape, circular}; C2 = {fruit, evergreen, tree}.

Given

Two candidate senses for each word. Tokens are already normalized; tree and trees would be reduced to the same form if necessary.

To find

The most compatible pair of senses.

Formula and why it applies

Use \"|G_{P_i}\\cap for every pair because this is original-style gloss–gloss comparison.

Step 1. P1 with C1

No shared word → 0.

Step 2. P1 with C2

Shared {evergreen, tree} → 2.

Step 3. P2 with each cone sense

P2–C1 has no overlap → 0; P2–C2 has no overlap → 0.

Step 4. Choose the best pair

The four scores are 0, 2, 0, 0. Maximum 2 belongs to P1–C2.

Answer. Pine is the evergreen tree; cone is its fruit-like reproductive structure.

Limitations: exact overlap misses synonyms, short glosses may produce zeros, and different preprocessing choices change the score. Extended Lesk may include related synset definitions or longer phrase overlaps; do not silently substitute it for a simple question.

Lesk: context aur dictionary mein common words gino; sabse bada overlap jeeta.

"},{"id":"u5-walker","title":"Thesaurus / Walker approach, with three complete examples","html":"

The Walker approach uses thesaurus categories as semantic evidence. Associate each candidate sense with a category. Count or weight context words that support that category. Choose the largest total.

\"\\operatorname{score}(s)=\\sum_{w\\in

\"I=1\" if the supplied category mapping says the context word supports the sense, otherwise \"I=0\". The summation symbol means add all those contributions. Weighted questions may replace 1 with a supplied weight. Category membership must be supplied or justified; it is not guessed from word frequency.

Question. Disambiguate bank using Walker scoring. Context words are money, interest, annum, fetch. The finance category supports money, interest and annum; the location category supports none of these words.

Given

Two categories: Finance and Location. Each supported word contributes 1.

To find

The two totals and selected bank sense.

Formula and why it applies

Sum the category indicators for each sense.

Step 1. Make a contribution table

WordFinanceLocation
money10
interest10
annum10
fetch00

Step 2. Add Finance

\"1+1+1+0=3\".

Step 3. Add Location

\"0+0+0+0=0\".

Step 4. Select

3 > 0, so select the financial institution.

Answer. Finance, with score 3 versus 0.

Question. Use Walker scoring for “The fisherman caught bass beside the lake.” Context {fisherman,caught,lake}. The Fish category contains all three clues; the Music category contains none.

Given

Fish and Music senses, binary category membership given.

To find

The correct bass sense.

Formula and why it applies

Score = sum of context-category matches.

Step 1. Record Fish matches

fisherman → 1; caught → 1; lake → 1.

Step 2. Add Fish score

\"1+1+1=3\".

Step 3. Add Music score

\"0+0+0=0\".

Step 4. Choose

Fish has the larger score.

Answer. Bass refers to a fish.

Question. Score “The player used a bat in the match.” Context {player,match}. Sport supports both words with weights 2 and 1; Animal supports neither. Use the supplied weighted Walker scheme.

Given

Sport weights {player:2, match:1}; Animal weights {player:0, match:0}.

To find

Weighted sense scores.

Formula and why it applies

Use the sum of supplied weights, not an unweighted count, because the question explicitly gives weights.

Step 1. Sport contribution of player

Add 2.

Step 2. Sport contribution of match

Add 1. Sport total = 2 + 1 = 3.

Step 3. Animal total

0 + 0 = 0.

Step 4. Compare

3 > 0 selects the sporting implement.

Answer. Bat is the sports implement.

Walker: dictionary ke exact words nahi, context kis topic/category ko support karta hai woh count.

"},{"id":"u5-graph","title":"Random-walk WSD and supervised alternatives","html":"

Graph-based WSD represents candidate senses as vertices and semantic relationships as weighted edges. A random walk repeatedly transfers score along these edges. Senses supported by other strongly connected context senses tend to obtain higher ranks.

  1. Create a vertex for every candidate sense of every relevant word.
  2. Connect semantically related senses; similarity or gloss overlap can set edge weights.
  3. Normalize outgoing edge weights into transition probabilities.
  4. Run a graph-ranking method such as PageRank.
  5. For each target word, choose its highest-ranked candidate sense.
church context
→
bell: instrument sense ↔ ring: sound sense
→
mutually supported senses receive rank

One common rank update is

\"r_{t+1}(j)=\\frac{1-d}{N}+d\\sum_i

\"r_t(i)\" is current score of vertex \"i\", \"P(i\\to is its outgoing probability to \"j\", \"N\" is the number of vertices, and \"d\" is the probability of following a graph edge. The remaining probability \"1-d\" gives a uniform restart here. A personalized variant uses a nonuniform restart. The course requires the graph method concept; numerical details depend on supplied ranking rules.

Question. A two-vertex toy sense graph has transition matrix rows A:[0.2,0.8], B:[0.4,0.6]. Starting scores are A=0.5, B=0.5. Find one random-walk update with no restart.

Given

Rows denote source vertices, columns destination vertices. d = 1.

To find

New scores at A and B.

Formula and why it applies

Use \"r_{t+1}(j)=\\sum_i: incoming contributions are added.

Step 1. Contribution to A from A

\"0.5\\times0.2=0.1\".

Step 2. Contribution to A from B

\"0.5\\times0.4=0.2\", hence \"r_1(A)=0.1+0.2=0.3\".

Step 3. Contribution to B from A and B

\"0.5\\times0.8=0.4\"; \"0.5\\times0.6=0.3\".

Step 4. Add B and verify total

\"r_1(B)=0.4+0.3=0.7\"; \"0.3+0.7=1\".

Answer. A=0.3, B=0.7 after one update. This is one iteration, not a converged PageRank result.

Other approaches explained without assuming ML knowledge

A supervised classifier learns from examples such as “cash … bank” labelled FINANCE and “river … bank” labelled RIVER. A decision list applies ranked context rules; a decision tree branches on clues; an SVM separates feature representations; a neural model learns contextual patterns. Naive Bayes, developed next, multiplies class-specific context probabilities.

Semi-supervised bootstrapping begins with a few labelled examples or high-confidence rules, labels suitable unlabelled cases, and retrains. Unsupervised clustering groups usages by context similarity without gold sense labels; an additional mapping is needed to name the discovered groups.

Graph WSD: meanings ko jodo; jo meaning context ke strong meanings se supported hai uska rank badhega.

"}]},{"id":"u5-naive-bayes","title":"Naive Bayes WSD: probabilities to a complete numerical","unit":5,"mode":"both","scope":"Unit 5 · PDF pages 49–57","sections":[{"id":"u5-bayes-meaning","title":"Prior, likelihood and posterior—before the formula","html":"

A probability is a number from 0 to 1 measuring how likely an event is under a stated model. A class probability of \"3/6=0.5\" means 3 of 6 training examples belong to that class.

Conditional probability means probability inside a restricted group. \"P(\\text{loan}\\mid reads “probability of the word loan given the finance sense”. The vertical bar means “given”; it is not division.

SymbolNamePlain meaning
\"P(F)\"PriorHow common finance is before reading the new context
\"P(C\\midLikelihoodHow compatible the observed context is with finance
\"P(F\\midPosteriorHow likely finance is after seeing the context
\"P(C)\"EvidenceTotal probability of this context under all senses
\"P(F\\mid

Bayes' theorem reverses a conditional direction using the prior. Do not swap \"P(F\\mid and \"P(C\\mid: one asks about the sense, the other about the evidence.

The naive conditional independence assumption says that, once the sense is fixed, the context-word features can be multiplied as if they were independent. It is an approximation: natural-language words are not actually independent in general.

Training examples → count classes
→
Prior per sense
→
Count words within each sense
→
Likelihood of query clues
→
Multiply → compare → normalize if asked

Prior = pehle kitna common; likelihood = clue kitna fit; posterior = clue dekhne ke baad meaning ki chance.

"},{"id":"u5-bayes-steps","title":"Multinomial Naive Bayes: the exact solving method","html":"

Naive Bayes selects the sense with the largest prior × likelihood product. The context evidence denominator is identical across candidate senses, so it can be omitted for comparison.

\"\\operatorname{score}(s)=P(s)\\prod_j

For add-one multinomial smoothing:

\"P(w\\mid

\"C(w,s)\" counts occurrences of word \"w\" in training contexts of sense \"s\"; \"N_s\" is the total number of word occurrences in those contexts; \"V\" is the shared vocabulary size. It is not the number of training sentences. Repeated query words contribute repeated factors.

Always keep these counts separate

Class priors use number of examples. Word likelihoods in this multinomial model use number of word occurrences. The vocabulary counts unique words across all classes.

  1. List the class examples and calculate priors.
  2. Write one shared vocabulary and count its size.
  3. Count total word occurrences in each class.
  4. Count each query word in each class.
  5. Apply the given smoothing rule.
  6. Multiply each prior by all its query likelihoods.
  7. Compare scores; divide each score by their sum only if a normalized posterior is requested.
\"P(s\\mid

Why smoothing is needed: an unseen clue gives \"0/N_s=0\"; one zero makes the whole product zero. Add-one smoothing redistributes probability across every vocabulary word and prevents these zero factors. Use it only when requested, taught for the task, or state the assumption.

Different model warning: a Bernoulli Naive Bayes question counts presence/absence per document and may include absence factors. Do not use its denominator in a multinomial word-count question.

Pehle sentences se prior; phir words se likelihood; dono counts mix mat kar.

"},{"id":"u5-bayes-mam","title":"Mam's bank numerical—every count and every multiplication","html":"

Question. Classify the sense of “bank” using the training table below. The sentence is: “As the stream flowed gently past the shore, we set up camp by the bank of the river, discussing the possibility of borrowing a loan to buy a boat for future trips.” Use only the highlighted feature words stream, shore and loan, and add-one multinomial smoothing.

Given

Training rowContext wordsSense
1money, deposit, loanFinance
2river, water, shoreRiverbank
3account, savings, investmentFinance
4boat, river, streamRiverbank
5branch, deposit, checkFinance
6fishing, shore, riverRiverbank

Query feature set: {stream, shore, loan}. Other sentence words are not included as features in this supplied example.

To find

The most probable bank sense and normalized posteriors.

Formula and why it applies

Naive Bayes selects the sense with the largest prior × likelihood product. The context evidence denominator is identical across candidate senses, so it can be omitted for comparison.

\"\\operatorname{score}(s)=P(s)\\prod_j

For add-one multinomial smoothing:

\"P(w\\mid

\"C(w,s)\" counts occurrences of word \"w\" in training contexts of sense \"s\"; \"N_s\" is the total number of word occurrences in those contexts; \"V\" is the shared vocabulary size. It is not the number of training sentences. Repeated query words contribute repeated factors.

Step 1. Calculate priors

Finance appears in 3 of the 6 rows; Riverbank appears in 3 of 6.

\"P(F)=\\frac36=\\frac12,\\qquad

Step 2. Construct the vocabulary

{money, deposit, loan, river, water, shore, account, savings, investment, boat, stream, fishing, branch, check}. Thus V=14. Repeated deposit or river is counted once in vocabulary.

Step 3. Count word occurrences per class

There are 3 finance rows × 3 words = 9 word occurrences. There are 3 river rows × 3 words = 9. Thus \"N_F=N_R=9\", and each smoothed denominator is \"9+14=23\".

Step 4. Count the three query words

WordFinance countRiverbank count
stream01
shore02
loan10

Step 5. Calculate finance likelihoods

\"P(\\text{stream}\\mid
\"P(\\text{shore}\\mid
\"P(\\text{loan}\\mid

Step 6. Multiply the finance score

\"\\operatorname{score}(F)=\\frac12\\times\\frac1{23}\\times\\frac1{23}\\times\\frac2{23}=\\frac1{23^3}=\\frac1{12167}\\approx0.00008219.\"

Step 7. Calculate river likelihoods

\"P(\\text{stream}\\mid
\"P(\\text{shore}\\mid
\"P(\\text{loan}\\mid

Step 8. Multiply the river score

\"\\operatorname{score}(R)=\\frac12\\times\\frac2{23}\\times\\frac3{23}\\times\\frac1{23}=\\frac3{12167}\\approx0.00024657.\"

Step 9. Compare scores

3/12167 is larger than 1/12167. Riverbank wins even though “loan” is financial evidence: the combined clues matter.

Step 10. Normalize if a posterior is requested

\"P(F\\mid
\"P(R\\mid

Answer. Bank is the riverbank sense. Scores are 1/12167 and 3/12167; normalized model posteriors are 0.25 and 0.75.

Without smoothing: stream and shore are absent in Finance; loan is absent in Riverbank. Both unsmoothed scores become zero, so there is no valid winning class from those products. Also, unnormalized scores should not be labelled as final posterior probabilities.

Zero word? Add-one: count+1 upar, total words+vocabulary neeche. Last mein score bada wala sense.

"},{"id":"u5-bayes-practice","title":"More WSD numericals: given likelihoods, unequal priors and repeated clues","html":"

Question. For bass, let P(Fish)=0.6 and P(Music)=0.4. Given Fish likelihoods P(caught|Fish)=0.5, P(lake|Fish)=0.4 and Music likelihoods P(caught|Music)=0.05, P(lake|Music)=0.1, classify context {caught,lake} and normalize.

Given

Priors and likelihoods are provided directly; no smoothing or training-count estimation is requested.

To find

The winning sense and its posterior.

Formula and why it applies

\"\\operatorname{score}(s)=P(s)P(\\text{caught}\\mid, because the two features are treated as conditionally independent.

Step 1. Fish multiplication

\"0.6\\times0.5=0.3\", then \"0.3\\times0.4=0.12\".

Step 2. Music multiplication

\"0.4\\times0.05=0.02\", then \"0.02\\times0.1=0.002\".

Step 3. Total evidence

\"0.12+0.002=0.122\".

Step 4. Normalize

\"P(Fish\\mid
\"P(Music\\mid

Answer. Fish, with model posterior approximately 0.983607.

Question. Disambiguate bank from the repeated context tokens {loan, loan, river}. Priors are Finance=2/3 and River=1/3. Vocabulary V=4. Finance has N=8 total tokens, loan count 3 and river count 0; River has N=4, loan count 0 and river count 2. Apply add-one smoothing.

Given

Multinomial counts are supplied. The two appearances of loan must both contribute.

To find

Scores and normalized Finance posterior.

Formula and why it applies

Naive Bayes selects the sense with the largest prior × likelihood product. The context evidence denominator is identical across candidate senses, so it can be omitted for comparison.

\"\\operatorname{score}(s)=P(s)\\prod_j

For add-one multinomial smoothing:

\"P(w\\mid

\"C(w,s)\" counts occurrences of word \"w\" in training contexts of sense \"s\"; \"N_s\" is the total number of word occurrences in those contexts; \"V\" is the shared vocabulary size. It is not the number of training sentences. Repeated query words contribute repeated factors.

Step 1. Finance denominator

\"8+4=12\". \"P(loan\\mid; \"P(river\\mid.

Step 2. River denominator

\"4+4=8\". \"P(loan\\mid; \"P(river\\mid.

Step 3. Finance product

\"\\operatorname{score}(F)=\\frac23\\times\\frac13\\times\\frac13\\times\\frac1{12}=\\frac1{162}\\approx0.00617284.\"

Step 4. River product

\"\\operatorname{score}(R)=\\frac13\\times\\frac18\\times\\frac18\\times\\frac38=\\frac1{512}\\approx0.001953125.\"

Step 5. Compare and normalize

\"P(F\\mid

Answer. Finance wins. Repeated loan contributes a squared likelihood; do not reduce the query to a unique-word set.

Answer-writing order in an exam

Write the class priors, vocabulary size, class token totals, a likelihood table, both score products and the final class. Keep fractions until the comparison where possible. If the question only asks classification, normalization is optional; if it asks posterior probabilities, normalization is required.

Repeated word aaye toh factor bhi repeat. Sirf word-set bana diya toh multinomial answer badal jayega.

"}]}],"formulas":[{"id":"formula-index","title":"All M2 formulas","unit":0,"sections":[{"id":"all","title":"The M2 formula sheet","html":"

Read each formula with its meaning. The remaining lessons in this tab contain the complete worked examples for these methods. Use the same vocabulary, boundary, case and token-count conventions throughout one solution.

N-gram count

\"\\max(T-n+1,0)\"

Unpadded sequence of T tokens. Count only valid contiguous windows.

Unigram MLE

\"P(w)=C(w)/N\"

N is the training-token total under the stated boundary convention.

Bigram MLE

\"P(w_i\\mid

Denominator = history count.

Trigram MLE

\"P(w_i\\mid

Denominator = two-word history count.

Add-one / Laplace

\"P(w\\mid

Add one to every possible vocabulary continuation.

Sentence probability

\"P(W)\\approx\\prod_iP(w_i\\mid

Multiply in order; include an end factor if specified.

Perplexity

\"PP=P(W)^{-1/N}\"

N is the number of predicted tokens/events used by the question.

Emission

\"P(w\\mid

Word given tag; count tokens with that tag.

Transition

\"P(t_j\\mid

Next tag given previous tag; count outgoing transitions.

HMM joint path

\"P(W,T)=\\pi_{t_1}b_{t_1}(w_1)\\prod_{i=2}^{n}a_{t_{i-1},t_i}b_{t_i}(w_i)\"

Multiply initial, emission and transition factors; add end probability if used.

Tag sequences

\"K^T\"

K tags, T word positions; count before constraints.

Viterbi

\"\\delta_i(t)=b_t(w_i)\\max_s[\\delta_{i-1}(s)a_{s,t}]\"

Choose the largest predecessor score; store a backpointer.

Lesk

\"score(s)=|context\\cap

Use the specified normalisation and overlap convention.

Naive Bayes class prior

\"P(c)=D_c/D\"

D counts labelled examples; this is not the word-token total.

Smoothed NB likelihood

\"P(w\\mid

N_c counts training tokens in class c.

NB decision

\"\\hat

Select label with largest score; normalise only for posterior values.

Yaad rakho: Condition jis par hai, denominator usi ka; sequence mein multiply; options mein maximum.

"}],"scope":"M2 · formulas with meaning · full examples follow"},{"id":"f-counts","title":"N-grams & probability","unit":0,"sections":[{"id":"n-grams-construction","title":"Unigrams, bigrams and trigrams: construct them correctly","html":"

Definition. An n-gram is a contiguous sequence of n items. For word n-grams, the items are consecutive word tokens. A unigram contains one word; a bigram contains two; a trigram contains three. Character n-grams use characters instead, so always identify the unit being counted.

Tokens: we | study | natural | language
→
Move a window one token at a time
→
Bigrams: we study / study natural / natural language

For m tokens without padding, the number of n-gram positions is max(0, m−n+1). Repeated positions may contain the same n-gram type. A bag-of-n-grams representation counts each n-gram type and loses the overall sentence order beyond the local order inside each n-gram.

\"\\text{Number

Question. List all word unigrams, bigrams and trigrams for “the prime minister of our country”. Do not add boundary tokens.

Given

Six tokens: the, prime, minister, of, our, country. n = 1, 2 and 3.

To find

All three lists and their counts.

Formula and why it applies

\"\\#\\text{positions}=m-n+1\"

Each window must contain consecutive words and move one position. No wraparound or invented final word is allowed.

Solution

Step 1. Take one word at a time

Unigrams: (the), (prime), (minister), (of), (our), (country). Count = 6−1+1 = 6.

Step 2. Take two consecutive words

Bigrams: (the, prime), (prime, minister), (minister, of), (of, our), (our, country). Count = 6−2+1 = 5.

Step 3. Take three consecutive words

Trigrams: (the, prime, minister), (prime, minister, of), (minister, of, our), (of, our, country). Count = 6−3+1 = 4.

Final answer. 6 unigrams, 5 bigrams and 4 trigrams.

Question. Construct bigrams for “I like tea” using one start and one end marker.

Given

Original tokens: I, like, tea. Add <s> and </s> once each.

To find

The boundary-aware bigram sequence.

Formula and why it applies

\"\\#\\text{bigrams}=(m+2)-1=m+1\"

The start context predicts the first word, and the last word predicts that the sentence ends.

Solution

Step 1. Write the padded sequence

<s> I like tea </s>

Step 2. Move a two-token window

(<s>, I), (I, like), (like, tea), (tea, </s>).

Step 3. Count the windows
\"3+1=4\"
Final answer. There are four boundary-aware bigrams.

Question. Construct trigrams for “I like tea” using two start markers and one end marker.

Given

Original sentence has three words. For a trigram model each predicted token needs two prior context tokens.

To find

All trigram prediction events.

Formula and why it applies

\"P(w_i\\mid

Two start markers provide a complete context even for the first word.

Solution

Step 1. Pad with two starts

<s> <s> I like tea </s>

Step 2. Slide a three-token window

(<s>, <s>, I), (<s>, I, like), (I, like, tea), (like, tea, </s>).

Step 3. Count predicted events

The model predicts I, like, tea and the end marker: four predictions.

Final answer. Four trigrams. Start markers are context; they are not ordinary predicted words.

Question. Give the bag-of-bigrams counts for “a b a b” without padding.

Given

Four tokens in their given order.

To find

Bigram positions, distinct bigram types and their counts.

Formula and why it applies

\"\\#\\text{positions}=4-2+1=3\"

A bag stores frequencies; repeated bigrams must not be discarded.

Solution

Step 1. Enumerate all positions

(a,b), (b,a), (a,b).

Step 2. Group identical pairs
Bigram typeCount
a b2
b a1
Final answer. Three bigram positions but two distinct bigram types.

Yaad rakho: n-gram mein n words; context mein n−1 words.

"},{"id":"unigram-estimation","title":"Unigram estimation and complete worked examples","html":"

Maximum likelihood estimation (MLE) chooses probabilities equal to observed relative frequencies. In a unigram model, every word is predicted independently of its surrounding words. If a word occurs c times in N training tokens, its estimate is c/N. To score m specified words, multiply their m unigram probabilities.

\"P_{\\mathrm{MLE}}(w)=\\frac{C(w)}{N}\\qquad

Choose a tokenization convention before counting. Lowercasing, punctuation handling and boundary markers can change the vocabulary and counts. Never count words with one convention and form the denominator with another. A unigram model gives the same product to rearrangements of the same word multiset; that is one of its limitations.

Question. Train an unpadded, case-insensitive unigram model on “I like tea” and “I like coffee”. Find the probability of “I like tea”.

Given

Training sentences: I like tea; I like coffee. Use lowercase words; exclude punctuation and sentence boundaries.

To find

The vocabulary, word probabilities and test-sentence probability.

Formula and why it applies

\"P(w)=C(w)/N,\\qquad

The unigram approximation ignores order and uses total token count for every denominator.

Solution

Step 1. Count every training token

Each sentence contains three words, so N = 3+3 = 6.

Step 2. Count types and frequencies
WordCountMLE probability
i22/6 = 1/3
like22/6 = 1/3
tea11/6
coffee11/6

Vocabulary size V = 4.

Step 3. Write one factor per test word
\"P=\\frac{2}{6}\\times\\frac{2}{6}\\times\\frac{1}{6}\"
Step 4. Multiply the fractions
\"P=\\frac{4}{216}=\\frac{1}{54}\\approx0.018519\"
Final answer. The unigram sentence probability is approximately 0.018519.

Question. A corpus contains counts a: 4, b: 3 and c: 1. Find the unigram probability of “a b a”.

Given

C(a)=4, C(b)=3, C(c)=1. No boundary markers.

To find

P(a b a).

Formula and why it applies

\"P(a\\,b\\,a)=P(a)P(b)P(a)\"

A repeated test word contributes a probability factor at every occurrence; do not include it just once.

Solution

Step 1. Find the total
\"N=4+3+1=8\"
Step 2. Estimate the required probabilities
\"P(a)=4/8=1/2,\\qquad
Step 3. Use both occurrences of a
\"P=\\frac12\\times\\frac38\\times\\frac12\"
Step 4. Simplify
\"P=\\frac3{32}=0.09375\"
Final answer. P(a b a) = 0.09375.

Yaad rakho: Repeated word ka probability bhi repeat multiply hoga.

Question. An unsmoothed unigram model is trained on “red blue red”. Find the probability of “red green”.

Given

N=3. Counts: red=2, blue=1, green=0. The test contains red and green.

To find

The unsmoothed test probability and the reason for its value.

Formula and why it applies

\"P(red\\

MLE assigns zero to an unseen outcome. Smoothing is needed if an unseen in-vocabulary item must receive probability.

Solution

Step 1. Estimate red
\"P(red)=2/3\"
Step 2. Estimate the unseen word
\"P(green)=0/3=0\"
Step 3. Multiply
\"(2/3)\\times0=0\"
Final answer. Unsmoothed probability = 0. This reflects missing training evidence, not proof that the phrase is impossible.
"},{"id":"bigram-count-table","title":"Bigram counts: build the table before multiplying","html":"

A bigram probability table uses the previous word as its row and the next word as its column. Divide each cell count by its row's total outgoing count. With one end marker on every sentence, each ordinary training word has a continuation, so its outgoing count equals its token frequency. Do not create a bigram that crosses from one sentence into the next.

\"P(w\\mid
Add boundaries to each sentence separately
→
List adjacent pairs
→
Count repeated pairs
→
Divide each row by its total
→
Follow the test sentence through the table

Question. Train a bigram model on “I like tea”, “I like coffee” and “you like tea”. Use one start and one end marker. Find the probability of “I like tea”.

Given

Corpus: <s> I like tea </s>; <s> I like coffee </s>; <s> you like tea </s>. Preserve these tokens.

To find

All nonzero bigram estimates and the test probability.

Formula and why it applies

\"P(w\\mid

Every sentence starts and ends independently. Four probability factors are required for the three test words plus end.

Solution

Step 1. List the training pairs

Sentence 1: (<s>,I), (I,like), (like,tea), (tea,</s>).
Sentence 2: (<s>,I), (I,like), (like,coffee), (coffee,</s>).
Sentence 3: (<s>,you), (you,like), (like,tea), (tea,</s>).

Step 2. Create the nonzero count table
History → nextPair countHistory countProbability
<s> → I232/3
<s> → you131/3
I → like221
you → like111
like → tea232/3
like → coffee131/3
tea → </s>221
coffee → </s>111
Step 3. Write the test path
\"P=P(I\\mid\\langle
Step 4. Substitute each matching row
\"P=\\frac23\\times1\\times\\frac23\\times1\"
Step 5. Multiply
\"P=\\frac49\\approx0.444444\"
Final answer. P(I like tea) = 4/9 ≈ 0.444444.

Question. Using the same three-sentence corpus given here—“I like tea”, “I like coffee”, “you like tea”—find the bigram probabilities of “you like coffee” and “tea like I”. Include boundaries.

Given

Corpus with one start/end marker per sentence. Counts: start→you=1/3; you→like=1; like→coffee=1/3; coffee→end=1. No training sentence starts with tea.

To find

The two test probabilities.

Formula and why it applies

\"P(\\text{sentence})=\\prod

A model can assign a nonzero probability to a sentence it never saw if all of its local bigrams were seen.

Solution

Step 1. Score you like coffee
\"P=\\frac13\\times1\\times\\frac13\\times1=\\frac19\\approx0.111111\"
Step 2. Inspect tea like I
\"P(tea\\mid\\langle
Step 3. Apply the zero factor

The full product is zero, regardless of the remaining factors.

Final answer. P(you like coffee) = 1/9; P(tea like I) = 0. A new sentence is not necessarily a zero-probability sentence.

Question. Obtain the unsmoothed bigram probability of “students are from Vellore” from this corpus: “I am from Vellore”; “I am a teacher”; “students are good and are from various cities”; “students from Vellore do engineering”. Use sentence boundaries.

Given

Four training sentences, each padded with one <s> and </s>. Test sequence: <s> students are from Vellore </s>.

To find

The five required conditional probabilities and their product.

Formula and why it applies

\"P(w\\mid

Count all occurrences in the entire corpus. In particular, are occurs twice inside the third sentence, and Vellore is followed by end once and do once.

Solution

Step 1. Count the histories
HistoryOccurrences with continuation
<s>4
students2
are2
from3
Vellore2
Step 2. Count the required pairs
Required pairCountProbability
<s> students22/4
students are11/2
are from11/2
from Vellore22/3
Vellore </s>11/2
Step 3. Write the complete product
\"P=\\frac24\\times\\frac12\\times\\frac12\\times\\frac23\\times\\frac12\"
Step 4. Multiply progressively
\"\\frac24\\times\\frac12=\\frac14,\\quad
\"\\frac18\\times\\frac23=\\frac1{12},\\quad
Final answer. P = 1/24 ≈ 0.0416667.

Yaad rakho: Same sentence mein word do baar aaye toh count bhi do.

"},{"id":"trigram-estimation","title":"Trigram estimates: two-word history, not two denominators","html":"

A trigram model predicts the next word from the two preceding words. The numerator is the count of a triple. The denominator is the count of its two-word history with an available next token. Do not multiply two independent bigram probabilities and call that a trigram probability.

\"P(w_i\\mid

Question. A corpus contains 12 occurrences of “prime minister”. Eight are followed by “of”, three by “said”, and one by “visited”. Estimate the three trigram probabilities.

Given

History h = (prime, minister). Count of this history = 12. Triple counts for of, said, visited are 8, 3, 1.

To find

The conditional distribution of the next word.

Formula and why it applies

\"P(w\\mid

All three candidates share one two-word history, so they share denominator 12.

Solution

Step 1. Calculate of
\"P(of\\mid
Step 2. Calculate said
\"P(said\\mid
Step 3. Calculate visited
\"P(visited\\mid
Step 4. Confirm normalization
\"8/12+3/12+1/12=12/12=1\"
Final answer. The probabilities are 2/3, 1/4 and 1/12.

Question. Train a trigram model on “I like tea”, “I like coffee” and “you like tea”. Use two start markers and one end marker. Score “I like tea”.

Given

Three sentences. Padded first sentence is <s> <s> I like tea </s>; pad the other two in the same way.

To find

Four trigram prediction probabilities and the sentence probability.

Formula and why it applies

\"P=\\prod_i

Each prediction uses a two-token context; the two start markers supply the first history.

Solution

Step 1. Count the first-word event

The history (<s>,<s>) occurs three times, and two are followed by I.

\"P(I\\mid\\langle
Step 2. Count the second-word event

(<s>,I) occurs twice; both are followed by like.

\"P(like\\mid\\langle
Step 3. Count the third-word event

(I,like) occurs twice; one is followed by tea.

\"P(tea\\mid
Step 4. Count the end event

(like,tea) occurs twice; both are followed by end.

\"P(\\langle/s\\rangle\\mid
Step 5. Multiply all four
\"P=\\frac23\\times1\\times\\frac12\\times1=\\frac13\"
Final answer. Trigram P(I like tea) = 1/3. It differs from the bigram estimate because the context and denominator changed.

Question. A trigram model supplies P(we|<s>,<s>)=0.4, P(study|<s>,we)=0.5, P(NLP|we,study)=0.8 and P(</s>|study,NLP)=0.25. Score “we study NLP”.

Given

Four trigram conditional factors. The end marker is included.

To find

The complete sentence probability.

Formula and why it applies

\"P=P(we\\mid

Use each supplied two-word history exactly; do not swap its word order.

Solution

Step 1. Insert the values
\"P=0.4\\times0.5\\times0.8\\times0.25\"
Step 2. Multiply the first pair
\"0.4\\times0.5=0.2\"
Step 3. Include NLP
\"0.2\\times0.8=0.16\"
Step 4. Include the end
\"0.16\\times0.25=0.04\"
Final answer. P = 0.04.

Yaad rakho: Trigram: teen-word count upar, pehle do-word count neeche.

Why larger n is not automatically better. More context can capture local order more precisely, but each longer history occurs less often. Parameter count, storage and sparsity grow. A model trained on Shakespeare may assign poor probabilities to financial news because training and test distributions differ. Keep a held-out test set; do not evaluate only on memorized training sentences.

"}],"scope":"Formulas explained through full worked examples"},{"id":"f-smoothing","title":"Smoothing & perplexity","unit":0,"sections":[{"id":"laplace-basics","title":"Laplace smoothing: why both numerator and denominator change","html":"

Smoothing reallocates some probability mass to events that received zero or unreliable estimates from finite training data. Laplace, or add-one, smoothing adds one pseudo-count to every possible next outcome. If there are V possible outcomes, the total added count is V, so adding 1 only to the numerator would produce probabilities that do not sum to 1.

\"P_{\\mathrm{Laplace}}(w)=\\frac{C(w)+1}{N+V}\"
\"P_{\\mathrm{Laplace}}(w\\mid

For unigram smoothing, N is the total training token count. For bigram smoothing, C(h) is the count of the particular history. In both cases V is the number of outcomes being predicted, not the number of training sentences and not the number of distinct bigrams. When end is a possible prediction, include it in V; the start marker is usually context only.

Observed counts: 3, 1, 0
→
Add one to every outcome: 4, 2, 1
→
New total: 7
→
Smoothed probabilities: 4/7, 2/7, 1/7

Question. A fixed unigram vocabulary is {a,b,c}. The training counts are 3,1,0. Compute unsmoothed and Laplace-smoothed probabilities.

Given

C(a)=3, C(b)=1, C(c)=0; N=4; V=3.

To find

The complete distributions.

Formula and why it applies

\"P(w)=C(w)/N,\\qquad

The zero-count c is still in the fixed vocabulary, so it must receive one pseudo-count and contribute to V.

Solution

Step 1. Compute unsmoothed values
WordUnsmoothed
a3/4 = 0.75
b1/4 = 0.25
c0/4 = 0
Step 2. Calculate the new denominator
\"N+V=4+3=7\"
Step 3. Add one to each numerator
\"C(a)+1=4,\\quad
Step 4. Divide
WordSmoothed
a4/7 ≈ 0.571429
b2/7 ≈ 0.285714
c1/7 ≈ 0.142857
Step 5. Confirm the total
\"4/7+2/7+1/7=1\"
Final answer. Smoothed probabilities: a=4/7, b=2/7, c=1/7. Common a loses mass; unseen c gains mass.

Question. After history I, a corpus records like three times, study once and sleep zero times. The allowed next vocabulary is {like,study,sleep}. Compute all Laplace-smoothed bigram probabilities.

Given

C(I,like)=3; C(I,study)=1; C(I,sleep)=0. History count C(I)=4. V=3.

To find

P_L(like|I), P_L(study|I), P_L(sleep|I).

Formula and why it applies

\"P_L(w\\mid

This is smoothing within the I row; use its total 4, not the global corpus token count.

Solution

Step 1. Form the common denominator
\"C(I)+V=4+3=7\"
Step 2. Calculate like
\"P_L(like\\mid
Step 3. Calculate study
\"P_L(study\\mid
Step 4. Calculate sleep
\"P_L(sleep\\mid
Final answer. The row is (4/7, 2/7, 1/7). An unseen bigram is no longer forced to zero.

Question. A fixed vocabulary has five possible next tokens. A history h is unseen in training. What is its add-one probability for each next token?

Given

C(h)=0; C(h,w)=0 for each outcome; V=5.

To find

P_L(w|h) for each token.

Formula and why it applies

\"P_L(w\\mid

All outcomes receive equal pseudo-counts because the training data gives no preference for this history.

Solution

Step 1. Add the pseudo-count
\"0+1=1\"
Step 2. Form the denominator
\"0+5=5\"
Step 3. Divide and sum
\"P_L(w\\mid
Final answer. Every outcome gets 0.2 under this fixed-vocabulary model.

Vocabulary warning. Add-one does not automatically define probabilities for every word in the world. Unknown words require a declared vocabulary/UNK convention. If a question supplies V, use that stated V rather than inventing additional tokens.

Yaad rakho: Add-one: upar +1, neeche +V; bigram mein neeche history ka count.

"},{"id":"smoothed-sentence","title":"A complete unseen-bigram question from corpus to final answer","html":"

Question. Consider the corpus “I love to code in Python. I love to learn new things. I code every day.” Obtain the bigram probabilities needed for “I love Python”. Explain the zero-probability issue, then use Laplace smoothing to score it. Treat each sentence separately, preserve case, remove punctuation, and include an end token among predicted outcomes.

Given

Three sentences padded with one start/end marker. Distinct words: I, love, to, code, in, Python, learn, new, things, every, day. With </s>, V=12. Start is context only.

To find

Unsmoothed and smoothed P(I love Python </s> | <s>).

Formula and why it applies

\"P(w\\mid

The test contains an unseen transition love→Python. Smoothing assigns probability to that transition and renormalizes every required history row.

Solution

Step 1. Count the contexts
HistoryCount
<s>3
I3
love2
Python1
Step 2. Count each required pair
PairCount
<s> I3
I love2
love Python0
Python </s>1
Step 3. Compute the unsmoothed product
\"P=\\frac33\\times\\frac23\\times\\frac02\\times\\frac11=0\"

Both words love and Python were seen, but their adjacent pair was not. This is an unseen bigram, not an unknown-word case.

Step 4. Smooth the first event
\"P_L(I\\mid\\langle
Step 5. Smooth the second event
\"P_L(love\\mid
Step 6. Smooth the unseen event
\"P_L(Python\\mid
Step 7. Smooth the end event
\"P_L(\\langle/s\\rangle\\mid
Step 8. Multiply the smoothed factors
\"P_L=\\frac4{15}\\times\\frac3{15}\\times\\frac1{14}\\times\\frac2{13}\"
\"P_L=\\frac{24}{40950}=\\frac4{6825}\\approx0.000586081\"
Final answer. Unsmoothed P=0. With the stated V=12 and boundaries, Laplace-smoothed P≈0.000586081.

If an exam states a different convention—for example excluding boundary tokens from the entire calculation—recompute both V and the set of probability factors consistently. Do not combine a boundary-aware numerator with a boundary-free denominator.

"},{"id":"adjusted-count-discount","title":"Adjusted counts and relative discounting","html":"

The slides also express smoothing through an adjusted count c*. Instead of writing the smoothed probability with denominator N+V, they rescale its numerator so the denominator can remain N. Algebraically these are equivalent:

\"c^*=(c+1)\\frac{N}{N+V},\\qquad
\"d=\\frac{c^*}{c}\\quad(c>0)\"

The relative discount d compares adjusted and original counts. It is undefined when c=0 because division by zero is undefined. Add-one smoothing moves the distribution toward uniformity: frequent events are reduced, but some rare seen events may increase. It is not correct to claim that every nonzero count must decrease under Laplace smoothing.

Question. For N=20 and V=5, a word has original count c=10. Find its adjusted count, smoothed probability and relative discount.

Given

N=20; V=5; c=10.

To find

c*, P_L and d.

Formula and why it applies

\"c^*=(c+1)N/(N+V),\\quad

These expressions rewrite add-one smoothing without changing its probability.

Solution

Step 1. Form the new total
\"N+V=20+5=25\"
Step 2. Find the adjusted count
\"c^*=(10+1)\\frac{20}{25}=11(0.8)=8.8\"
Step 3. Find the probability
\"P_L=8.8/20=0.44\"
Step 4. Find the relative discount
\"d=8.8/10=0.88\"
Final answer. c*=8.8, P_L=0.44, d=0.88. The original probability 0.5 is reduced.

Question. Under N=20, V=5, a rare word occurs once. Compute its adjusted count and compare the original and smoothed probabilities.

Given

N=20; V=5; c=1.

To find

c*, original P and smoothed P_L.

Formula and why it applies

\"c^*=(c+1)N/(N+V)\"

The same smoothing rule may increase a rare event's probability.

Solution

Step 1. Original probability
\"P=1/20=0.05\"
Step 2. Adjusted count
\"c^*=(1+1)(20/25)=1.6\"
Step 3. Smoothed probability
\"P_L=1.6/20=0.08\"
Final answer. c*=1.6 and P_L=0.08, which exceeds the original 0.05.

Question. For N=20 and V=5, an in-vocabulary word has count zero. Find its adjusted count and smoothed probability. Can a relative discount be calculated?

Given

c=0; N=20; V=5.

To find

c*, P_L and whether d exists.

Formula and why it applies

\"c^*=(c+1)N/(N+V),\\qquad

The pseudo-count creates positive mass for the unseen vocabulary item.

Solution

Step 1. Adjusted count
\"c^*=(0+1)(20/25)=0.8\"
Step 2. Smoothed probability
\"P_L=0.8/20=0.04\"
Step 3. Relative discount

d would be 0.8/0. That is undefined, so do not report a finite discount.

Final answer. c*=0.8 and P_L=0.04; relative discount is undefined for c=0.
"},{"id":"evaluation-perplexity","title":"Perplexity: meaning, formula and four complete examples","html":"

Intrinsic evaluation evaluates a model directly on a held-out test corpus, for example using perplexity. Extrinsic evaluation places it inside a downstream application, such as a speech recognizer or translation system, and measures the application's performance. Extrinsic testing is closer to actual use but often slower and more expensive. A lower perplexity is not a guarantee of improvement on every downstream task.

Perplexity is the inverse probability of a test sequence, normalized by the number of prediction events. Intuitively, it is the model's effective average number of equally plausible choices. A model that always gives the correct next word probability 1/4 has perplexity 4. Lower is better when the models are tested on the same data, vocabulary, tokenization and boundary convention.

\"\\operatorname{PP}(W)=P(W)^{-1/N}=\\sqrt[N]{\\frac1{P(W)}}\"
SymbolMeaning
WThe complete test sequence.
P(W)Product of the probabilities assigned to the actual next tokens.
NNumber of scored prediction events under the stated convention.
Exponent −1/NTake the reciprocal, then the Nth root.

For calculator entry, use parentheses: P raised to (−1 ÷ N), or (1 ÷ P) raised to (1 ÷ N). Retain several digits in intermediate products. The ordinary start marker is usually given context, not a predicted event; the end marker counts if its probability is included.

Question. A model assigns probability 0.0625 to a test sequence containing four scored tokens. Compute its perplexity.

Given

P(W)=0.0625=1/16; N=4.

To find

PP(W).

Formula and why it applies

\"\\operatorname{PP}(W)=\\left(\\frac1{P(W)}\\right)^{1/N}\"

The root removes the effect of sequence length; the reciprocal makes lower surprise correspond to lower perplexity.

Solution

Step 1. Take the reciprocal
\"1/P(W)=1/0.0625=16\"
Step 2. Use the fourth root
\"\\operatorname{PP}=16^{1/4}\"
Step 3. Evaluate
\"2^4=16\\quad\\Rightarrow\\quad\\operatorname{PP}=2\"
Final answer. Perplexity = 2.

Question. Two language models score the same three-token test sequence. Model A gives next-token probabilities 0.5, 0.5, 0.5. Model B gives 0.5, 0.25, 0.25. Compute their perplexities and select the better intrinsic result.

Given

Same sequence and N=3 for both models. A factors: 0.5,0.5,0.5. B factors: 0.5,0.25,0.25.

To find

Both sequence probabilities and perplexities.

Formula and why it applies

\"P(W)=\\prod_i

A fair comparison uses identical test tokens and scoring conventions.

Solution

Step 1. Calculate A's probability
\"P_A=0.5\\times0.5\\times0.5=0.125=1/8\"
Step 2. Calculate A's perplexity
\"\\operatorname{PP}_A=8^{1/3}=2\"
Step 3. Calculate B's probability
\"P_B=0.5\\times0.25\\times0.25=0.03125=1/32\"
Step 4. Calculate B's perplexity
\"\\operatorname{PP}_B=32^{1/3}\\approx3.174802\"
Step 5. Compare

2 is lower than 3.174802; A places more probability on the observed sequence.

Final answer. Model A is better by perplexity on this test: 2 versus about 3.174802.

Question. The bigram training corpus is “This is a dog”, “This is a cat”, “I love my cat”, “This is my name”. Score “This is my cat”, including an end marker, and compute perplexity using five predicted events.

Given

Each training sentence has one start and end marker. Test predictions: This, is, my, cat, </s>. Thus N=5.

To find

Bigram sentence probability and perplexity.

Formula and why it applies

\"P(w\\mid

Start is given context. Four words plus the end marker produce five probability factors.

Solution

Step 1. Count the required transitions
TransitionPair count / history count
<s>→This3/4
This→is3/3
is→my1/3
my→cat1/2
cat→</s>2/2
Step 2. Form the product
\"P=\\frac34\\times1\\times\\frac13\\times\\frac12\\times1=\\frac18\"
Step 3. Take the reciprocal
\"1/P=8\"
Step 4. Take the fifth root
\"\\operatorname{PP}=8^{1/5}\\approx1.515717\"
Final answer. P(W)=1/8 and perplexity≈1.515717 using five predicted events.

If the question explicitly states N = 6: use that supplied convention and report PP = 81/6 ≈ 1.414214. The class slide counts both boundary markers in N. Write the chosen N beside the formula; do not silently switch between 5 and 6. Also, This occurs three times and is follows it three times, so P(is|This)=3/3, not 3/4.

Question. A uniform model assigns probability 1/10 to each actual next word in a five-token test. Find its perplexity.

Given

Five scored tokens, each with assigned probability 0.1. N=5.

To find

The sentence probability and perplexity.

Formula and why it applies

\"P(W)=(1/10)^5,\\qquad

Uniform equal probabilities make the effective number of choices especially clear.

Solution

Step 1. Multiply the five factors
\"P(W)=0.1^5=0.00001=10^{-5}\"
Step 2. Invert
\"1/P(W)=10^5\"
Step 3. Take the fifth root
\"\\operatorname{PP}=(10^5)^{1/5}=10\"
Final answer. Perplexity = 10.

Yaad rakho: Probability high → surprise low → perplexity low.

"},{"id":"log-probabilities","title":"Log probabilities: replace long products with sums","html":"

Probabilities are at most 1, so multiplying many of them produces very small numbers. Computers can round extremely small positive products to zero: numerical underflow. Log probabilities convert multiplication into addition and are more stable for long sequences. Natural logs use base e; log₂ uses base 2. Choose one base consistently.

\"\\ln
\"P(W)=e^{\\ln

The log of a probability below 1 is negative. A value closer to zero means a larger probability: ln(0.5) ≈ −0.693 is larger than ln(0.1) ≈ −2.303. A zero probability has log −∞; taking logs does not solve the unseen-event problem. Smoothing handles that separate problem.

Question. The next-word probabilities are 0.5, 0.25 and 0.5. Calculate the natural log probability, ordinary probability and perplexity.

Given

p₁=0.5; p₂=0.25; p₃=0.5; N=3.

To find

ln P(W), P(W), and PP.

Formula and why it applies

\"\\ln

Logarithms preserve products as sums and the exponent converts the result back.

Solution

Step 1. Compute individual logs
\"\\ln0.5\\approx-0.693147,\\quad\\ln0.25\\approx-1.386294\"
Step 2. Add all three terms
\"\\ln
Step 3. Recover the probability
\"P=e^{-2.772589}\\approx0.0625\"
Step 4. Find average negative log probability
\"-\\ln
Step 5. Exponentiate
\"\\operatorname{PP}=e^{0.924196}\\approx2.519842\"
Final answer. ln P≈−2.772589, P=0.0625, PP≈2.519842.

Question. A model has a total natural log probability −12 over six scored tokens. Calculate its perplexity.

Given

ln P(W)=−12; N=6.

To find

PP without forming the tiny ordinary probability.

Formula and why it applies

\"\\operatorname{PP}=\\exp(-\\ln

The logarithmic formula directly uses the supplied log score.

Solution

Step 1. Negate the log score
\"-\\ln
Step 2. Divide by token count
\"12/6=2\"
Step 3. Apply exponential
\"\\operatorname{PP}=e^2\\approx7.389056\"
Final answer. Perplexity≈7.389056.

Question. A language model's average negative log₂ probability is 3 bits per token. Find its perplexity.

Given

Cross-entropy H=3 bits/token. Log base is 2.

To find

PP.

Formula and why it applies

\"\\operatorname{PP}=2^H\"

The inverse operation of log₂ is raising 2 to that power; use e only for natural logs.

Solution

Step 1. Identify the log base

Bits mean base 2.

Step 2. Raise the base to H
\"\\operatorname{PP}=2^3=8\"
Final answer. Perplexity = 8.

Yaad rakho: ln use kiya toh e se wapas; log₂ use kiya toh 2 se wapas.

"}],"scope":"Formulas explained through full worked examples"},{"id":"f-hmm","title":"HMM & decoding","unit":0,"sections":[{"id":"markov-chain","title":"Markov chains: read a transition table and multiply a path","html":"

A Markov chain is a probabilistic model of a sequence of states. Under the first-order Markov assumption, the next state's probability depends on the current state rather than the complete previous history. The initial probability tells us how likely each first state is. A transition probability tells us how likely a move from one state to another is.

\"P(s_t\\mid
\"P(s_1,\\ldots,s_T)=P(s_1)\\prod_{t=2}^{T}P(s_t\\mid

For T states in a path, use one initial factor and T−1 transition factors. A row-oriented transition matrix must have each complete row sum to 1. Always read the row/column labels; some exam diagrams use a transposed arrangement.

Current → nextRainDry
Rain0.30.7
Dry0.20.8

Initial probabilities: P(Rain)=0.4, P(Dry)=0.6. Rows describe the current state; columns describe the next state.

Question. For the Rain/Dry Markov chain with initial P(Rain)=0.4 and P(Dry)=0.6, and transitions Rain→Rain=0.3, Rain→Dry=0.7, Dry→Rain=0.2, Dry→Dry=0.8, find the probability of Dry, Dry, Rain, Rain.

Given

Current → nextRainDry
Rain0.30.7
Dry0.20.8

Initial probabilities: P(Rain)=0.4, P(Dry)=0.6. Rows describe the current state; columns describe the next state.

To find

P(Dry,Dry,Rain,Rain).

Formula and why it applies

\"P(D,D,R,R)=P(D)P(D\\mid

Each step conditions on the immediately preceding state. The sequence contains four states, so it needs one initial plus three transition factors.

Solution

Step 1. Start in Dry
\"P(D)=0.6\"
Step 2. Move Dry to Dry
\"0.6\\times0.8=0.48\"
Step 3. Move Dry to Rain
\"0.48\\times0.2=0.096\"
Step 4. Move Rain to Rain
\"0.096\\times0.3=0.0288\"
Final answer. Path probability = 0.0288.

Question. For initial probabilities Rain=0.4, Dry=0.6 and transition rows Rain=(0.3,0.7), Dry=(0.2,0.8) in column order Rain,Dry, find the probability of Rain, Dry, Rain.

Given

Current → nextRainDry
Rain0.30.7
Dry0.20.8

Initial probabilities: P(Rain)=0.4, P(Dry)=0.6. Rows describe the current state; columns describe the next state.

To find

P(Rain,Dry,Rain).

Formula and why it applies

\"P(R,D,R)=P(R)P(D\\mid

Do not replace Dry→Rain with Rain→Dry: the two directional probabilities differ.

Solution

Step 1. Initial Rain
\"P(R)=0.4\"
Step 2. First transition
\"P(D\\mid
Step 3. Second transition
\"P(R\\mid
Final answer. Path probability = 0.056.

Question. For the same fully specified chain—initial Rain=0.4, Dry=0.6; Rain→Rain=0.3, Rain→Dry=0.7, Dry→Rain=0.2, Dry→Dry=0.8—find P(Dry,Dry). Also find the conditional probability of next Dry given current Dry.

Given

Current → nextRainDry
Rain0.30.7
Dry0.20.8

Initial probabilities: P(Rain)=0.4, P(Dry)=0.6. Rows describe the current state; columns describe the next state.

To find

The two-state joint probability and the one-step conditional probability.

Formula and why it applies

\"P(D,D)=P(D)P(D\\mid

The joint path includes an uncertain initial state. The conditional transition assumes the current state is already Dry.

Solution

Step 1. Read the conditional transition
\"P(D\\mid
Step 2. Include the initial state for the joint path
\"P(D,D)=0.6\\times0.8=0.48\"
Final answer. P(Dry,Dry)=0.48; P(next Dry|current Dry)=0.8. These answer different questions.

Yaad rakho: Path: pehle start probability, phir har arrow ki probability.

"},{"id":"transition-emission-estimation","title":"Estimate transition and emission probabilities from tagged data","html":"
\"P(t'\\mid
\"P(w\\mid

In a transition denominator, count the previous tag's outgoing events. In an emission denominator, count all tokens bearing that tag. With explicit end transitions the totals align conveniently. Keep the two numerator types separate: tag→tag counts are not word→tag counts.

Question. A tagged corpus contains 116454 determiner tags DT, each counted with an outgoing event. DT is followed by NN 56509 times. Estimate P(NN|DT).

Given

C(DT)=116454; C(DT,NN)=56509.

To find

The DT→NN transition probability.

Formula and why it applies

\"P(NN\\mid

The denominator is the number of previous DT contexts because DT is the condition.

Solution

Step 1. Identify the transition count

DT followed by NN occurs 56509 times.

Step 2. Identify the conditional total

There are 116454 DT contexts.

Step 3. Divide
\"P(NN\\mid
Final answer. P(NN|DT)≈0.485247.

Question. The tag VBZ occurs 21627 times, and 10073 of those tokens are the word “is”. Estimate P(is|VBZ).

Given

C(VBZ)=21627; C(is,VBZ)=10073.

To find

The emission probability of is from VBZ.

Formula and why it applies

\"P(is\\mid

The condition is the tag VBZ; we ask which word it emits.

Solution

Step 1. Choose the word-tag count

The required word is is with tag VBZ: count 10073.

Step 2. Choose the tag total

All VBZ-tagged word tokens: count 21627.

Step 3. Divide
\"P(is\\mid
Final answer. P(is|VBZ)≈0.465760. This is different from P(VBZ|is).

Question. Use the tagged sentences “dogs/N chase/V cats/N” and “cats/N sleep/V”. Estimate the initial tag probabilities, all nonzero transitions including END, and emissions.

Given

Corpus: START dogs/N chase/V cats/N END; START cats/N sleep/V END.

To find

Initial distribution, transition rows and emission rows.

Formula and why it applies

\"\\pi_t=C(START,t)/2,\\quad

The corpus explicitly tells us the tags, so we can count training frequencies directly.

Solution

Step 1. Count tag tokens

N occurs 3 times: dogs, cats, cats. V occurs 2 times: chase, sleep.

Step 2. Count sentence starts

Both sentences start with N.

\"\\pi_N=2/2=1,\\quad
Step 3. Count transitions
PairCount
N→V2
N→END1
V→N1
V→END1
Step 4. Normalize transition rows
CurrentTo NTo VTo END
N02/31/3
V1/201/2
Step 5. Count and normalize emissions
Tagdogscatschasesleep
N1/32/300
V001/21/2
Final answer. π=(1,0); transitions and emissions are given in the completed tables. Each complete probability row sums to 1.

Yaad rakho: Tag→tag table aur tag→word table ko mix mat karna.

"},{"id":"hot-cold-paths","title":"HMM path probabilities: three complete numerical examples","html":"

The following model uses hidden weather states Hot (H) and Cold (C). Observations are numbers 1, 2 and 3. These numbers are observation labels, not quantities to multiply directly. Their emission probabilities come from the table.

Initial stateProbability
H0.8
C0.2
From → toHC
H0.70.3
C0.40.6
Emission stateObservation 1Observation 2Observation 3
H0.20.40.4
C0.50.40.1

Question. An HMM has π(H)=0.8, π(C)=0.2; H→H=0.7, H→C=0.3, C→H=0.4, C→C=0.6. Emissions for observations (1,2,3) are H:(0.2,0.4,0.4) and C:(0.5,0.4,0.1). Find the joint probability of hidden path H,C,H and observations 1,3,1.

Given

Hidden path H→C→H. Observations 1→3→1. Model probabilities are specified in the question. There is no END factor.

To find

P(H,C,H; 1,3,1).

Formula and why it applies

\"P=\\pi_H

There are three hidden states and observations: three emissions, two transitions, and one initial probability.

Solution

Step 1. Start Hot and emit 1
\"0.8\\times0.2=0.16\"
Step 2. Transition Hot to Cold
\"0.16\\times0.3=0.048\"
Step 3. Cold emits 3
\"0.048\\times0.1=0.0048\"
Step 4. Transition Cold to Hot
\"0.0048\\times0.4=0.00192\"
Step 5. Hot emits the last 1
\"0.00192\\times0.2=0.000384\"
Final answer. Joint probability = 0.000384 = 3.84×10⁻⁴.

Question. For π(H)=0.8, π(C)=0.2; H→H=0.7, H→C=0.3, C→H=0.4, C→C=0.6; emissions H:(0.2,0.4,0.4), C:(0.5,0.4,0.1) for labels (1,2,3), find the joint probability of path H,H,H and observations 1,3,1.

Given

State path H→H→H, observations 1→3→1; no END factor.

To find

P(H,H,H;1,3,1).

Formula and why it applies

\"P=\\pi_H

The state remains Hot; each transition still contributes its probability 0.7.

Solution

Step 1. Initial state and first emission
\"0.8(0.2)=0.16\"
Step 2. First self-transition
\"0.16(0.7)=0.112\"
Step 3. Emit observation 3
\"0.112(0.4)=0.0448\"
Step 4. Second self-transition
\"0.0448(0.7)=0.03136\"
Step 5. Emit final observation 1
\"0.03136(0.2)=0.006272\"
Final answer. Joint probability = 0.006272. A self-transition is not automatically probability 1.

Question. For π(H)=0.8, π(C)=0.2; transitions H:(0.7,0.3), C:(0.4,0.6) in next-state order H,C; emissions H:(0.2,0.4,0.4), C:(0.5,0.4,0.1) for observations 1,2,3, find the score of path C,C,H with observations 1,3,1.

Given

Hidden path C→C→H. Observation sequence 1→3→1. No end-state factor.

To find

The joint path-and-observation probability.

Formula and why it applies

\"P=\\pi_C

The initial state is Cold, so use π(C), not π(H). Each emission is selected from the state at the same position.

Solution

Step 1. Start Cold and emit 1
\"0.2(0.5)=0.1\"
Step 2. Stay Cold
\"0.1(0.6)=0.06\"
Step 3. Emit 3 from Cold
\"0.06(0.1)=0.006\"
Step 4. Move Cold to Hot
\"0.006(0.4)=0.0024\"
Step 5. Emit 1 from Hot
\"0.0024(0.2)=0.00048\"
Final answer. Joint probability = 0.00048.

Yaad rakho: Har column par: state ka emission. Columns ke beech: transition.

"},{"id":"tagging-corpus-case","title":"POS HMM worked corpus: Mary, Jane, can, will and spot","html":"

The class example contains words that can be either names or ordinary verbs/modal auxiliaries. The term is modal (M), not “model”. To avoid guessing tags from spelling, the training corpus below supplies them explicitly and lowercases words for the emission vocabulary.

Training sentenceGold tags
mary jane can see willN N M V N
spot will see maryN M V N
will jane spot maryM N V N
mary will pat spotN M V N

There are 9 noun tokens, 4 modal tokens and 4 verb tokens. Each sentence includes a start and end transition. For nouns, the next event can be another noun, modal, verb or END.

Current → nextNMVEND
START3/41/400
N1/93/91/94/9
M1/403/40
V4/4000
Tag → wordmaryjanecanseewillspotpat
N4/92/9001/92/90
M001/403/400
V0002/401/41/4

Question. Using the fully tagged four-sentence corpus above, calculate the joint score of “will jane spot mary” assigned tags M,N,V,N, including END.

Given

Required factors from the tables: START→M=1/4; emit will|M=3/4; M→N=1/4; jane|N=2/9; N→V=1/9; spot|V=1/4; V→N=1; mary|N=4/9; N→END=4/9.

To find

The specified tag-path score, including words and sentence end.

Formula and why it applies

\"P=\\pi_M

Both transitions and emissions are needed. A tag transition alone cannot tell us whether a particular word is plausible.

Solution

Step 1. Start and emit will
\"\\frac14\\times\\frac34=\\frac3{16}\"
Step 2. Move to N and emit jane
\"\\frac3{16}\\times\\frac14\\times\\frac29=\\frac1{96}\"
Step 3. Move to V and emit spot
\"\\frac1{96}\\times\\frac19\\times\\frac14=\\frac1{3456}\"
Step 4. Move to N and emit mary
\"\\frac1{3456}\\times1\\times\\frac49=\\frac1{7776}\"
Step 5. End the sentence
\"\\frac1{7776}\\times\\frac49=\\frac1{17496}\\approx0.0000571559\"
Final answer. The joint score is 1/17496 ≈ 0.0000571559.

Question. For the same tagged corpus and tables, explain the unsmoothed score of “will can spot mary” assigned tags M,V,N,N.

Given

The proposed second word is can with hidden tag V. In the training corpus, can occurs only with M, so b_V(can)=0.

To find

Whether the complete path score is zero and what this means.

Formula and why it applies

\"P(\\text{path,words})=(\\text{other

A product containing one zero factor is zero even if every other factor is positive.

Solution

Step 1. Find the critical emission
\"b_V(can)=C(can,V)/C(V)=0/4=0\"
Step 2. Use the full-product consequence
\"P(\\text{path,words})=0\"
Step 3. Interpret correctly

This unsmoothed corpus model has no evidence for can as a verb. It does not prove that a verb usage of can is impossible in English.

Final answer. The unsmoothed path score is zero.
"},{"id":"sequence-count","title":"Why exhaustive search grows exponentially","html":"

If each of T observed words can receive any of N tags, there are N choices at each position. Thus the total number of complete tag sequences is N multiplied by itself T times. This counts possible assignments before grammar rules or zero probabilities eliminate any.

\"\\text{Number

Question. A four-word sentence is tagged using three possible POS tags per word. How many complete tag sequences are possible?

Given

N=3 possible tags per position; T=4 word positions.

To find

The number of tag sequences.

Formula and why it applies

\"M=N^T\"

Every independent position contributes a factor of three choices.

Solution

Step 1. Write one factor per position
\"M=3\\times3\\times3\\times3\"
Step 2. Multiply
\"3\\times3=9,\\quad9\\times3=27,\\quad27\\times3=81\"
Final answer. 81 possible tag sequences.

Question. A five-word sentence has two candidate tags per word. Count its tag sequences.

Given

N=2; T=5.

To find

M.

Formula and why it applies

\"M=N^T\"

The length is the exponent, not a multiplier.

Solution

Step 1. Substitute
\"M=2^5\"
Step 2. Evaluate
\"2\\times2\\times2\\times2\\times2=32\"
Final answer. 32 possible sequences.

Question. Using 45 possible tags, compare the number of tag sequences for sentences of length 2 and 3.

Given

N=45. First T=2, then T=3.

To find

Both counts and the growth factor.

Formula and why it applies

\"M_2=45^2,\\quad

Adding one position multiplies the possibilities by the number of tag choices.

Solution

Step 1. Two words
\"45^2=2025\"
Step 2. Three words
\"45^3=2025\\times45=91125\"
Step 3. Compare
\"91125/2025=45\"
Final answer. 2 words: 2025 paths. 3 words: 91125 paths. One additional word multiplies the count by 45.

The slides introduce Viterbi to avoid enumerating every path. It reuses the best prefix score for each ending state. For a dense first-order HMM, it needs O(TN²) recurrence work instead of enumerating NT complete paths.

Yaad rakho: Choices per word ki power = sentence length.

"},{"id":"viterbi-explained","title":"Viterbi decoding: compare incoming paths, keep the best, trace back","html":"

The Viterbi algorithm finds the most likely hidden-state sequence for a fixed observation sequence. At each position, it keeps the best path ending in each state. This is safe because, under the HMM assumptions, future extensions depend on that ending state, not the discarded details of the prefix.

\"\\delta_1(j)=\\pi_jb_j(o_1)\"
\"\\delta_t(j)=b_j(o_t)\\max_i[\\delta_{t-1}(i)a_{ij}]\"
\"\\psi_t(j)=\\arg\\max_i[\\delta_{t-1}(i)a_{ij}]\"
SymbolMeaning in ordinary words
δₜ(j)Score of the best prefix that ends in state j at position t.
ψₜ(j)Previous state that supplied that best prefix; keep it as a backpointer.
maxChoose the larger candidate score; do not add.
argmaxRemember which previous state won, rather than only its value.
Initialize: start × first emission
→
For each next state: compare incoming scores
→
Multiply winner by this state's emission
→
Keep score + winning predecessor
→
Choose best final state and trace arrows backward

Do not choose the best state independently at each word. The best last state can come from a different best predecessor than a local guess. Backpointers recover one consistent complete path. If scores tie, either tied path is optimal under the model; state the tie.

Question. Use an HMM with π(H)=0.8, π(C)=0.2, transitions H:(0.7,0.3), C:(0.4,0.6) in H,C order, and emissions H:(0.2,0.4,0.4), C:(0.5,0.4,0.1) for observations 1,2,3. Find the most likely hidden path for observations 1,3,1 using Viterbi.

Given

Initial probabilities: π(H)=0.8, π(C)=0.2. Transition matrix: H→H=0.7, H→C=0.3, C→H=0.4, C→C=0.6. Emissions for labels (1,2,3): H=(0.2,0.4,0.4), C=(0.5,0.4,0.1). There is no END-state factor.

To find

The best hidden-state sequence and its joint score.

Formula and why it applies

\"\\delta_1(j)=\\pi_jb_j(o_1),\\quad\\delta_t(j)=b_j(o_t)\\max_i[\\delta_{t-1}(i)a_{ij}]\"

Viterbi keeps two best prefix scores per column, one ending H and one ending C.

Solution

Step 1. Initialize at observation 1
\"\\delta_1(H)=0.8(0.2)=0.16\"
\"\\delta_1(C)=0.2(0.5)=0.10\"
Step 2. At observation 3, calculate the H cell
\"H\\to
\"C\\to
\"\\delta_2(H)=\\max(0.112,0.040)(0.4)=0.0448\"

Backpointer for this H cell: H.

Step 3. At observation 3, calculate the C cell
\"H\\to
\"C\\to
\"\\delta_2(C)=\\max(0.048,0.060)(0.1)=0.006\"

Backpointer for this C cell: C.

Step 4. At final observation 1, calculate H
\"H\\to
\"C\\to
\"\\delta_3(H)=0.03136(0.2)=0.006272\"

Backpointer: H.

Step 5. At final observation 1, calculate C
\"H\\to
\"C\\to
\"\\delta_3(C)=0.01344(0.5)=0.00672\"

Backpointer: H.

Step 6. Select the final state

0.00672 > 0.006272, so the best final state is C.

Step 7. Trace back

Final C came from H at position2. That H came from H at position1. Reading forward gives H,H,C.

Step 8. Verify the recovered complete path
\"0.8(0.2)(0.7)(0.4)(0.3)(0.5)=0.00672\"
Final answer. Best path H→H→C with joint score 0.00672.

Question. For π(H)=0.8, π(C)=0.2; transitions H:(0.7,0.3), C:(0.4,0.6); emissions H:(0.2,0.4,0.4), C:(0.5,0.4,0.1) for labels1,2,3, decode observations 3,1.

Given

Initial probabilities: π(H)=0.8, π(C)=0.2. Transition matrix: H→H=0.7, H→C=0.3, C→H=0.4, C→C=0.6. Emissions for labels (1,2,3): H=(0.2,0.4,0.4), C=(0.5,0.4,0.1). There is no END-state factor.

To find

Best two-state hidden path and score.

Formula and why it applies

\"\\delta_1(j)=\\pi_jb_j(3),\\quad\\delta_2(j)=b_j(1)\\max_i[\\delta_1(i)a_{ij}]\"

The same recurrence applies even though the observation order changed.

Solution

Step 1. Initialize observation 3
\"\\delta_1(H)=0.8(0.4)=0.32,\\quad\\delta_1(C)=0.2(0.1)=0.02\"
Step 2. Calculate the final H
\"\\max(0.32(0.7),0.02(0.4))(0.2)\"
\"=\\max(0.224,0.008)(0.2)=0.0448\"

Winning predecessor H.

Step 3. Calculate the final C
\"\\max(0.32(0.3),0.02(0.6))(0.5)\"
\"=\\max(0.096,0.012)(0.5)=0.048\"

Winning predecessor H.

Step 4. Select and trace

Final C wins with 0.048 and came from H. The path is H,C.

Final answer. Best path H→C, score 0.048.

Question. For π(H)=0.8, π(C)=0.2; transitions H:(0.7,0.3), C:(0.4,0.6); emissions H:(0.2,0.4,0.4), C:(0.5,0.4,0.1) for labels1,2,3, decode observations 2,2.

Given

Initial probabilities: π(H)=0.8, π(C)=0.2. Transition matrix: H→H=0.7, H→C=0.3, C→H=0.4, C→C=0.6. Emissions for labels (1,2,3): H=(0.2,0.4,0.4), C=(0.5,0.4,0.1). There is no END-state factor.

To find

Best path and score.

Formula and why it applies

\"\\delta_1(j)=\\pi_jb_j(2),\\quad\\delta_2(j)=b_j(2)\\max_i[\\delta_1(i)a_{ij}]\"

Both states emit2 with probability 0.4, but their initial and transition probabilities still differ.

Solution

Step 1. Initialize
\"\\delta_1(H)=0.8(0.4)=0.32,\\quad\\delta_1(C)=0.2(0.4)=0.08\"
Step 2. Final H
\"\\max(0.32(0.7),0.08(0.4))(0.4)\"
\"=\\max(0.224,0.032)(0.4)=0.0896\"

Predecessor H.

Step 3. Final C
\"\\max(0.32(0.3),0.08(0.6))(0.4)\"
\"=\\max(0.096,0.048)(0.4)=0.0384\"

Predecessor H.

Step 4. Select and trace

The final H score 0.0896 is larger. Its predecessor is H.

Final answer. Best path H→H with score 0.0896.

Yaad rakho: Har destination ke liye incoming paths compare; winner ka arrow yaad.

"},{"id":"joint-vs-posterior","title":"A path score is not the probability of the observed sentence","html":"

Question. An HMM has π(H)=0.8, π(C)=0.2; transitions H:(0.7,0.3), C:(0.4,0.6); emissions H:(0.2,0.4,0.4), C:(0.5,0.4,0.1) for labels1,2,3. For observations 1,3, calculate the probability of the observations by summing all four two-state paths.

Given

Initial probabilities: π(H)=0.8, π(C)=0.2. Transition matrix: H→H=0.7, H→C=0.3, C→H=0.4, C→C=0.6. Emissions for labels (1,2,3): H=(0.2,0.4,0.4), C=(0.5,0.4,0.1). There is no END-state factor.

To find

P(observations 1,3), not just the best-path score.

Formula and why it applies

\"P(o_1,o_2)=\\sum_{s_1,s_2}\\pi_{s_1}b_{s_1}(o_1)a_{s_1s_2}b_{s_2}(o_2)\"

The hidden path is unspecified, so its mutually exclusive alternatives must be summed.

Solution

Step 1. Score H,H
\"0.8(0.2)(0.7)(0.4)=0.0448\"
Step 2. Score H,C
\"0.8(0.2)(0.3)(0.1)=0.0048\"
Step 3. Score C,H
\"0.2(0.5)(0.4)(0.4)=0.016\"
Step 4. Score C,C
\"0.2(0.5)(0.6)(0.1)=0.006\"
Step 5. Add all paths
\"P(1,3)=0.0448+0.0048+0.016+0.006=0.0716\"
Final answer. Observation probability =0.0716. The best single-path score 0.0448 is a different quantity.

If asked for the posterior of path H,H given observations 1,3, normalize: 0.0448/0.0716 ≈0.625698. Viterbi chooses the best path using joint scores because the observation denominator is common to every candidate. It does not compute that denominator by taking a maximum.

"}],"scope":"Formulas explained through full worked examples"},{"id":"f-wsd","title":"WSD: Lesk & Naive Bayes","unit":0,"sections":[{"id":"u5-lesk","title":"Lesk algorithm, from words to overlap scores","html":"

Lesk is a knowledge-based WSD method that chooses senses whose dictionary descriptions have the greatest overlap with contextual evidence. Original Lesk compares definitions of neighbouring words' senses. A common simplified version compares each target-sense gloss directly with the sentence context. State which version you use.

Simplified Lesk procedure

  1. Remove the target word from the context; tokenize and normalize the remaining content words.
  2. Obtain the gloss for each candidate sense and apply the same normalization.
  3. Count shared words between the context and each gloss.
  4. Select the sense with maximum overlap. If scores tie, report the tie or use a specified tie-break rule.
\"\\operatorname{score}(s)=|C\\cap

\"C\" is the set of context words, \"G_s\" the set of words in sense \"s\"'s gloss, \"\\cap\" means common elements, \"|\\cdot|\" means count, and \"\\arg\\max\" means “the sense giving the largest score”. This version counts unique matching words once.

Question. Use simplified Lesk to disambiguate “bank” in “The river water rose near the bank.” Use C = {river, water, rose, near}; financial gloss = {institution, money, deposit}; river gloss = {land, river, water, edge}.

Given

The normalized context and two gloss sets are given. Count exact shared words.

To find

The score of each sense and the selected meaning.

Formula and why it applies

Use \"\\operatorname{score}(s)=|C\\cap because the task specifies context–gloss overlap.

Step 1. Compare the financial sense

No word occurs in both {river, water, rose, near} and {institution, money, deposit}. Overlap = {}.

Step 2. Count the financial score

\"\\operatorname{score}(F)=0\".

Step 3. Compare the river sense

Shared words are {river, water}. “Rose” and “near” are not in the gloss.

Step 4. Count and select

\"\\operatorname{score}(R)=2\". Since \"2>0\", choose the river-edge sense.

Answer. Bank means land beside a river.

Question. Use simplified Lesk for “The dog barked loudly.” Treat barked as bark. Context C = {dog, loudly}; sound gloss = {sharp, cry, dog}; tree gloss = {outer, covering, tree}; ship gloss = {sailing, ship}.

Given

Three glosses and normalized context {dog, loudly}.

To find

Select the sense of bark.

Formula and why it applies

\"\\operatorname{score}(s)=|C\\cap. Lemmatization lets barked match the target lemma bark; it does not create additional overlap words.

Step 1. Sound sense

Intersection = {dog}; score = 1.

Step 2. Tree sense

Intersection = {}; score = 0.

Step 3. Ship sense

Intersection = {}; score = 0.

Step 4. Compare

The sound sense has the unique maximum: 1 > 0.

Answer. Bark is the dog's cry.

Question. Apply definition–definition Lesk to “pine cone”. Pine senses: P1 = {evergreen, tree, needle, leaf}; P2 = {waste, sorrow, illness}. Cone senses: C1 = {solid, shape, circular}; C2 = {fruit, evergreen, tree}.

Given

Two candidate senses for each word. Tokens are already normalized; tree and trees would be reduced to the same form if necessary.

To find

The most compatible pair of senses.

Formula and why it applies

Use \"|G_{P_i}\\cap for every pair because this is original-style gloss–gloss comparison.

Step 1. P1 with C1

No shared word → 0.

Step 2. P1 with C2

Shared {evergreen, tree} → 2.

Step 3. P2 with each cone sense

P2–C1 has no overlap → 0; P2–C2 has no overlap → 0.

Step 4. Choose the best pair

The four scores are 0, 2, 0, 0. Maximum 2 belongs to P1–C2.

Answer. Pine is the evergreen tree; cone is its fruit-like reproductive structure.

Limitations: exact overlap misses synonyms, short glosses may produce zeros, and different preprocessing choices change the score. Extended Lesk may include related synset definitions or longer phrase overlaps; do not silently substitute it for a simple question.

Lesk: context aur dictionary mein common words gino; sabse bada overlap jeeta.

"},{"id":"u5-walker","title":"Thesaurus / Walker approach, with three complete examples","html":"

The Walker approach uses thesaurus categories as semantic evidence. Associate each candidate sense with a category. Count or weight context words that support that category. Choose the largest total.

\"\\operatorname{score}(s)=\\sum_{w\\in

\"I=1\" if the supplied category mapping says the context word supports the sense, otherwise \"I=0\". The summation symbol means add all those contributions. Weighted questions may replace 1 with a supplied weight. Category membership must be supplied or justified; it is not guessed from word frequency.

Question. Disambiguate bank using Walker scoring. Context words are money, interest, annum, fetch. The finance category supports money, interest and annum; the location category supports none of these words.

Given

Two categories: Finance and Location. Each supported word contributes 1.

To find

The two totals and selected bank sense.

Formula and why it applies

Sum the category indicators for each sense.

Step 1. Make a contribution table

WordFinanceLocation
money10
interest10
annum10
fetch00

Step 2. Add Finance

\"1+1+1+0=3\".

Step 3. Add Location

\"0+0+0+0=0\".

Step 4. Select

3 > 0, so select the financial institution.

Answer. Finance, with score 3 versus 0.

Question. Use Walker scoring for “The fisherman caught bass beside the lake.” Context {fisherman,caught,lake}. The Fish category contains all three clues; the Music category contains none.

Given

Fish and Music senses, binary category membership given.

To find

The correct bass sense.

Formula and why it applies

Score = sum of context-category matches.

Step 1. Record Fish matches

fisherman → 1; caught → 1; lake → 1.

Step 2. Add Fish score

\"1+1+1=3\".

Step 3. Add Music score

\"0+0+0=0\".

Step 4. Choose

Fish has the larger score.

Answer. Bass refers to a fish.

Question. Score “The player used a bat in the match.” Context {player,match}. Sport supports both words with weights 2 and 1; Animal supports neither. Use the supplied weighted Walker scheme.

Given

Sport weights {player:2, match:1}; Animal weights {player:0, match:0}.

To find

Weighted sense scores.

Formula and why it applies

Use the sum of supplied weights, not an unweighted count, because the question explicitly gives weights.

Step 1. Sport contribution of player

Add 2.

Step 2. Sport contribution of match

Add 1. Sport total = 2 + 1 = 3.

Step 3. Animal total

0 + 0 = 0.

Step 4. Compare

3 > 0 selects the sporting implement.

Answer. Bat is the sports implement.

Walker: dictionary ke exact words nahi, context kis topic/category ko support karta hai woh count.

"},{"id":"u5-bayes-steps","title":"Multinomial Naive Bayes: the exact solving method","html":"

Naive Bayes selects the sense with the largest prior × likelihood product. The context evidence denominator is identical across candidate senses, so it can be omitted for comparison.

\"\\operatorname{score}(s)=P(s)\\prod_j

For add-one multinomial smoothing:

\"P(w\\mid

\"C(w,s)\" counts occurrences of word \"w\" in training contexts of sense \"s\"; \"N_s\" is the total number of word occurrences in those contexts; \"V\" is the shared vocabulary size. It is not the number of training sentences. Repeated query words contribute repeated factors.

Always keep these counts separate

Class priors use number of examples. Word likelihoods in this multinomial model use number of word occurrences. The vocabulary counts unique words across all classes.

  1. List the class examples and calculate priors.
  2. Write one shared vocabulary and count its size.
  3. Count total word occurrences in each class.
  4. Count each query word in each class.
  5. Apply the given smoothing rule.
  6. Multiply each prior by all its query likelihoods.
  7. Compare scores; divide each score by their sum only if a normalized posterior is requested.
\"P(s\\mid

Why smoothing is needed: an unseen clue gives \"0/N_s=0\"; one zero makes the whole product zero. Add-one smoothing redistributes probability across every vocabulary word and prevents these zero factors. Use it only when requested, taught for the task, or state the assumption.

Different model warning: a Bernoulli Naive Bayes question counts presence/absence per document and may include absence factors. Do not use its denominator in a multinomial word-count question.

Pehle sentences se prior; phir words se likelihood; dono counts mix mat kar.

"},{"id":"u5-bayes-mam","title":"Mam's bank numerical—every count and every multiplication","html":"

Question. Classify the sense of “bank” using the training table below. The sentence is: “As the stream flowed gently past the shore, we set up camp by the bank of the river, discussing the possibility of borrowing a loan to buy a boat for future trips.” Use only the highlighted feature words stream, shore and loan, and add-one multinomial smoothing.

Given

Training rowContext wordsSense
1money, deposit, loanFinance
2river, water, shoreRiverbank
3account, savings, investmentFinance
4boat, river, streamRiverbank
5branch, deposit, checkFinance
6fishing, shore, riverRiverbank

Query feature set: {stream, shore, loan}. Other sentence words are not included as features in this supplied example.

To find

The most probable bank sense and normalized posteriors.

Formula and why it applies

Naive Bayes selects the sense with the largest prior × likelihood product. The context evidence denominator is identical across candidate senses, so it can be omitted for comparison.

\"\\operatorname{score}(s)=P(s)\\prod_j

For add-one multinomial smoothing:

\"P(w\\mid

\"C(w,s)\" counts occurrences of word \"w\" in training contexts of sense \"s\"; \"N_s\" is the total number of word occurrences in those contexts; \"V\" is the shared vocabulary size. It is not the number of training sentences. Repeated query words contribute repeated factors.

Step 1. Calculate priors

Finance appears in 3 of the 6 rows; Riverbank appears in 3 of 6.

\"P(F)=\\frac36=\\frac12,\\qquad

Step 2. Construct the vocabulary

{money, deposit, loan, river, water, shore, account, savings, investment, boat, stream, fishing, branch, check}. Thus V=14. Repeated deposit or river is counted once in vocabulary.

Step 3. Count word occurrences per class

There are 3 finance rows × 3 words = 9 word occurrences. There are 3 river rows × 3 words = 9. Thus \"N_F=N_R=9\", and each smoothed denominator is \"9+14=23\".

Step 4. Count the three query words

WordFinance countRiverbank count
stream01
shore02
loan10

Step 5. Calculate finance likelihoods

\"P(\\text{stream}\\mid
\"P(\\text{shore}\\mid
\"P(\\text{loan}\\mid

Step 6. Multiply the finance score

\"\\operatorname{score}(F)=\\frac12\\times\\frac1{23}\\times\\frac1{23}\\times\\frac2{23}=\\frac1{23^3}=\\frac1{12167}\\approx0.00008219.\"

Step 7. Calculate river likelihoods

\"P(\\text{stream}\\mid
\"P(\\text{shore}\\mid
\"P(\\text{loan}\\mid

Step 8. Multiply the river score

\"\\operatorname{score}(R)=\\frac12\\times\\frac2{23}\\times\\frac3{23}\\times\\frac1{23}=\\frac3{12167}\\approx0.00024657.\"

Step 9. Compare scores

3/12167 is larger than 1/12167. Riverbank wins even though “loan” is financial evidence: the combined clues matter.

Step 10. Normalize if a posterior is requested

\"P(F\\mid
\"P(R\\mid

Answer. Bank is the riverbank sense. Scores are 1/12167 and 3/12167; normalized model posteriors are 0.25 and 0.75.

Without smoothing: stream and shore are absent in Finance; loan is absent in Riverbank. Both unsmoothed scores become zero, so there is no valid winning class from those products. Also, unnormalized scores should not be labelled as final posterior probabilities.

Zero word? Add-one: count+1 upar, total words+vocabulary neeche. Last mein score bada wala sense.

"},{"id":"u5-bayes-practice","title":"More WSD numericals: given likelihoods, unequal priors and repeated clues","html":"

Question. For bass, let P(Fish)=0.6 and P(Music)=0.4. Given Fish likelihoods P(caught|Fish)=0.5, P(lake|Fish)=0.4 and Music likelihoods P(caught|Music)=0.05, P(lake|Music)=0.1, classify context {caught,lake} and normalize.

Given

Priors and likelihoods are provided directly; no smoothing or training-count estimation is requested.

To find

The winning sense and its posterior.

Formula and why it applies

\"\\operatorname{score}(s)=P(s)P(\\text{caught}\\mid, because the two features are treated as conditionally independent.

Step 1. Fish multiplication

\"0.6\\times0.5=0.3\", then \"0.3\\times0.4=0.12\".

Step 2. Music multiplication

\"0.4\\times0.05=0.02\", then \"0.02\\times0.1=0.002\".

Step 3. Total evidence

\"0.12+0.002=0.122\".

Step 4. Normalize

\"P(Fish\\mid
\"P(Music\\mid

Answer. Fish, with model posterior approximately 0.983607.

Question. Disambiguate bank from the repeated context tokens {loan, loan, river}. Priors are Finance=2/3 and River=1/3. Vocabulary V=4. Finance has N=8 total tokens, loan count 3 and river count 0; River has N=4, loan count 0 and river count 2. Apply add-one smoothing.

Given

Multinomial counts are supplied. The two appearances of loan must both contribute.

To find

Scores and normalized Finance posterior.

Formula and why it applies

Naive Bayes selects the sense with the largest prior × likelihood product. The context evidence denominator is identical across candidate senses, so it can be omitted for comparison.

\"\\operatorname{score}(s)=P(s)\\prod_j

For add-one multinomial smoothing:

\"P(w\\mid

\"C(w,s)\" counts occurrences of word \"w\" in training contexts of sense \"s\"; \"N_s\" is the total number of word occurrences in those contexts; \"V\" is the shared vocabulary size. It is not the number of training sentences. Repeated query words contribute repeated factors.

Step 1. Finance denominator

\"8+4=12\". \"P(loan\\mid; \"P(river\\mid.

Step 2. River denominator

\"4+4=8\". \"P(loan\\mid; \"P(river\\mid.

Step 3. Finance product

\"\\operatorname{score}(F)=\\frac23\\times\\frac13\\times\\frac13\\times\\frac1{12}=\\frac1{162}\\approx0.00617284.\"

Step 4. River product

\"\\operatorname{score}(R)=\\frac13\\times\\frac18\\times\\frac18\\times\\frac38=\\frac1{512}\\approx0.001953125.\"

Step 5. Compare and normalize

\"P(F\\mid

Answer. Finance wins. Repeated loan contributes a squared likelihood; do not reduce the query to a unique-word set.

Answer-writing order in an exam

Write the class priors, vocabulary size, class token totals, a likelihood table, both score products and the final class. Keep fractions until the comparison where possible. If the question only asks classification, normalization is optional; if it asks posterior probabilities, normalization is required.

Repeated word aaye toh factor bhi repeat. Sirf word-set bana diya toh multinomial answer badal jayega.

"}],"scope":"Formulas explained through full worked examples"}],"papers":[{"id":"final-2025","title":"Solved final paper · December 2025","unit":"PYQ","scope":"M2-relevant questions from an actual TEE paper · original marks retained for longer practice","sections":[{"id":"final-2025-q1b","title":"Q1B · Chunking and chinking","html":"
Question. Explain the concepts of chunking and chinking in Natural Language Processing (NLP). Discuss how Part-of-Speech (POS) tagging supports these processes, and illustrate your explanation with a suitable example. [5 marks]

Answer

Chunking groups adjacent tagged words into shallow phrases, such as noun phrases, without building a complete sentence parse. Chinking removes a specified sequence of tags or words from a chunk, splitting a larger candidate chunk when required.

POS tags provide the pattern

A POS tag describes a grammatical category: DT = determiner, JJ = adjective, NN = singular noun, VBD = past-tense verb and IN = preposition. A noun-phrase rule can be written as NP: {<DT>?<JJ>*<NN>+}: an optional determiner, zero or more adjectives, followed by one or more nouns.

Worked example

WordThesmallcatsatonthemat
TagDTJJNNVBDINDTNN

Applying the NP rule gives [The small cat] sat on [the mat]. The first chunk contains DT–JJ–NN; the second contains DT–NN. “sat on” is not included because it does not match the noun-phrase rule.

For chinking, begin with one broad chunk: [The small cat sat on the mat]. Remove the VBD and IN sequence “sat on”, for example with }<VBD|IN>+{. The result is [The small cat] sat on [the mat].

POS tagging therefore supplies grammatical labels; chunking uses labels to select phrases, while chinking uses labels to exclude unwanted material. Incorrect tags can lead to incorrect chunks.

Yaad rakho: Chunking = phrase ko jodo; chinking = beech ka unwanted part nikalo.

","m2":true,"units":"4","marks":5},{"id":"final-2025-q1c","title":"Q1C · Hyponymy, hypernymy and WSD","html":"
Question. Define hyponymy and hypernymy with suitable linguistic examples. How can hypernymy be applied in Word Sense Disambiguation (WSD) to resolve the correct meaning of polysemous words? Provide an example? [5 marks]

Answer

Hyponymy is the “is-a-kind-of” relation from a specific concept to a broader concept. Hypernymy is the inverse relation from the broader concept to the specific one. “Rose” is a hyponym of “flower”; “flower” is a hypernym of “rose”.

rose / tulip → flower → plant

The arrow points towards a broader category. Hyponymy is not a part–whole relation: a wheel is part of a car, but it is not a kind of car.

Using the hierarchy for WSD

WSD selects the intended sense of a word from its context. Obtain candidate senses from a lexical resource such as WordNet, inspect their hypernym chains, and favour the chain most compatible with nearby words.

  1. Sentence: “The mouse clicked the icon.”
  2. Candidate 1: mouse → rodent → mammal → animal. Candidate 2: mouse → pointing device → computer peripheral.
  3. The context words “clicked” and “icon” belong to the computer-use domain. They support the device hierarchy, rather than the animal hierarchy.
  4. Select the computer-pointing-device sense. In “The mouse ate cheese,” the animal hierarchy is more compatible.

Hypernym evidence supplies semantic generalisation; it is a useful signal rather than a guarantee, because short or unusual contexts may still be ambiguous.

Yaad rakho: Hyponym specific hai; hypernym uska bada category hai.

","m2":true,"units":"5","marks":5},{"id":"final-2025-q2b","title":"Q2B · Laplace-smoothed bigram probability","html":"
Question. Consider the following training corpus:
1. artists paint murals in city
2. musicians play songs in city
3. painters paint murals on walls
4. signs are near city
Using a bigram language model with Laplace smoothing, evaluate the probability of the test sentence: “artists paint murals near city”. Assume the <s> and </s> for each sentence. [10 marks]

Answer

Given

Test: artists paint murals near city. Add one start token <s> and one end token </s> to every training and test sentence.

To find

The Laplace-smoothed bigram probability of the complete test sentence, including its end token.

Formula: which one and why

\"P_{+1}(w_i\\mid

C(previous,current) counts that adjacent pair in the training corpus. C(previous) counts its occurrences as a history. V is the vocabulary size. Add-one smoothing is necessary because an unseen pair must not force the whole sentence probability to zero.

\"P(w_1,\\ldots,w_n,\\langle/s\\rangle\\mid\\langle

Solution

Step 1. Fix the token and vocabulary convention

Lowercase words and remove the final punctuation. The supplied course key counts both boundary tokens in V. Following that convention: are, artists, city, in, murals, musicians, near, on, paint, painters, play, signs, songs, walls, <s>, </s> gives V = 16. The start marker supplies history; it is not an extra probability factor before the first word.

Step 2. List every required transition

The complete test chain is <s> → artists → paint → murals → near → city → </s>.

Step 3. Count first, then add one

Required bigramHistory countPair countSmoothed probability
<s> → artists41(1+1)/(4+16) = 1/10
artists → paint11(1+1)/(1+16) = 2/17
paint → murals22(2+1)/(2+16) = 1/6
murals → near20(0+1)/(2+16) = 1/18
near → city11(1+1)/(1+16) = 2/17
city → </s>33(3+1)/(3+16) = 4/19

Count only adjacent words, never words that merely appear in the same sentence. The unseen pairs have count zero, so their numerator becomes one.

Step 4. Multiply all six probabilities

\"P=\\frac{2}{20}\\times\\frac{2}{17}\\times\\frac{3}{18}\\times\\frac{1}{18}\\times\\frac{2}{17}\\times\\frac{4}{19}\"

Start with subtotal 1. At each row, multiply the previous subtotal by exactly one next factor. Simplify the fraction after multiplication, but do not round it.

Factor being includedPrevious subtotal × next factor = new subtotal
1. <s> → artists\"1\\times\\frac{1}{10}=\\frac{1}{10}\"
2. artists → paint\"\\frac{1}{10}\\times\\frac{2}{17}=\\frac{1}{85}\"
3. paint → murals\"\\frac{1}{85}\\times\\frac{1}{6}=\\frac{1}{510}\"
4. murals → near\"\\frac{1}{510}\\times\\frac{1}{18}=\\frac{1}{9180}\"
5. near → city\"\\frac{1}{9180}\\times\\frac{2}{17}=\\frac{1}{78030}\"
6. city → </s>\"\\frac{1}{78030}\\times\\frac{4}{19}=\\frac{2}{741285}\"

The last subtotal is the complete product. Only now convert the fraction to a decimal.

\"P=\\frac{2}{741285}\\approx

Step 5. Interpret the result

This is a joint sentence probability, so a small value is normal. Do not add transition probabilities; each word must occur in sequence, so the chain multiplies them.

Final answer

P ≈ 2.69801763 × 10-6 with the stated course vocabulary convention.

Yaad rakho: Pair count mein 1 jodo, previous-word count mein V; phir saare fractions multiply karo.

Convention note: Some implementations exclude <s> from the predicted vocabulary. Their V is one smaller and their numerical probability differs. In an exam, state V explicitly and follow the convention specified in the question or class solution.

","m2":true,"units":"3","marks":10},{"id":"final-2025-q4b","title":"Q4B · HMM emission, transition and sequence","html":"
Question. Consider the following POS tagged corpus. Apply Hidden Markov Model (HMM) approach to:
1. Cat/Noun chase/Verb mice/Noun
2. The/Det dog/Noun runs/Verb
3. A/Det cat/Noun eats/Verb cheese/Noun
i. Construct the emission probability matrix.
ii. Construct the transmission probability matrix.
iii. Calculate the probability of sequence of events: The/Det cat/Noun eats/Verb mice/Noun.
Note: Include start and end characters for all the sentences. [10 marks]

Answer

Given

SentenceTagged sequence
1cat/Noun chase/Verb mice/Noun
2the/Det dog/Noun runs/Verb
3a/Det cat/Noun eats/Verb cheese/Noun

Test words: the cat eats mice. Given tag sequence: Det → Noun → Verb → Noun. Include Start and End for all sentences. Lowercase the corpus so sentence-initial Cat and cat are one word.

To find

Emission probabilities, transition probabilities and the joint probability of the specified tag-and-word sequence.

Formula: which one and why

\"P(w\\mid

An emission probability asks: among tokens carrying this tag t, what fraction are the word w?

\"P(t_j\\mid

A transition probability asks: among transitions leaving tag tᵢ, what fraction go to tⱼ? Include End in the outgoing counts. The paper says “transmission”; the standard HMM term is transition.

\"P(\\mathbf

Solution

Step 1. Count tokens for each tag

Det has 2 tokens; Noun has 5 tokens; Verb has 3 tokens. The repeated noun cat occurs twice with Noun; every other displayed noun occurs once. The three verb types each occur once.

Step 2. Build the emission matrix

Tag / wordacatchasecheesedogeatsmicerunsthe
Det1/200000001/2
Noun02/501/51/501/500
Verb001/3001/301/30

Each row sums to one. A zero means no occurrence under that tag in this unsmoothed training corpus; no add-one smoothing was requested.

Step 3. Write the three tag chains

Start → Noun → Verb → Noun → End
Start → Det → Noun → Verb → End
Start → Det → Noun → Verb → Noun → End

Step 4. Count outgoing transitions

FromTo DetTo NounTo VerbTo EndOutgoing total
Start21003
Det02002
Noun00325
Verb02013

Step 5. Divide each row by its outgoing total

From / ToDetNounVerbEnd
Start2/31/300
Det0100
Noun003/52/5
Verb02/301/3

End is terminal, so it has no estimated outgoing row. The four nonterminal rows each sum to one.

Step 6. Choose the factors for the given sequence

Event in orderProbability
Start → Det2/3
Det emits the1/2
Det → Noun1
Noun emits cat2/5
Noun → Verb3/5
Verb emits eats1/3
Verb → Noun2/3
Noun emits mice1/5
Noun → End2/5

Step 7. Multiply, including the end transition

\"P=\\frac23\\times\\frac12\\times1\\times\\frac25\\times\\frac35\\times\\frac13\\times\\frac23\\times\\frac15\\times\\frac25\"

Start with subtotal 1. At each row, multiply the previous subtotal by exactly one next factor. Simplify the fraction after multiplication, but do not round it.

Factor being includedPrevious subtotal × next factor = new subtotal
1. Start → Det\"1\\times\\frac{2}{3}=\\frac{2}{3}\"
2. Det emits the\"\\frac{2}{3}\\times\\frac{1}{2}=\\frac{1}{3}\"
3. Det → Noun\"\\frac{1}{3}\\times1=\\frac{1}{3}\"
4. Noun emits cat\"\\frac{1}{3}\\times\\frac{2}{5}=\\frac{2}{15}\"
5. Noun → Verb\"\\frac{2}{15}\\times\\frac{3}{5}=\\frac{2}{25}\"
6. Verb emits eats\"\\frac{2}{25}\\times\\frac{1}{3}=\\frac{2}{75}\"
7. Verb → Noun\"\\frac{2}{75}\\times\\frac{2}{3}=\\frac{4}{225}\"
8. Noun emits mice\"\\frac{4}{225}\\times\\frac{1}{5}=\\frac{4}{1125}\"
9. Noun → End\"\\frac{4}{1125}\\times\\frac{2}{5}=\\frac{8}{5625}\"

The last subtotal is the complete product. Only now convert the fraction to a decimal.

\"P=\\frac8{5625}\\approx0.0014222222\"

Final answer

Joint sequence probability = 8/5625 ≈ 0.0014222222. This is the probability of the specified words and tags together. It is not the probability of the word sequence summed over every possible tag path.

Yaad rakho: Tag se tag = transition; tag se word = emission. Dono alternate multiply, end ko mat bhoolo.

Boundary reminder: The initial factor is P(Det | Start) = 2/3, based on three sentence starts, and the final factor is P(End | Noun) = 2/5. The overall fraction of Det tokens, 2/10, is not the start probability. Using it and omitting End, as in the handwritten key’s last line, computes a different quantity.

","m2":true,"units":"4","marks":10},{"id":"final-2025-q5b","title":"Q5B · Naïve Bayes word-sense classification","html":"
Question. Consider the sentence: “The bat hit the ball hard on the cricket field.” Two possible senses of “bat”: Sense 1: Bat (Sports equipment – cricket/baseball); Sense 2: Bat (Nocturnal flying mammal).
Sentence IDContext wordsSense
1ball, cricket, score, stadiumSports
2night, cave, wings, flyAnimal
3player, run, match, batSports
4nocturnal, vampire, insectAnimal
5team, field, bowl, batSports
6dark, flying, cave, nightAnimal
a) Using the Naïve Bayes algorithm, classify the sense of “bat” in the given sentence. Apply Laplace Smoothing wherever applicable.
b) Discuss the significance of Laplace Smoothing in Naïve Bayes Classification. [10 marks]

Answer

Given

There are three Sports examples and three Animal examples. Treat training context lists as bag-of-words token counts. Exclude the ambiguous target “bat” from the test context. Use the fixed training vocabulary; ignore out-of-vocabulary test words hit and hard. The usable test features are ball, cricket and field.

To find

The posterior-score comparison for Sports and Animal, and the role of smoothing.

Formula: which one and why

\"\\mathrm{score}(s)=P(s)\\prod_{w\\in\\mathrm{context}}P(w\\mid
\"P(w\\mid

P(s) is the class prior, C(w,s) counts occurrences of context word w in class s, Nₛ is the total number of context tokens in that class and V is the shared vocabulary size. Conditional independence lets us multiply word probabilities. The common denominator P(context) is unnecessary when only comparing classes.

Solution

Step 1. Calculate the priors

\"P(S)=\\frac36=0.5,\\qquad

Step 2. Count class tokens and vocabulary

Sports has 4 + 4 + 4 = 12 tokens. Animal has 4 + 3 + 4 = 11 tokens. The shared training vocabulary has 20 distinct words: ball, cricket, score, stadium, player, run, match, bat, team, field, bowl, night, cave, wings, fly, nocturnal, vampire, insect, dark, flying. “bat”, “night” and “cave” repeat but count only once each towards V.

Step 3. Look up each usable context word

FeatureSports countAnimal countP(word | Sports)P(word | Animal)
ball10(1+1)/(12+20) = 2/32(0+1)/(11+20) = 1/31
cricket102/321/31
field102/321/31

Step 4. Multiply the Sports score

\"\\mathrm{score}(S)=0.5\\times\\frac2{32}\\times\\frac2{32}\\times\\frac2{32}=0.0001220703125\"

Step 5. Multiply the Animal score

\"\\mathrm{score}(A)=0.5\\times\\frac1{31}\\times\\frac1{31}\\times\\frac1{31}\\approx0.00001678359236\"

Step 6. Compare the scores

0.0001220703125 is greater than 0.00001678359236. Therefore select Sports. Optional normalization gives P(Sports | retained features) ≈ 0.87913; normalization is not needed for the decision.

Final answer

The intended sense is bat as sports equipment. Ball, cricket and field favour that sense.

Yaad rakho: Pehle class prior, phir har context-word likelihood multiply; bigger score wali sense choose karo.

Why Laplace smoothing matters

Without smoothing, all three test features have zero count in Animal. One zero factor makes the entire class score zero, even if other evidence could support that class. Add-one smoothing assigns nonzero probability to every word in the defined vocabulary and redistributes probability mass. It does not mean every unseen word outside the vocabulary can be inserted without changing the vocabulary model.

If following the supplied paper key exactly

The key also multiplies probabilities for hit and hard using the same denominators 32 and 31. That convention produces Sports = 1.19209290 × 10⁻⁷ and Animal ≈ 1.74647163 × 10⁻⁸, again selecting Sports. Its treatment of out-of-vocabulary words is not a normalized fixed-vocabulary model. State your feature convention clearly; the complete calculation above uses a consistent fixed training vocabulary.

","m2":true,"units":"5","marks":10}]},{"id":"reexam-2026","title":"Solved re-exam · February 2026","unit":"PYQ","scope":"M2-relevant questions from an actual TEE paper · original marks retained for longer practice","sections":[{"id":"reexam-2026-q1b","title":"Q1B · Content and function words","html":"
Question. i. Define function words and content words.
ii. Which category (function words and content words) belong to open class and closed class? Justify your answer.
iii. For the sentence “The small cat is sleeping under the tree”, identify the function words and content words. [5 marks]

Answer

Content words carry much of a sentence’s lexical meaning, usually including nouns, main verbs, adjectives and many adverbs. Function words primarily express grammatical relationships, including determiners, prepositions, conjunctions, pronouns and auxiliary verbs.

CategoryTypical classReason
Content wordsOpen classNew nouns, verbs and adjectives enter the language readily, such as new technology names
Function wordsClosed classThe grammatical inventory is relatively stable; new determiners or pronouns are much less frequently added
WordRole in this sentenceCategory
TheDeterminerFunction
smallAdjectiveContent
catNounContent
isAuxiliary supporting progressive tenseFunction
sleepingMain verbContent
underPrepositionFunction
theDeterminerFunction
treeNounContent

Final answer: content words = small, cat, sleeping, tree; function words = The, is, under, the. “is” is auxiliary here; a word’s role must be judged in context. “Closed” means relatively resistant to new members, not that change is logically impossible.

Yaad rakho: Content words scene batate hain; function words grammar ka connection banate hain.

","m2":true,"units":"4","marks":5},{"id":"reexam-2026-q2b","title":"Q2B · Laplace smoothing on a second corpus","html":"
Question. Consider the following training corpus:
1. cats chase mice in the garden
2. dogs bark loudly at night
3. birds sing in the morning
4. children play games in the park
Using a bigram language model with Laplace smoothing, evaluate the probability of the test sentence: “cats bark in the park.” Assume <s> and </s> as start and end of the sentence. [10 marks]

Answer

Given

Test: cats bark in the park. Add one start token <s> and one end token </s> to every training and test sentence.

To find

The Laplace-smoothed bigram probability of the complete test sentence, including its end token.

Formula: which one and why

\"P_{+1}(w_i\\mid

C(previous,current) counts that adjacent pair in the training corpus. C(previous) counts its occurrences as a history. V is the vocabulary size. Add-one smoothing is necessary because an unseen pair must not force the whole sentence probability to zero.

\"P(w_1,\\ldots,w_n,\\langle/s\\rangle\\mid\\langle

Solution

Step 1. Fix the token and vocabulary convention

Lowercase words and remove the final punctuation. The supplied course key counts both boundary tokens in V. Following that convention: at, bark, birds, cats, chase, children, dogs, games, garden, in, loudly, mice, morning, night, park, play, sing, the, <s>, </s> gives V = 20. The start marker supplies history; it is not an extra probability factor before the first word.

Step 2. List every required transition

The complete test chain is <s> → cats → bark → in → the → park → </s>.

Step 3. Count first, then add one

Required bigramHistory countPair countSmoothed probability
<s> → cats41(1+1)/(4+20) = 1/12
cats → bark10(0+1)/(1+20) = 1/21
bark → in10(0+1)/(1+20) = 1/21
in → the33(3+1)/(3+20) = 4/23
the → park31(1+1)/(3+20) = 2/23
park → </s>11(1+1)/(1+20) = 2/21

Count only adjacent words, never words that merely appear in the same sentence. The unseen pairs have count zero, so their numerator becomes one.

Step 4. Multiply all six probabilities

\"P=\\frac{2}{24}\\times\\frac{1}{21}\\times\\frac{1}{21}\\times\\frac{4}{23}\\times\\frac{2}{23}\\times\\frac{2}{21}\"

Start with subtotal 1. At each row, multiply the previous subtotal by exactly one next factor. Simplify the fraction after multiplication, but do not round it.

Factor being includedPrevious subtotal × next factor = new subtotal
1. <s> → cats\"1\\times\\frac{1}{12}=\\frac{1}{12}\"
2. cats → bark\"\\frac{1}{12}\\times\\frac{1}{21}=\\frac{1}{252}\"
3. bark → in\"\\frac{1}{252}\\times\\frac{1}{21}=\\frac{1}{5292}\"
4. in → the\"\\frac{1}{5292}\\times\\frac{4}{23}=\\frac{1}{30429}\"
5. the → park\"\\frac{1}{30429}\\times\\frac{2}{23}=\\frac{2}{699867}\"
6. park → </s>\"\\frac{2}{699867}\\times\\frac{2}{21}=\\frac{4}{14697207}\"

The last subtotal is the complete product. Only now convert the fraction to a decimal.

\"P=\\frac{4}{14697207}\\approx

Step 5. Interpret the result

This is a joint sentence probability, so a small value is normal. Do not add transition probabilities; each word must occur in sequence, so the chain multiplies them.

Final answer

P ≈ 2.72160554 × 10-7 with the stated course vocabulary convention.

Yaad rakho: Pair count mein 1 jodo, previous-word count mein V; phir saare fractions multiply karo.

Convention note: Some implementations exclude <s> from the predicted vocabulary. Their V is one smaller and their numerical probability differs. In an exam, state V explicitly and follow the convention specified in the question or class solution.

","m2":true,"units":"3","marks":10},{"id":"reexam-2026-q3a","title":"Q3A · Top-down parsing","html":"
Question. In the context of natural language processing, explain the concept of top-down and bottom-up parsing approaches. Apply top-down syntax parsing approach to the sentence “The tall boy saw a girl in the park.” and draw a parse tree using the CFG grammar rules:
S → NP VP; NP → Det Adj N | Det N; VP → V NP | VP PP; PP → P NP.
Lexicon: Det → {the | a}; Adj → {tall}; N → {boy | girl | park}; V → {saw}; P → {in}. [10 marks]

Answer

AspectTop-downBottom-up
StartThe start symbol SThe input words
OperationExpand a nonterminal using grammar productionsCombine words and constituents using productions in reverse
SuccessLeaves match the complete inputAll input is reduced to S
Possible difficultyNaive recursive descent can loop on left recursion or need backtrackingA naive shift-reduce parser may face shift/reduce conflicts

Given

The sentence contains the tokens the / tall / boy / saw / a / girl / in / the / park. Lowercase the initial “The” to match the lexicon. The full stop is punctuation outside the listed grammar.

To find

A derivation beginning at S and a parse tree matching the whole sentence.

Rule: which one and why

A context-free production replaces one nonterminal with its right-hand sequence. A top-down derivation begins at S and expands the desired structure. The supplied grammar attaches the PP through VP → VP PP; it does not provide NP → NP PP.

Step 1. Expand the sentence

S ⇒ NP VP. Choose NP → Det Adj N for “the tall boy”.

Step 2. Match the subject words

Det ⇒ the, Adj ⇒ tall, N ⇒ boy. The remaining material must be generated by VP.

Step 3. Allow a verb phrase followed by a PP

Expand the outer VP using VP → VP PP. Then expand the inner VP using VP → V NP. This finite derivation selects the recursive rule once.

Step 4. Match the verb and object

V ⇒ saw. Choose NP → Det N, then Det ⇒ a and N ⇒ girl.

Step 5. Expand the prepositional phrase

PP ⇒ P NP. P ⇒ in. NP ⇒ Det N, with Det ⇒ the and N ⇒ park.

Step 6. Check the complete leaf order

The leaves read “the tall boy saw a girl in the park”, exactly the normalized input. The PP “in the park” attaches to the VP under this supplied grammar.

SNPVPDetAdjNVPPPVNPPNPDetNDetNThetallboysawagirlinthepark
Top-down parse tree showing the PP attached to VP.

Complete derivation

S ⇒ NP VP ⇒ Det Adj N VP ⇒ the tall boy VP ⇒ the tall boy VP PP ⇒ the tall boy V NP PP ⇒ the tall boy saw Det N PP ⇒ the tall boy saw a girl P NP ⇒ the tall boy saw a girl in Det N ⇒ the tall boy saw a girl in the park.

Implementation caution: VP → VP PP is left-recursive. A naive recursive-descent implementation may loop; it must transform or otherwise handle the grammar. A valid finite top-down derivation, as shown above, still exists.

Yaad rakho: Rule jo diya hai wahi use karo: yahan “in the park” VP ka part hai.

","m2":true,"units":"4","marks":10},{"id":"reexam-2026-q3b","title":"Q3B · Lesk word-sense disambiguation","html":"
Question. Consider the sentence: “The bank can ensure your deposits are safe.” The word “bank” has the following two possible senses (WordNet-style glosses):
Bank₁: Sloping land especially beside a body of water.
Bank₂: A financial institution that accepts deposits and channels the money into lending activities.
i. Apply the LESK algorithm to determine the correct sense of the word “bank” in the given sentence. Show the steps clearly.
ii. Briefly discuss limitations of the LESK algorithm in handling ambiguous words in real-world contexts. [10 marks]

Answer

Given

Context sentence and the two supplied dictionary glosses. Use a simplified Lesk method: lowercase, remove function words, exclude the target bank and compare distinct content-word overlap. Do not add synonyms that the question does not supply.

To find

Which candidate gloss has the greatest overlap with the local context.

Formula: which one and why

\"\\mathrm{score}(s)=|C\\cap

C is the set of context content words. Gₛ is the content-word set of the gloss for sense s. The intersection contains words appearing in both; its size is the score. This is an overlap count, not a probability.

Solution

Step 1. Prepare the context

Remove The, can, your, are and the target bank. A reasonable content set is C = {ensure, deposits, safe}. If “can” is retained under another fixed stop-list it matches neither gloss and does not change the result.

Step 2. Prepare Bank₁’s gloss

G₁ = {sloping, land, especially, beside, body, water}. There is no shared word with {ensure, deposits, safe}.

Step 3. Calculate Bank₁’s score

\"C\\cap

Step 4. Prepare Bank₂’s gloss and count overlap

G₂ = {financial, institution, accepts, deposits, channels, money, lending, activities}. The shared word is deposits.

Step 5. Calculate Bank₂’s score and select

\"C\\cap

1 > 0, so choose the financial-institution sense.

Final answer

Bank₂: financial institution. The explicit overlap “deposits” supports this choice.

Yaad rakho: Lesk: context aur har meaning ki definition mein common words count karo; highest overlap choose karo.

ii. Limitations

Lemmatization, richer glosses and semantic similarity can improve variants, but the exact variant and tie-breaking rule should be stated.

","m2":true,"units":"5","marks":10},{"id":"reexam-2026-q4b","title":"Q4B · A second HMM sequence","html":"
Question. Consider the following POS tagged corpus. Apply Hidden Markov Model (HMM) approach to:
1. Dog/Noun eats/Verb bread/Noun
2. The/Det cat/Noun sleeps/Verb
3. A/Det dog/Noun eats/Verb fish/Noun
i. Construct the emission probability matrix.
ii. Construct the transmission probability matrix.
iii. Calculate the probability of the given sequence of events: The/Det dog/Noun eats/Verb bread/Noun.
Note: Include start and end characters for all the sentences. [10 marks]

Answer

The repeated verb eats has a different emission count from the final paper; recalculate it rather than copying the previous numerical result.

Given

SentenceTagged sequence
1dog/Noun eats/Verb bread/Noun
2the/Det cat/Noun sleeps/Verb
3a/Det dog/Noun eats/Verb fish/Noun

Lowercase the corpus. Test sequence: the/Det dog/Noun eats/Verb bread/Noun. Add Start and End markers.

To find

Emission and transition matrices, then the probability of the supplied tagged sequence.

Formula: which one and why

\"P(w\\mid
\"P(t_j\\mid

Emission divides the word’s tag-specific count by the total tokens with that tag. Transition divides the tag-pair count by all transitions leaving the previous tag, including End.

\"P(\\mathbf

Solution

Step 1. Count tag totals and word frequencies

Det: the=1, a=1, total2. Noun: dog=2, bread=1, cat=1, fish=1, total5. Verb: eats=2, sleeps=1, total3.

Step 2. Construct emission matrix

Tag / wordabreadcatdogeatsfishsleepsthe
Det1/20000001/2
Noun01/51/52/501/500
Verb00002/301/30

Step 3. Count transitions from the three chains

Start → Noun → Verb → Noun → End
Start → Det → Noun → Verb → End
Start → Det → Noun → Verb → Noun → End

FromDet countNoun countVerb countEnd countTotal
Start21003
Det02002
Noun00325
Verb02013

Step 4. Divide outgoing counts

From / toDetNounVerbEnd
Start2/31/300
Det0100
Noun003/52/5
Verb02/301/3

All nonterminal rows sum to one. No smoothing is requested.

Step 5. Write every sequence factor

EventProbability
Start → Det2/3
Det emits the1/2
Det → Noun1
Noun emits dog2/5
Noun → Verb3/5
Verb emits eats2/3
Verb → Noun2/3
Noun emits bread1/5
Noun → End2/5

Step 6. Multiply all factors

\"P=\\frac23\\times\\frac12\\times1\\times\\frac25\\times\\frac35\\times\\frac23\\times\\frac23\\times\\frac15\\times\\frac25\"

Start with subtotal 1. At each row, multiply the previous subtotal by exactly one next factor. Simplify the fraction after multiplication, but do not round it.

Factor being includedPrevious subtotal × next factor = new subtotal
1. Start → Det\"1\\times\\frac{2}{3}=\\frac{2}{3}\"
2. Det emits the\"\\frac{2}{3}\\times\\frac{1}{2}=\\frac{1}{3}\"
3. Det → Noun\"\\frac{1}{3}\\times1=\\frac{1}{3}\"
4. Noun emits dog\"\\frac{1}{3}\\times\\frac{2}{5}=\\frac{2}{15}\"
5. Noun → Verb\"\\frac{2}{15}\\times\\frac{3}{5}=\\frac{2}{25}\"
6. Verb emits eats\"\\frac{2}{25}\\times\\frac{2}{3}=\\frac{4}{75}\"
7. Verb → Noun\"\\frac{4}{75}\\times\\frac{2}{3}=\\frac{8}{225}\"
8. Noun emits bread\"\\frac{8}{225}\\times\\frac{1}{5}=\\frac{8}{1125}\"
9. Noun → End\"\\frac{8}{1125}\\times\\frac{2}{5}=\\frac{16}{5625}\"

The last subtotal is the complete product. Only now convert the fraction to a decimal.

\"P=\\frac{16}{5625}\\approx0.0028444444\"

Final answer

P = 16/5625 ≈ 0.0028444444 for these words and specified tags jointly. The two occurrences of eats give P(eats | Verb) = 2/3.

Yaad rakho: Same tag structure ho sakta hai, lekin word emission counts dobara count karna.

Boundary reminder: Include P(Det | Start) = 2/3 and P(End | Noun) = 2/5. The handwritten key’s final line instead inserts the overall Det fraction 2/10 and omits End; the complete boundary-aware probability requested here is 0.0028444444.

","m2":true,"units":"4","marks":10}]}],"mocks":[{"id":"mock-a","title":"M2 Mock A · core calculations","unit":"Practice","sections":[{"id":"instructions","title":"Paper instructions","html":"

Practice paper · 10 marks · 60 minutes. Q1 and Q2 are compulsory (2 marks each). Attempt any two of Q3, Q4 and Q5 (3 marks each). All five solutions are included below their questions. These are newly authored practice questions, not predictions of the actual paper.

Use about 8 minutes each for Q1–Q2, 16 minutes each for the two chosen 3-mark questions, and the remaining 12 minutes to read, choose and check. Write the formula and substitutions even if a final calculation gets stuck.

"},{"id":"q1","title":"Q1 (2 marks)","html":"

Q1

For “we study language models”, list the unigrams, bigrams and trigrams. Do not add boundary markers. (2 marks)

Given

Four tokens in the stated order.

To find

All contiguous n-grams for n=1,2,3.

Formula and why it applies

\"\\#\\text{

T is the number of tokens. Slide a window of n consecutive tokens, one position at a time.

Solution

Step 1. Unigrams, window size 1

(we), (study), (language), (models); count=4.

Step 2. Bigrams, window size 2

(we study), (study language), (language models); count=4−2+1=3.

Step 3. Trigrams, window size 3

(we study language), (study language models); count=4−3+1=2.

4 unigrams, 3 bigrams and 2 trigrams, listed above.
"},{"id":"q2","title":"Q2 (2 marks)","html":"

Q2

A corpus has C(read)=10, C(read books)=6 and C(read papers)=4. Find both bigram conditional probabilities. (2 marks)

Given

read has 10 continuation opportunities; 6 are books and 4 are papers.

To find

P(books | read) and P(papers | read).

Formula and why it applies

\"P(w\\mid

h is the preceding word. Divide each pair count by the count of its shared history.

Solution

Step 1. Probability of books

\"P(\\text{books}\\mid\\text{read})=\\frac6{10}=0.6\"

Step 2. Probability of papers

\"P(\\text{papers}\\mid\\text{read})=\\frac4{10}=0.4\"

Step 3. Check the complete row

\"0.6+0.4=1\"
P(books | read)=0.6 and P(papers | read)=0.4.
"},{"id":"q3","title":"Q3 (3 marks)","html":"

Q3

Training corpus: “I like tea” and “I like coffee”. Lowercase all words and add <s> and </s> to each sentence. Using add-one bigram smoothing and a vocabulary containing both boundary symbols, find P(“i like tea”). (3 marks)

Given

Padded corpus: <s> i like tea </s>; <s> i like coffee </s>. V={<s>, i, like, tea, coffee, </s>}, |V|=6.

To find

Product of the four smoothed transition probabilities, including the end transition.

Formula and why it applies

\"P_{+1}(w\\mid

Every possible vocabulary continuation gets one pseudocount. Both boundaries are counted in V because the question explicitly requires the course convention.

Solution

Step 1. Count histories and pairs

TransitionPair countHistory count
<s> → i22
i → like22
like → tea12
tea → </s>11

Step 2. Compute first probability

\"P(i\\mid\\langle

Step 3. Compute middle probabilities

\"P(\\text{like}\\mid
\"P(\\text{tea}\\mid\\text{like})=\\frac{1+1}{2+6}=\\frac14\"

Step 4. Compute end probability

\"P(\\langle/

Step 5. Multiply

\"P=\\frac38\\times\\frac38\\times\\frac14\\times\\frac27=\\frac9{896}\\approx0.010044643\"
Sentence probability = 9/896 ≈ 0.010044643.
"},{"id":"q4","title":"Q4 (3 marks)","html":"

Q4

An HMM has P(N|start)=0.6, P(V|N)=0.5, P(end|V)=0.4, P(dogs|N)=0.5 and P(run|V)=0.8. Find the joint probability of words “dogs run” with tag path N,V, including the end transition. (3 marks)

Given

The five required transition/emission factors are supplied.

To find

P(dogs run, N V, end).

Formula and why it applies

\"P(w,t)=P(t_1\\mid

Transitions move between tags. Emissions generate words from tags. Include both types of factor.

Solution

Step 1. Place the factors along the path

start → N: 0.6
→
emit dogs: 0.5
→
N → V: 0.5
→
emit run: 0.8
→
V → end: 0.4

Step 2. First word contribution

\"0.6\\times0.5=0.3\"

Step 3. Enter V and emit run

\"0.3\\times0.5=0.15\"
\"0.15\\times0.8=0.12\"

Step 4. End the sentence

\"0.12\\times0.4=0.048\"
Joint path-and-word probability = 0.048. This is not a normalised posterior over tag paths.
"},{"id":"q5","title":"Q5 (3 marks)","html":"

Q5

For a two-sense Naive Bayes WSD model, P(F)=P(R)=0.5. After smoothing, P(loan|F)=0.4, P(money|F)=0.3, P(loan|R)=0.1 and P(money|R)=0.1. Classify “bank” using context {loan, money} and obtain the winning sense’s posterior. (3 marks)

Given

F=financial sense; R=river sense. Two exhaustive senses; conditional independence of the context words is assumed.

To find

Two scores, selected sense and normalised posterior.

Formula and why it applies

\"s(c)=P(c)P(\\text{loan}\\mid
\"P(c\\mid

Multiply the prior by the two likelihoods, then normalise because a posterior is requested.

Solution

Step 1. Financial score

\"s(F)=0.5\\times0.4\\times0.3=0.06\"

Step 2. River score

\"s(R)=0.5\\times0.1\\times0.1=0.005\"

Step 3. Total evidence score

\"0.06+0.005=0.065\"

Step 4. Normalise winner

\"P(F\\mid
Financial sense; posterior ≈0.923077.
"}],"scope":"M2 · confirmed 2 + 2 + any two of 3 + 3 + 3 · worked practice"},{"id":"mock-b","title":"M2 Mock B · unseen pairs & decoding","unit":"Practice","sections":[{"id":"instructions","title":"Paper instructions","html":"

Practice paper · 10 marks · 60 minutes. Q1 and Q2 are compulsory (2 marks each). Attempt any two of Q3, Q4 and Q5 (3 marks each). All five solutions are included below their questions. These are newly authored practice questions, not predictions of the actual paper.

Use about 8 minutes each for Q1–Q2, 16 minutes each for the two chosen 3-mark questions, and the remaining 12 minutes to read, choose and check. Write the formula and substitutions even if a final calculation gets stuck.

"},{"id":"q1","title":"Q1 (2 marks)","html":"

Q1

A language model gives a test sequence probability of 0.0016 across four predicted tokens. Calculate perplexity. (2 marks)

Given

P=0.0016; N=4.

To find

PP.

Formula and why it applies

\"PP=P^{-1/N}=\\left(\\frac1P\\right)^{1/N}\"

Use the number of predicted events, not an unrelated training token count.

Solution

Step 1. Reciprocal

\"\\frac1{0.0016}=625\"

Step 2. Fourth root

\"PP=625^{1/4}=5\"
Perplexity=5.
"},{"id":"q2","title":"Q2 (2 marks)","html":"

Q2

A first-order Markov model starts in state A with probability 0.7. P(B|A)=0.4 and P(A|B)=0.3. Compute the path probability A→B→A, with no end-state factor. (2 marks)

Given

Start A=0.7; A→B=0.4; B→A=0.3.

To find

Probability of this state path.

Formula and why it applies

\"P(A,B,A)=P(A)P(B\\mid

First-order dependence uses only the previous state for each transition.

Solution

Step 1. First transition contribution

\"0.7\\times0.4=0.28\"

Step 2. Second transition contribution

\"0.28\\times0.3=0.084\"
Path probability=0.084.
"},{"id":"q3","title":"Q3 (3 marks)","html":"

Q3

For a fixed vocabulary of five possible continuations, C(very)=7 and C(very bright)=0. Calculate the unsmoothed and Laplace-smoothed probabilities of bright after very. (3 marks)

Given

History count=7; pair count=0; |V|=5.

To find

MLE and add-one probabilities.

Formula and why it applies

\"P_{MLE}(w\\mid
\"P_{+1}(w\\mid

MLE uses only observations. Laplace reserves probability for all known vocabulary outcomes, including unseen pairs.

Solution

Step 1. Unsmoothed estimate

\"P_{MLE}(\\text{bright}\\mid\\text{very})=\\frac07=0\"

Step 2. Add one to the pair count

\"0+1=1\"

Step 3. Add vocabulary size to the history count

\"7+5=12\"

Step 4. Divide

\"P_{+1}(\\text{bright}\\mid\\text{very})=\\frac1{12}\\approx0.083333\"
MLE=0; Laplace=1/12≈0.083333. The zero count means unseen in this corpus, not impossible language.
"},{"id":"q4","title":"Q4 (3 marks)","html":"

Q4

A two-state HMM has initial probabilities P(N)=0.6, P(V)=0.4. Transition rows (N→N,N→V)=(0.3,0.7) and (V→N,V→V)=(0.8,0.2). Emissions are P(fish|N)=0.5, P(fish|V)=0.4, P(swim|N)=0.1 and P(swim|V)=0.6. Find the best tag path for “fish swim”, without end transitions. (3 marks)

Given

Two states N,V; all start, transition and relevant emission probabilities are supplied.

To find

Largest joint score and its tag sequence.

Formula and why it applies

\"\\delta_1(t)=P(t)P(w_1\\mid
\"\\delta_2(t)=P(w_2\\mid

Viterbi keeps the best path ending at each tag and stores its predecessor. Enumerating four paths gives the same result for this tiny example.

Solution

Step 1. Initialise fish

\"\\delta_1(N)=0.6\\times0.5=0.3\"
\"\\delta_1(V)=0.4\\times0.4=0.16\"

Step 2. Score both paths ending in N

\"NN:0.3\\times0.3\\times0.1=0.009\"
\"VN:0.16\\times0.8\\times0.1=0.0128\"

Keep VN at N; predecessor V.

Step 3. Score both paths ending in V

\"NV:0.3\\times0.7\\times0.6=0.126\"
\"VV:0.16\\times0.2\\times0.6=0.0192\"

Keep NV at V; predecessor N.

Step 4. Choose and backtrack

0.126 > 0.0128. End at V and follow its stored predecessor N.

Best sequence: N,V, with joint score 0.126.
"},{"id":"q5","title":"Q5 (3 marks)","html":"

Q5

Apply simplified Lesk to “bank” with preprocessed context {river, water, flooded}. Gloss R is {sloping, land, river, water}; gloss F is {financial, institution, money, loan}. (3 marks)

Given

Two already preprocessed gloss sets and one context set.

To find

Overlap scores and selected sense.

Formula and why it applies

\"\\text{score}(s)=|\\text{context}\\cap\\text{gloss}(s)|\"

Simplified Lesk chooses the sense whose gloss shares the largest number of context words, using the specified set-overlap convention.

Solution

Step 1. Intersect the river gloss

{river, water, flooded} ∩ {sloping, land, river, water} = {river, water}.

Step 2. Count the river overlap

\"score(R)=2\"

Step 3. Intersect and count the financial gloss

No shared words; intersection is empty, so score(F)=0.

Step 4. Compare

2 > 0, hence select the river-bank sense.

River-bank sense, with overlap 2 versus 0.
"}],"scope":"M2 · confirmed 2 + 2 + any two of 3 + 3 + 3 · worked practice"},{"id":"mock-c","title":"M2 Mock C · mixed numerical practice","unit":"Practice","sections":[{"id":"instructions","title":"Paper instructions","html":"

Practice paper · 10 marks · 60 minutes. Q1 and Q2 are compulsory (2 marks each). Attempt any two of Q3, Q4 and Q5 (3 marks each). All five solutions are included below their questions. These are newly authored practice questions, not predictions of the actual paper.

Use about 8 minutes each for Q1–Q2, 16 minutes each for the two chosen 3-mark questions, and the remaining 12 minutes to read, choose and check. Write the formula and substitutions even if a final calculation gets stuck.

"},{"id":"q1","title":"Q1 (2 marks)","html":"

Q1

A corpus contains five occurrences tagged Noun. Three of them are “cat”. It contains four Verb tokens, one of which is “cat”. Calculate P(cat|Noun) and P(cat|Verb). (2 marks)

Given

C(N)=5; C(cat,N)=3; C(V)=4; C(cat,V)=1.

To find

The two emission probabilities.

Formula and why it applies

\"P(w\\mid

An emission conditions on the tag, so the denominator counts tokens of that tag.

Solution

Step 1. Noun emission

\"P(\\text{cat}\\mid

Step 2. Verb emission

\"P(\\text{cat}\\mid
P(cat|N)=0.6 and P(cat|V)=0.25.
"},{"id":"q2","title":"Q2 (2 marks)","html":"

Q2

Each of five words may receive any of three POS tags. How many possible tag sequences exist before applying constraints? (2 marks)

Given

T=5 token positions; K=3 choices at each position.

To find

Total tag sequences.

Formula and why it applies

\"K^T\"

Choose one tag independently for each position when counting candidates; multiply the number of options five times.

Solution

Step 1. Expand the power

\"3^5=3\\times3\\times3\\times3\\times3\"

Step 2. Multiply

\"3\\times3=9,\\quad9\\times3=27\"
\"27\\times3=81,\\quad81\\times3=243\"
243 tag sequences.
"},{"id":"q3","title":"Q3 (3 marks)","html":"

Q3

For trigram prediction, C(i like)=5, C(i like tea)=3 and C(i like coffee)=2. Find the MLE probability of each continuation. Then find add-one P(tea|i like) using a continuation vocabulary of size 4. (3 marks)

Given

Two-word history i like occurs 5 times; continuations tea=3, coffee=2; |V|=4.

To find

Two MLE values and the smoothed tea probability.

Formula and why it applies

\"P(w_i\\mid
\"P_{+1}(w\\mid

A trigram conditions on two preceding words. The same add-one count rule applies to that longer history.

Solution

Step 1. Tea MLE

\"P(\\text{tea}\\mid

Step 2. Coffee MLE

\"P(\\text{coffee}\\mid

Step 3. Smooth tea

\"P_{+1}(\\text{tea}\\mid
MLE tea=0.6; coffee=0.4; smoothed tea=4/9≈0.444444.
"},{"id":"q4","title":"Q4 (3 marks)","html":"

Q4

Training sentences with tags are: “the/D cat/N sleeps/V” and “a/D dog/N runs/V”. Estimate start/end-inclusive transition probabilities and the joint probability of “the cat sleeps” with path D,N,V. (3 marks)

Given

Two paths start→D→N→V→end. Four distinct words the,a,cat,dog and two verbs sleeps,runs. Use unsmoothed estimates and lowercase tokens.

To find

Transitions, required emissions and full path score.

Formula and why it applies

\"P(t_j\\mid
\"P(w\\mid

Transition denominators count outgoing transitions from the previous state. Emission denominators count all tokens tagged with the emitting state.

Solution

Step 1. Count transitions

start→D, D→N, N→V and V→end each occur twice. Each state has just one observed next state.

Step 2. Divide to get probabilities

\"P(D\\mid

Step 3. Estimate emissions

\"P(\\text{the}\\mid
\"P(\\text{cat}\\mid
\"P(\\text{sleeps}\\mid

Step 4. Multiply in word order

\"P=1\\times\\frac12\\times1\\times\\frac12\\times1\\times\\frac12\\times1=\\frac18=0.125\"
Joint probability=0.125. Each of the four observed transition probabilities is 1.
"},{"id":"q5","title":"Q5 (3 marks)","html":"

Q5

In a WSD training set, financial bank and river bank each have two labelled contexts. Token totals are N(F)=6 and N(R)=6. Vocabulary size is 8. Counts are C(loan,F)=2, C(loan,R)=0, C(water,F)=0, C(water,R)=3. Use add-one Naive Bayes to classify context {loan,water}. (3 marks)

Given

P(F)=P(R)=2/4=0.5; |V|=8; both class token totals=6.

To find

Smoothed scores and winning sense.

Formula and why it applies

\"P(w\\mid
\"s(c)=P(c)\\prod_wP(w\\mid

Class priors use labelled-context counts; word likelihood denominators use class token totals. Do not exchange those two totals.

Solution

Step 1. Find the denominator

\"6+8=14\"

Step 2. Financial likelihoods

\"P(loan\\mid

Step 3. River likelihoods

\"P(loan\\mid

Step 4. Compute financial score

\"s(F)=\\frac12\\times\\frac3{14}\\times\\frac1{14}=\\frac3{392}\\approx0.007653061\"

Step 5. Compute river score

\"s(R)=\\frac12\\times\\frac1{14}\\times\\frac4{14}=\\frac4{392}\\approx0.010204082\"

Step 6. Choose

4/392 > 3/392; the water evidence outweighs the loan evidence under these counts.

River sense wins. If normalised, P(R|context)=4/7≈0.571429.
"}],"scope":"M2 · confirmed 2 + 2 + any two of 3 + 3 + 3 · worked practice"}],"revision":[{"id":"m2-revision","title":"M2 · last-day revision","unit":0,"sections":[{"id":"order","title":"Last-day reading order","html":"

Start with the families most likely to require arithmetic: smoothed bigrams, HMM probabilities and Naive Bayes WSD. Then revise perplexity, n-gram counts and Lesk. Parsing, POS approaches, WordNet and lexical relations remain in the confirmed portion; do not remove them merely because the expected paper is mostly numerical.

Time blockRead and workWhat your written solution should contain
First 60–90 minN-grams, MLE and Laplace examplesPadded corpus → vocabulary → count table → each fraction → full product.
Next 60–90 minHMM transition/emission and path examplesTag totals → transition counts → matrices → alternating transition/emission factors.
Next 60–90 minWSD and Lesk examplesClass priors → vocabulary → class totals → smoothed word likelihoods → score comparison.
Next 45–60 minPerplexity, POS, parsing, WordNet, NERDefinitions, one meaningful example, a tree or table where needed.
Final 60 minOne M2 mock under exam timeBoth 2-mark questions and any two 3-mark questions; compare every step with the answers.

The writing format to repeat

Given
→
To find
→
Formula + meaning
→
Numbered calculations
→
Answer

If the question supplies the corpus, draw the count table before using the calculator. A long product should be built from fractions that you have already justified. Keep fractions or stored calculator values until the end.

"},{"id":"method","title":"Recognise the question and start correctly","html":"
Question clueUse this methodFirst thing to write
“next word”, “sentence probability”N-gram modelList the needed word transitions.
“unseen pair”, “Laplace”Add-one smoothingState V and count each history.
“perplexity”Reciprocal Nth rootState P(test sequence) and N.
“transition/emission matrix”HMM count estimationSeparate tag-to-tag counts from tag-to-word counts.
“given tag sequence”HMM joint pathInitial × emission × transition × emission …
“best tag sequence”Viterbi or enumerate tiny pathsInitialise one score per tag; retain predecessor maxima.
“gloss overlap”Simplified LeskPreprocess context and glosses consistently.
“classify word sense with counts”Naive Bayes WSDPriors from examples, likelihoods from tokens.
“parse tree”Given CFG + chosen parserShow grammar expansions or stack reductions.

Errors that change the answer

1. Dividing a conditional count by the whole corpus instead of its history count. 2. Adding 1 to the numerator without adding |V| to the denominator. 3. Forgetting the end transition. 4. Counting repeated words as new vocabulary types. 5. Using P(tag|word) in place of P(word|tag). 6. Calling an unnormalised score a posterior. 7. Silently mixing case, lemmatisation or boundary conventions.

Short theory answer format

For 2 marks: give the exact definition and one specific example or distinction. For 3 marks: add the mechanism or two ordered steps; include a small diagram/table where it carries information. The actual allocation is examiner-dependent, so these are writing guides, not an official marking scheme.

"}],"scope":"M2 · quick revision after completing the lessons"}],"about":[{"id":"scope","title":"Portion, paper pattern & sources","unit":0,"sections":[{"id":"sources","title":"What this NLP guide covers","html":"

M2 portion confirmed by you: Unit 3 from PDF slide 62 to 160; Unit 4 complete (128 slides); Unit 5 slides 1–57. Slide 62 of Unit 3 starts N-grams. Slide 57 of Unit 5 completes the Naive Bayes WSD example. Vector-space models start on slide 58 and belong to TEE here.

M2 pattern confirmed by you: both 2-mark questions compulsory; any two of the three 3-mark questions. Total attempted 10. Course policy gives one hour. Three newly authored, fully solved M2 practice papers use that pattern.

TEE: all seven units of the official AY 2026–27 syllabus. The new ZIP supplies faculty slide packs for Units 1–5 plus a regular-expressions handout. Units 6–7 are syllabus-based explanations supplemented with the standard references below; their faculty slide packs were not supplied. The official syllabus includes GloVe and spelling correction, and both are retained even though the policy’s lecture table abbreviates them.

Previous papers found: final exam dated 2 December 2025 and re-exam dated 9 February 2026, with answer keys, inside Semester V.zip. Programme copies were deduplicated. Every supplied question subpart is mapped to the current syllabus; no question was excluded as out of syllabus. Only wholly M2-eligible subparts appear in the M2 paper tab; mixed questions remain in the complete TEE papers.

The current syllabus lists course code 702AI0C037. The supplied policy and old papers use 702AI0C009; the policy also retains an older programme/semester header. Topic coverage follows the current official syllabus and your exact M2 cutoff, rather than treating the old header as a different course.

Conventions that must stay visible

Course bigram answer keys count both start and end symbols in vocabulary size. Worked PYQs state and follow that convention; other exercises state their own vocabulary explicitly. A start symbol is conditioning context, so alternative textbook conventions may exclude it from prediction vocabulary and produce a different smoothed number. HMM path scores and Naive Bayes class scores are labelled as unnormalised when applicable. Any normalisation, case-folding or lemmatisation assumption is stated before counting.

Reading references

Examples are teaching problems unless explicitly identified by their paper and question number. Practice-paper marks indicate intended answer length, not an official forecast. Reading alone cannot guarantee a score; writing the solutions independently is what makes the procedures usable under exam time.

TEE depth review · 10 October 2026

The core TEE route retains the faculty topics CYK/CKY, grammar conversion, interpolation and Stupid Backoff. Add-k and HMM forward/posterior calculations are available separately under Optional extensions; they are not part of the core reading or formula sequence. The two TEE timed-practice entries reproduce the supplied actual papers and are not additional unseen mock papers.

Method references: Jurafsky and Martin: Hidden Markov Models; Jurafsky and Martin: Constituency Parsing. New numerical inputs and worked traces are teaching examples.

"},{"id":"videos","title":"Video companions in your NotebookLM","html":"

Your M2 notebook contains the exact three slide ranges and checked numerical teaching examples. The requested video topics are N-grams and Laplace smoothing, HMM/Viterbi, and Naive Bayes WSD. Use the written solutions here for the complete calculations.

Open your NLP M2 notebook →

The notebook is in your signed-in Google account. Video generation and availability are shown in its Studio panel. Generated narration is a supplementary explanation; the numerical results in this guide have been independently checked.

"}],"scope":"NLP · M2 and TEE · edition 3 October 2026"}]},"tee":{"read":[{"id":"foundations","title":"Start here · language & probability","unit":0,"sections":[{"id":"language","title":"What NLP is doing","html":"

Natural Language Processing (NLP) is the study of computational methods for analysing, interpreting and producing human language. An NLP system receives text or speech and produces an output such as a tag, a meaning, a translation or a probability.

A central question in NLP is: how do counts from a training corpus become probabilities, and how do those probabilities help choose a sentence, a tag or a word sense? You will learn the vocabulary first, then do the arithmetic with complete working.

Training textexamples already observed
→
Countshow often things occur
→
Probabilitiescounts divided by totals
→
Decisioncompare possible answers
TermMeaningConcrete example
CorpusA collection of language data. The plural is corpora.Three sentences saved together form a tiny corpus.
DocumentOne text item within a collection.One review or one article.
TokenOne occurrence produced by tokenization.“cats like cats” has 3 word tokens.
TypeA distinct token value.The same text has 2 types: cats and like.
VocabularyThe set of token types recognised by the model.V = {cats, like}; vocabulary size is 2.
TrainingEstimating rules or numbers using examples.Count how often like follows cats.
TestApplying the learned model to a new example.Calculate a probability for a sentence not used to build the counts.

Three levels you will repeatedly see

LevelQuestion it answersExample
Word sequenceWhich word is likely next?After “drink”, is “water” likely?
SyntaxWhat role or structure does the word have?“book” is a noun in “a book” and a verb in “book tickets”.
SemanticsWhich meaning is intended?“bank” near “deposit” means a financial institution.

Yaad rakho: Word ka order = language model; word ka role = POS; word ka matlab = sense.

Your TEE route

Read these foundations, then continue through Units 1–7 in order. TEE includes preprocessing and vectors, corpora and regular expressions, language models, syntax, semantics, discourse and applications. The dedicated formula lessons and eight recovered previous papers follow the same methods. Original ambiguities are explained beside the affected answers.

Latest TEE pattern:100 marks in three hours; Q1 compulsory plus any four of Q2–Q7. Each full question carries20 marks. Historical December2023 ends at Q6; its choices are labelled separately. The M2 tab remains restricted to your confirmed midterm portion.

Open the complete paper-by-paper map, original marks and full answers →

Prepare the question forms seen in the latest papers first

The 2 December 2025 final and 9 February 2026 re-exam both combine theory with applied work. Use their complete questions in Solved papers; the two Timed papers are those actual papers, not new predictions.

Other faculty topics remain in the full syllabus route, but their authored exercises do not establish past-paper marks or frequency. Optional extensions can wait until the actual paper methods are secure.

"},{"id":"notation","title":"Tokens, counts and symbols","html":"

A formula is a short way of stating a calculation. First name the quantities; only then insert the numbers. In this guide, C means a count and P means a probability.

NotationRead it asWhat to do
\"C(\\text{cat})=3\"Count of cat is threeCount every occurrence, including repeats.
\"P(\\text{cat})\"Probability of catA number from 0 to 1.
\"P(\\text{cat}\\mid\\text{the})\"Probability of cat given theRestrict attention to occasions where the history is “the”.
\"\\sum_iSum of the x valuesAdd the terms: x₁ + x₂ + …
\"\\prod_iProduct of the x valuesMultiply the terms: x₁ × x₂ × …
\"\\arg\\max_cThe choice c with the largest scoreReturn a label, not the numeric score.
\"\\max_cLargest scoreReturn a number.

Example 1 · tokens and types

For the corpus “we study nlp” and “we like nlp”, find the token count, vocabulary and count of “we”.

Given

Two lowercase sentences. Ignore punctuation; do not add boundary markers.

To find

Token count N, vocabulary V, vocabulary size |V| and C(we).

Formula and why it applies

A token is an occurrence; a type is a distinct word. Add sentence lengths to count tokens; remove duplicates only when finding the vocabulary.

Solution

Step 1. List every occurrence

we, study, nlp, we, like, nlp.

Step 2. Count tokens

\"N=3+3=6\"

Step 3. Keep one copy of each type

V = {we, study, nlp, like}; |V| = 4.

Step 4. Count the requested word

“we” appears once in each sentence, hence C(we) = 2.

N = 6; |V| = 4; C(we) = 2.

Yaad rakho: Token mein repeat gin; type mein duplicate hata.

Example 2 · sum and product

Given probabilities 0.5, 0.2 and 0.1, compute their sum and their product.

Given

Three numbers: 0.5, 0.2, 0.1.

To find

Sum and product; recognise that these are different operations.

Formula and why it applies

\"\\sum_{i=1}^{3}p_i=p_1+p_2+p_3\"
\"\\prod_{i=1}^{3}p_i=p_1p_2p_3\"

Add for a sum. Multiply for a product. A sentence-chain calculation normally uses the product of its conditional probabilities.

Solution

Step 1. Add the first two numbers

\"0.5+0.2=0.7\"

Step 2. Complete the sum

\"0.7+0.1=0.8\"

Step 3. Multiply the first two numbers

\"0.5\\times0.2=0.1\"

Step 4. Complete the product

\"0.1\\times0.1=0.01\"
Sum = 0.8; product = 0.01.
"},{"id":"probability","title":"Probability, without skipped steps","html":"

Empirical probability estimates how likely an event is from its observed frequency. Divide the number of relevant observations by the total number of observations in the selected group. For a conditional probability, the denominator is the count of the condition, not the size of the entire corpus.

\"P(A)=\\frac{C(A)}{N}\"
\"P(B\\mid

Here A is the conditioning event, B is the event being predicted, C(A,B) counts observations containing both in the required relationship, and N is the total number of observations. “Given A” tells you which denominator to use.

Select the condition“given A”
→
Count matching outcomesC(A,B)
→
Divide by condition countC(A)

Example 1 · word probability

A corpus contains 20 tokens; “rain” occurs 5 times. Find the unigram probability of “rain”.

Given

N = 20 and C(rain) = 5.

To find

P(rain).

Formula and why it applies

\"P(w)=\\frac{C(w)}{N}\"

Unigram probability ignores neighbouring words. Every token in the corpus is one observation.

Solution

Step 1. Insert the count and total

\"P(\\text{rain})=\\frac{5}{20}\"

Step 2. Divide

\"\\frac{5}{20}=0.25\"
P(rain) = 0.25, or 25%.

Example 2 · conditional probability

In a corpus, “drink” is followed by another word 8 times. It is followed by “water” 6 times. Find P(water | drink).

Given

C(drink as a history) = 8; C(drink, water) = 6.

To find

Probability that the next word is water, given drink.

Formula and why it applies

\"P(\\text{water}\\mid\\text{drink})=\\frac{C(\\text{drink

The condition is drink, so use the 8 drink-histories as the denominator.

Solution

Step 1. Identify the favourable continuations

Six of the eight continuations are water.

Step 2. Divide

\"P(\\text{water}\\mid\\text{drink})=\\frac68=0.75\"
P(water | drink) = 0.75.

Yaad rakho: Bar ke right mein jo hai, denominator uska count.

Example 3 · multiply a sequence

A model gives P(a | start) = 0.4, P(cat | a) = 0.5 and P(end | cat) = 0.2. Find the probability of the complete sentence “a cat”.

Given

Start → a = 0.4; a → cat = 0.5; cat → end = 0.2.

To find

Probability of the three-transition path.

Formula and why it applies

\"P(\\text{a

Every transition must happen in sequence, so multiply the model’s conditional probabilities.

Solution

Step 1. Write the three factors in order

\"P=0.4\\times0.5\\times0.2\"

Step 2. Multiply the first pair

\"0.4\\times0.5=0.2\"

Step 3. Multiply by the last factor

\"0.2\\times0.2=0.04\"
Complete-sentence probability = 0.04.

Example 4 · compare labels

Two tag candidates have scores noun = 0.036 and verb = 0.048. Which tag does argmax return? If these are the only candidates, find the normalised noun probability.

Given

s(noun) = 0.036 and s(verb) = 0.048.

To find

Winning label and normalised noun probability.

Formula and why it applies

\"\\hat
\"P(c)=\\frac{s(c)}{\\sum_j

A score need not sum to 1. Dividing each score by the sum gives a distribution over the candidates.

Solution

Step 1. Compare scores

0.048 > 0.036, so verb wins.

Step 2. Find their total

\"0.036+0.048=0.084\"

Step 3. Divide the noun score by the total

\"P(\\text{noun})=\\frac{0.036}{0.084}=\\frac37\\approx0.428571\"
argmax returns “verb”. The normalised noun probability is approximately 0.428571.
"},{"id":"bayes","title":"Bayes’ rule: prior, evidence and answer","html":"

Bayes’ rule reverses a conditional probability. A word-sense classifier asks “which sense is most likely given these context words?” Training data more easily supplies “how likely are these words when this sense is used?” Bayes connects the two.

\"P(c\\mid
SymbolMeaning
cA candidate class, tag or word sense.
xThe observed evidence, such as a context word.
P(c)Prior: how common the class is before seeing the evidence.
P(x | c)Likelihood: how compatible the evidence is with that class.
P(c | x)Posterior: probability of the class after observing the evidence.
P(x)Evidence normaliser, shared by all classes for the same observation.

For ranking classes, calculate prior × likelihood. The common denominator does not affect which class is largest. For an actual posterior probability, divide by the sum of all class scores.

A complete Bayes calculation

A word has two senses: financial bank and river bank. Priors are 0.6 and 0.4. The probability of observing “loan” is 0.5 under the financial sense and 0.1 under the river sense. Find the most likely sense and its posterior.

Given

P(F) = 0.6; P(R) = 0.4; P(loan | F) = 0.5; P(loan | R) = 0.1. These two senses are exhaustive.

To find

Scores and P(F | loan).

Formula and why it applies

\"s(c)=P(c)P(\\text{loan}\\mid
\"P(F\\mid\\text{loan})=\\frac{s(F)}{s(F)+s(R)}\"

Multiply prior and likelihood to reverse the conditioning. Then normalise because a posterior was requested.

Solution

Step 1. Score the financial sense

\"s(F)=0.6\\times0.5=0.30\"

Step 2. Score the river sense

\"s(R)=0.4\\times0.1=0.04\"

Step 3. Add the scores

\"P(\\text{loan})=0.30+0.04=0.34\"

Step 4. Normalise the financial score

\"P(F\\mid\\text{loan})=\\frac{0.30}{0.34}=\\frac{15}{17}\\approx0.882353\"
Financial bank wins; its posterior probability is about 0.882353.

Yaad rakho: Pehle se kitna common × evidence kitna suit karta = score.

"},{"id":"calculator","title":"Small probabilities, logs and calculator use","html":"

Repeated multiplication of probabilities produces small numbers. A calculator may show 0.0000125 as 1.25 × 10⁻⁵; they are the same value. A logarithm turns a product into a sum. Log probabilities are often negative because probabilities between 0 and 1 have negative logs.

\"\\log(ab)=\\log
\"\\log_b

Use the log base specified in the question. Natural log is ln (base e); ordinary calculator log is usually base 10. Perplexity is simplest to evaluate using the direct power formula. Do not mix bases in the same calculation.

Calculator example · roots in perplexity

A model assigns probability 1/16 to a test sequence with 4 predicted tokens. Calculate perplexity.

Given

P = 1/16; N = 4 predicted tokens.

To find

Perplexity PP.

Formula and why it applies

\"PP=P^{-1/N}=\\left(\\frac1P\\right)^{1/N}\"

Perplexity takes the reciprocal probability and then its Nth root. N must match the number of predicted events in the product.

Solution

Step 1. Take the reciprocal

\"\\frac{1}{1/16}=16\"

Step 2. Take the fourth root

\"PP=16^{1/4}=2\"

Step 3. Interpret

A perplexity of 2 means an effective average uncertainty comparable to two equally likely alternatives. It is not 2% error.

PP = 2.

Calculator entry order

Enter the full product with parentheses. For the Nth root enter (1 ÷ P)^(1 ÷ N). Keep at least 6 significant digits while working; round the final answer. If your calculator offers Ans, reuse the stored value instead of typing a heavily rounded result.

Yaad rakho: Probability ke factors multiply; log mein add; perplexity mein ulta karke root.

"}],"scope":"Shared language and probability foundations · TEE"},{"id":"u1-introduction","title":"NLP foundations: language, tasks and the processing pipeline","unit":1,"mode":"tee","scope":"TEE · Unit 1 full","sections":[{"id":"u1-nlp","title":"What natural language processing actually does","html":"

Natural language processing (NLP) is the field concerned with computational methods for analysing, interpreting and generating human language. Natural languages such as English, Hindi and Marathi evolved through human use; their meanings are not specified by the rigid rules of a programming language.

An NLP system may receive text, speech converted to text, or another linguistic representation. Its output depends on the task: a category, a tag for every word, a translated sentence, a ranked document list or a generated answer. It does not always “understand” in the human sense; its performance must be evaluated for the specific task.

TaskInputOutput
Sentiment analysis“The service was excellent.”Positive sentiment
Text classificationAn emailSpam / not spam
POS tagging“Birds fly.”Birds/NOUN fly/VERB
Named entity recognition“Asha joined Google in Mumbai.”Asha: person; Google: organisation; Mumbai: location
Machine translationA Hindi sentenceA corresponding English sentence
Information retrievalA search queryRanked relevant documents
Question answeringQuestion plus knowledge/contextAn answer
SummarizationA long articleA shorter text preserving key information

How NLP relates to AI, ML and DL

AI is the broader study of systems performing tasks associated with intelligent behaviour. Machine learning learns patterns from data. Deep learning uses multiple trainable neural layers. NLP is an application/research field that can use hand-written rules, statistical ML or deep learning. Therefore, not every NLP system is a neural network.

Linguistics supplies knowledge about language structure. Statistics supplies uncertainty models. Computer science supplies algorithms, storage and efficient implementation. A practical NLP system combines these ideas.

NLP = human language ko computational task mein badlo: label, meaning, search ya generated sentence.

"},{"id":"u1-language","title":"Building blocks and levels of linguistic analysis","html":"

A phoneme is a sound unit that can distinguish meanings in a language, such as the /p/ and /b/ contrast in “pat” and “bat”. A phoneme is a sound, not necessarily one alphabet letter. A morpheme is the smallest meaning-bearing grammatical unit: cats = cat + plural -s; unhappy = un- + happy.

A lexeme is an abstract vocabulary item grouping its inflected forms: write, writes, wrote and writing belong to WRITE. A phrase is a group of words functioning together; a clause normally contains a predicate and its associated participants; a sentence expresses a complete grammatical unit.

LevelWhat it analysesConcrete example
Phonetics / phonologyPhysical sounds / sound patterns in a languageDistinguish bat from pat
Morphology / lexical analysisWord structure and lexical itemsun + help + ful → unhelpful
SyntaxGrammatical arrangement and relationshipsThe dog chased the cat: subject, verb, object
SemanticsMeaning of words and composed expressionsThe dog is the chaser; the cat is chased
DiscourseConnections across sentences“Riya arrived. She smiled.” → She refers to Riya
PragmaticsIntended meaning in the situation“It is cold here” may request closing the window
word forms
→
grammatical structure
→
sentence meaning
→
cross-sentence context
→
speaker intention

Three worked interpretations

“The cats are sleeping.” Morphology identifies cat + s as plural. Syntax connects “cats” to “are sleeping”. Semantics represents animals engaged in sleeping.

“Aman bought a laptop. It was expensive.” Sentence-level syntax alone does not identify “it”. Discourse connects “it” to the laptop across the sentence boundary.

“Wonderful, another power cut!” The word wonderful is positive literally, but situational context can indicate sarcasm and frustration. This is a pragmatic interpretation.

The analysis levels are conceptual descriptions. Modern systems may learn several jointly rather than executing a compulsory separate module for every level.

Morphology = word ke andar; syntax = sentence ka structure; semantics = meaning; discourse = sentences ka link; pragmatics = intention.

"},{"id":"u1-approaches","title":"Rule-based, statistical and deep-learning approaches","html":"

A rule-based system applies explicit patterns, dictionaries or linguistic rules written by people. Example: mark an email as a candidate company message when its sender matches a known domain. Rules are transparent but can be brittle under paraphrasing.

A statistical/ML system estimates useful patterns from examples. A supervised classifier learns from texts with known labels. Unsupervised learning finds structure without those labels, such as clusters of news articles. A deep-learning system learns layered representations and can model context using neural architectures.

CriterionRule basedStatistical MLDeep learning
Main knowledge sourceExpert-written rulesFeatures and training examplesData and learned representations
ExampleRegex + sentiment lexiconTF–IDF + Naive Bayes/SVMTransformer text classifier
Typical benefitInterpretable, little training dataEffective with suitable featuresLearns complex context
Typical limitationRule maintenance; exceptionsFeature dependence; data mismatchCompute/data needs; harder interpretation

Understanding the learning terms

A training example is one input instance. A label is its desired output. A feature is an input property used by the model. Training estimates parameters; inference applies fixed learned parameters to new input. Separate training, validation and test sets prevent evaluation from merely measuring memorization of examples already seen.

Supervised example: review → positive/negative, using labelled reviews. Unsupervised example: cluster documents by word usage without topic labels. Reinforcement learning: choose actions using feedback or reward over interactions; it is not the same as training an ordinary classifier from labelled text.

Rules: hum likhte hain. ML: examples se pattern. DL: representation bhi layers se seekhta hai.

"},{"id":"u1-nlu-nlg","title":"NLU versus NLG, explained through one assistant","html":"

Natural language understanding (NLU) maps language into an interpretation useful for a task: intents, entities, semantic relations or another meaning representation. Natural language generation (NLG) produces human-language output from data, a representation or another source.

“Book a ticket to Pune tomorrow.”
→
NLU: intent=book; destination=Pune; date=tomorrow
→
Application: search available journeys
→
NLG: “Here are the available journeys to Pune.”
FeatureNLUNLG
DirectionLanguage → interpretationData/representation → language
Main concernMeaning, intent, ambiguityContent choice, organization, fluent wording
ExampleFind destination in a requestWrite the booking confirmation
Common failureWrong destination or senseUnclear, unsupported or ungrammatical output

NLG stages in a classical pipeline

  1. Content determination: choose facts to communicate.
  2. Data interpretation: identify relationships or trends.
  3. Document planning: organize facts in a useful order.
  4. Sentence aggregation: combine related facts without repetition.
  5. Lexical and grammatical realization: choose words and construct grammatical sentences.
  6. Presentation: format the generated message for its audience.

For a weather report, facts {city=Pune, temperature=28°C, rain=yes} can become “Pune will have rain with a temperature of 28°C.” NLU would instead extract facts or intent from an incoming language message.

NLU = language se meaning andar. NLG = data se language bahar.

"},{"id":"u1-pipeline","title":"Generic NLP pipeline, evaluation and deployment","html":"

An NLP pipeline is an organized sequence of operations that transforms raw language into a task output and checks whether that output is useful. Not every task needs every cleaning operation.

Data acquisition
→
Preprocessing
→
Feature representation
→
Model
→
Evaluation
→
Deployment and monitoring
  1. Data acquisition: collect relevant text and, when needed, labels.
  2. Cleaning and preprocessing: handle markup, tokenize, normalize and decide which information to retain.
  3. Feature engineering: represent text numerically using BoW, TF–IDF or embeddings.
  4. Modelling: apply rules or train a statistical/neural model.
  5. Evaluation: test on held-out data using task-appropriate measures.
  6. Deployment: make the system available, log suitable feedback and monitor errors or data changes.

Concrete spam-filter example

Collect labelled emails → separate train/test data → tokenize text while preserving meaningful terms → fit vocabulary and IDF on training data → train classifier → evaluate spam detection on the held-out set → deploy an email service filter → monitor legitimate emails wrongly blocked.

Intrinsic evaluation measures model/task quality, such as precision, recall or tagging accuracy. Extrinsic evaluation measures effect in the wider application, such as whether users find answers faster. A higher internal metric does not automatically guarantee a better user outcome.

Data leakage: learning vocabulary statistics, labels or preprocessing decisions from the test set can make reported performance too optimistic. Fit learned transformations on training data, then apply them to validation/test data.

Collect → clean → numbers → model → test → use. Test data ko training ki information mat banao.

"},{"id":"u1-challenges","title":"Ambiguity and other challenges with precise examples","html":"

Ambiguity means an expression supports more than one interpretation. Good exam answers name the ambiguity and explain both readings.

ChallengeExampleWhy a system can fail
Lexical/sense ambiguitybank: finance or river edgeOne written word has several senses
Syntactic ambiguityI saw the man with a telescope.The telescope may modify saw or the man
Reference ambiguityRiya told Neha that she won.She may refer to either person without more context
SarcasmGreat, another delay.Literal positive word conveys frustration
NegationThe film was not good.Dropping not reverses meaning
Domain shift“Positive” in a medical reportMeaning/use differs by domain
Code mixingKal meeting reschedule karna.Several languages and spelling conventions
Noisy texttmrw, repeated letters, typosVocabulary/token boundaries become uncertain
Low-resource settingFew labelled examples in a languageModels lack representative training evidence
Bias and privacySkewed data or personal detailsErrors and harms can persist in deployed systems

Homophones sound alike but may differ in spelling (pair/pear). Homographs share spelling but can differ in meaning or pronunciation (lead the team / lead metal). These are different dimensions from syntactic ambiguity.

Long-distance dependencies, cultural knowledge, idioms, morphology and multilingual writing systems also complicate processing. For “kick the bucket”, literal word-by-word interpretation can miss the idiomatic meaning. For “The book that the students recommended is useful”, the verb is agrees with book across an intervening phrase.

Exam mein challenge ka naam + ek sentence + dono meanings/actual failure likho.

"}]},{"id":"u2-preprocess","title":"Text preprocessing, normalization and lexical diversity","unit":2,"mode":"tee","scope":"TEE · Unit 2 PDF pages 1–50","sections":[{"id":"u2-tokenize","title":"Tokenization: sentence, word, character and subword","html":"

Tokenization divides text into units called tokens for further processing. A token may be a word, punctuation mark, character or subword, depending on the tokenizer. Sentence segmentation identifies sentence boundaries. Tokenization is not necessarily just splitting on spaces.

MethodExampleTrade-off
Word“cats sleep” → [cats,sleep]Easy to interpret; unfamiliar words can be unseen tokens
Charactercat → [c,a,t]Handles spelling units; creates longer sequences
Subwordunhappiness → pieces such as [un,happi,ness] in an illustrative vocabularyBalances vocabulary size and rare-word representation
Sentence“Birds fly. Fish swim.” → two sentencesAbbreviations and punctuation complicate boundaries

Actual subword splits depend on the learned vocabulary; the example is illustrative, not a universal tokenizer output. Word segmentation is especially important where spaces do not reliably separate words or in compound-rich languages.

Question. Tokenize “Dr. Rao can't come today.” into words and punctuation under this rule: preserve Dr. and can't as whole tokens, and split only the final full stop.

Given

The tokenization rule is explicitly supplied.

To find

The resulting token sequence.

Formula and why it applies

Apply the stated boundary rule; do not assume every period or apostrophe is a split point.

Step 1. Handle abbreviation

Dr. remains one token.

Step 2. Handle contraction

can't remains one token.

Step 3. Write sequence

[Dr., Rao, can't, come, today, .]

Answer. 6 tokens under the specified rule. Another tokenizer may legitimately produce a different sequence.

Three difficult cases

New York may be two tokens but one named entity. 3.14 should not normally end a sentence at the decimal point. don't may remain whole or split into do + n't, depending on the tokenizer. Always state the convention when calculating counts.

Token = chosen unit. Space split ko universal rule mat samajh.

"},{"id":"u2-normalization","title":"Cleaning, case folding, punctuation and stopwords","html":"

Normalization reduces superficial variation so comparable forms can be processed consistently. Common operations include lowercasing, Unicode normalization, appropriate spelling correction, contraction handling, and optional stemming or lemmatization.

Stopwords are frequent words selected for removal for a particular task, such as the or a. Removing them may reduce feature size, but removing not, no or nor can destroy sentiment meaning. Punctuation can convey sentence boundaries, emotion or mathematical structure. Lowercasing can erase differences such as US versus us.

Question. Preprocess “The CATS are not sleeping!!!” by lowercasing, removing punctuation, removing only stopwords {the,are}, and lemmatizing plural cats→cat and sleeping→sleep.

Given

Operations and stopword set are fixed. Do not remove not.

To find

The final token list.

Formula and why it applies

Apply the requested operations in order, keeping a visible intermediate result after each transformation.

Step 1. Lowercase

the cats are not sleeping!!!

Step 2. Tokenize and remove punctuation

[the,cats,are,not,sleeping]

Step 3. Remove specified stopwords

Remove the and are → [cats,not,sleeping].

Step 4. Lemmatize

cats→cat; sleeping→sleep → [cat,not,sleep].

Answer. [cat,not,sleep]. The negative meaning is retained.

Task decides preprocessing

For a simple keyword-retrieval model, removing selected function words can help. For machine translation, question answering, grammar analysis or transformer input, aggressive removal may discard necessary syntax or violate the model's expected input format. Emoji can carry sentiment; convert or retain them when relevant instead of always deleting them.

Clean karna = useful signal bachake noise hatao. Har word/punctuation delete karna cleaning nahi.

"},{"id":"u2-stem-lemma","title":"Stemming versus lemmatization","html":"

Stemming applies string-based rules, often stripping suffixes, to reduce related forms to a stem. The result need not be a valid dictionary word. Lemmatization uses vocabulary and morphological analysis, often with POS information, to return a dictionary lemma.

CriterionStemmingLemmatization
BasisSurface form/suffix rulesLexicon and morphological/POS analysis
OutputMay be nonword, e.g. studiDictionary form, e.g. study
Context/POSUsually little or noneOften important
Irregular formsMay miss went→goCan map went→go with verb analysis
CostGenerally simplerUsually more linguistic processing

Porter applies a sequence of suffix-removal rules subject to conditions; Snowball provides related language-specific stemming algorithms; Lancaster is often more aggressive; regex stemming uses explicit patterns; Lovins is another established stemming approach. Exact output depends on the algorithm and implementation.

Three worked reductions

  1. studies, studying: an English stemmer may yield studi/study depending on the rule sequence; verb lemmatization aims at study. State the exact algorithm if exact outputs are asked.
  2. better: adjective-aware lemmatization can map it to good; naïve suffix removal cannot infer that irregular relation.
  3. saw: as a past-tense verb it can become see; as a cutting-tool noun it remains saw. POS and context matter.

Over-stemming merges forms that should remain distinct; under-stemming fails to merge related forms. Normalization can increase recall in search while losing useful distinctions, so it is a task-dependent choice.

Stem = rules se cut; lemma = dictionary form, meaning/POS ka dhyan.

"},{"id":"u2-ttr","title":"Type–token ratio: three fully worked sums","html":"

A token is one occurrence; a type is a distinct token form under the chosen normalization. In “cat cat dog”, there are three tokens and two types. Type–token ratio (TTR) measures observed lexical diversity.

\"\\operatorname{TTR}=\\frac{\\text{number

A larger TTR means more distinct words relative to text length in that sample. It is strongly affected by text length and preprocessing, so comparisons should use comparable samples.

Question. Find TTR for “the cat saw the cat” after lowercasing and ignoring punctuation. Keep all words.

Given

Tokens [the,cat,saw,the,cat].

To find

Type count, token count and TTR.

Formula and why it applies

\"TTR=V/N\", where \"V\" counts unique forms and \"N\" counts all occurrences.

Step 1. Count tokens

There are 5 positions → N=5.

Step 2. List types

{the,cat,saw} → V=3.

Step 3. Divide

\"TTR=3/5=0.6\".

Answer. TTR=0.6, or 60%.

Question. Find TTR for “Run run RUN” before and after lowercasing.

Given

Before lowercasing, case distinguishes forms. There are 3 occurrences in either case.

To find

Two TTR values.

Formula and why it applies

\"TTR=V/N\". Recount types after normalization.

Step 1. Before case folding

{Run,run,RUN} has 3 types; N=3.

Step 2. First ratio

\"3/3=1\".

Step 3. After case folding

Tokens [run,run,run]; types {run} → V=1.

Step 4. Second ratio

\"1/3\\approx0.333333\".

Answer. Before=1; after≈0.333333.

Question. A text has 80 tokens and 20 types. Another has 50 tokens and 25 types. Find and compare their TTRs.

Given

Text A: N=80,V=20. Text B: N=50,V=25.

To find

Both ratios.

Formula and why it applies

\"TTR=V/N\".

Step 1. Text A

\"20/80=0.25\".

Step 2. Text B

\"25/50=0.5\".

Step 3. Interpret

B has a larger observed type proportion, but unequal text lengths prevent treating this alone as an unbiased measure of richer language.

Answer. A=0.25, B=0.5.

Tokens = sab occurrences; types = unique names; TTR = unique ÷ total.

"}]},{"id":"u2-representation","title":"Text to numbers: label encoding, one-hot, BoW and TF–IDF","unit":2,"mode":"tee","scope":"TEE · Unit 2 PDF pages 51–92","sections":[{"id":"u2-vocab","title":"Vocabulary and vectors—what each coordinate means","html":"

A vocabulary is the ordered collection of distinct terms allowed by a representation. Its order defines what every coordinate means. A vector is simply an ordered number-list. A document–term matrix puts one document in each row and one vocabulary term in each column.

With vocabulary [cat,dog,fish], the count vector [2,0,1] means cat appears twice, dog zero times and fish once. With [dog,cat,fish], the same text becomes [0,2,1]. A vector without its vocabulary order is incomplete.

Documentcatdogfish
cat cat fish201
dog fish011

Nominal categories have no inherent order, such as cities. Ordinal categories do have an order, such as low < medium < high. Assigning arbitrary word IDs does not create meaningful numerical distances between words.

Vector ka har box ek fixed word ka hai. Pehle column order fix, phir numbers.

"},{"id":"u2-label-onehot","title":"Label and one-hot encoding: three complete examples","html":"

Label encoding assigns each category or word a unique integer ID. One-hot encoding assigns a length-\"V\" vector with one 1 at the word's vocabulary position and zeros elsewhere.

\"\\operatorname{onehot}(w)_j=\\begin{cases}1,&j=\\operatorname{index}(w)\\\\0,&\\text{otherwise.}\\end{cases}\"

The index is an ID, not a meaning score. One-hot represents one token/category; a multiword document generally needs multiple token vectors or an aggregation scheme.

Question. Encode “dog cat dog” using vocabulary [cat,dog,fish], IDs starting at 0. Give label IDs and a one-hot vector for every token.

Given

cat→0, dog→1, fish→2. Vector columns have the same order.

To find

The ID sequence and three one-hot rows.

Formula and why it applies

Replace each word by its ID for labels; set its vocabulary position to 1 for one-hot.

Step 1. Label IDs

dog→1, cat→0, dog→1 → [1,0,1].

Step 2. First dog

[0,1,0].

Step 3. Cat

[1,0,0].

Step 4. Second dog

[0,1,0]. Repeated tokens keep the same representation.

Answer. IDs [1,0,1]; one-hot rows [0,1,0], [1,0,0], [0,1,0].

Question. Encode the city sequence Mumbai, Delhi, Mumbai, Pune using vocabulary [Delhi,Mumbai,Pune] and label IDs starting at 1.

Given

Delhi→1, Mumbai→2, Pune→3.

To find

IDs and one-hot rows.

Formula and why it applies

One-hot uses three coordinates regardless of whether label IDs begin at 0 or 1.

Step 1. IDs

[2,1,2,3].

Step 2. Mumbai rows

Each Mumbai → [0,1,0].

Step 3. Delhi and Pune

Delhi→[1,0,0]; Pune→[0,0,1].

Answer. Rows [0,1,0], [1,0,0], [0,1,0], [0,0,1].

Question. One-hot encode categories [red,green,blue] for a linear regression with an intercept. Then give dummy coding after dropping blue.

Given

Full column order [red,green,blue]. Blue is the baseline.

To find

Full and reduced codes.

Formula and why it applies

Full one-hot rows sum to 1. With an intercept this creates a linear dependency; dropping one category column gives k−1 independent dummy indicators.

Step 1. Full encoding

Categoryredgreenblue
red100
green010
blue001

Step 2. Drop blue column

Categoryredgreen
red10
green01
blue00

Step 3. Interpret zero row

[0,0] now identifies the baseline blue, not missing data.

Answer. Full width 3; reduced dummy width 2. Dropping a column is model-dependent, not a requirement for every neural embedding input.

Limitations: one-hot is large and sparse and does not express semantic similarity. Unseen words need a defined unknown-token or vocabulary-update policy. Label IDs are compact but should not be treated as an ordered scalar feature for nominal words.

Label = ID number. One-hot = us ID ke box mein 1, baaki 0.

"},{"id":"u2-bow","title":"Bag of words and bag of n-grams","html":"

Bag of words (BoW) represents a document by the counts of vocabulary terms, discarding their positions. A binary BoW variant stores only presence/absence; a count BoW stores frequencies. State which is used.

\"x_{d,j}=C(v_j,d).\"

\"x_{d,j}\" is document \"d\"'s coordinate for vocabulary term \"v_j\". \"C(v_j,d)\" is its occurrence count.

A bag of n-grams uses short contiguous token sequences as features. Bigrams can distinguish “not good” from “good”, but increasing n enlarges the vocabulary and creates more rare features.

Question. Build count and binary BoW for D1=“cat cat dog” and D2=“dog fish”. Use vocabulary [cat,dog,fish] and no stopword removal.

Given

Two tokenized documents, fixed vocabulary order.

To find

Count vectors and binary vectors.

Formula and why it applies

Count BoW uses \"C(v_j,d)\". Binary BoW is 1 when that count is positive and 0 otherwise.

Step 1. D1 counts

cat=2, dog=1, fish=0 → [2,1,0].

Step 2. D2 counts

cat=0, dog=1, fish=1 → [0,1,1].

Step 3. Convert to presence

D1 [1,1,0]; D2 [0,1,1].

Answer. Count rows [2,1,0], [0,1,1]; binary rows [1,1,0], [0,1,1].

Question. After lowercasing and removing {is,a,means}, obtain BoW vectors for D1=“Continuous learning is learning lifetime”, D2=“Continuous learning means lifetime learning”, D3=“Learning is a good practice”. Use [continuous,good,learning,lifetime,practice].

Given

Three documents and exact stopword set from the supplied slides.

To find

The document–term count matrix.

Formula and why it applies

Remove only the listed stopwords, then count every remaining token in the fixed column order.

Step 1. Clean D1

[continuous,learning,learning,lifetime] → [1,0,2,1,0].

Step 2. Clean D2

[continuous,learning,lifetime,learning] → [1,0,2,1,0].

Step 3. Clean D3

[learning,good,practice] → [0,1,1,0,1].

Step 4. Compare D1 and D2

They have identical BoW vectors despite different original wording/order.

Answer. Rows [1,0,2,1,0], [1,0,2,1,0], [0,1,1,0,1].

Question. Build a bag of bigrams for “not good but not bad” with vocabulary [not good,good but,but not,not bad]. Do not add sentence boundary tokens.

Given

Five tokens; use adjacent pairs, stride 1.

To find

The bigram count vector.

Formula and why it applies

A sequence of \"L\" tokens has \"L-1\" adjacent bigrams when no boundaries are added.

Step 1. Enumerate pairs

(not,good), (good,but), (but,not), (not,bad).

Step 2. Count each vocabulary bigram

Each appears once.

Step 3. Write vector

[1,1,1,1].

Answer. Four bigram features, each with count 1.

BoW: order bhool, word-count rakh. Bag of bigrams: neighbouring word-pair count.

"},{"id":"u2-tfidf","title":"TF–IDF from the exact slide convention","html":"

TF–IDF weights a term highly when it is frequent in a particular document but uncommon across the corpus. The supplied Unit 2 slides use normalized term frequency and base-10 inverse document frequency:

\"\\operatorname{TF}(t,d)=\\frac{C(t,d)}{N_d}\"
\"\\operatorname{IDF}(t)=\\log_{10}\\frac{D}{\\operatorname{df}(t)}\"
\"\\operatorname{TFIDF}(t,d)=\\operatorname{TF}(t,d)\\operatorname{IDF}(t).\"

\"N_d\" is the document's token count after the specified preprocessing; \"D\" is number of documents; \"\\operatorname{df}(t)\" is the number of documents containing the term at least once. A word repeated ten times in one document contributes one to document frequency.

Why the log? It compresses the ratio's scale. If a term occurs in all documents, \"D/df=1\", so IDF=0 under this unsmoothed convention. Other libraries use smoothed IDF, different TF or normalization. Use the question's formula; do not silently substitute a library default.

Question. A corpus contains 10 documents. The word model appears in 2 documents and occurs 3 times in a 12-token document. Compute TF, IDF and TF–IDF using the slide convention.

Given

D=10, df(model)=2, C(model,d)=3, Nd=12; log base 10.

To find

The TF–IDF weight.

Formula and why it applies

\"TF=C/N_d\", \"IDF=\\log_{10}(D/df)\", then multiply.

Step 1. TF

\"3/12=0.25\".

Step 2. Document ratio

\"10/2=5\".

Step 3. IDF

\"\\log_{10}5\\approx0.698970\".

Step 4. Weight

\"0.25\\times0.698970\\approx0.174743\".

Answer. TF=0.25, IDF≈0.698970, TF–IDF≈0.174743.

Question. A term occurs 4 times in a 20-token document and is present in all 8 corpus documents. Calculate its TF–IDF.

Given

C=4, Nd=20, D=8, df=8.

To find

Its weight under unsmoothed base-10 IDF.

Formula and why it applies

\"TF=C/N_d\", \"IDF=\\log_{10}(D/df)\".

Step 1. TF

\"4/20=0.2\".

Step 2. IDF

\"\\log_{10}(8/8)=\\log_{10}1=0\".

Step 3. Multiply

\"0.2\\times0=0\".

Answer. Weight 0: this term does not distinguish documents under this convention.

Question. Find TF–IDF vectors for D1=“cat cat dog”, D2=“dog fish”, D3=“cat fish fish”, using [cat,dog,fish], normalized TF and log10 IDF.

Given

No further preprocessing. Each term occurs in exactly two of the three documents.

To find

All three vectors.

Formula and why it applies

First count df by documents, then compute each document's TF, then multiply by its term's IDF.

Step 1. Common IDF

\"D=3\", \"df(cat)=df(dog)=df(fish)=2\". Each IDF \"=\\log_{10}(3/2)\\approx0.176091\".

Step 2. D1 term frequencies

Length 3 → \"[2/3,1/3,0]\".

Step 3. D1 weights

\"[(2/3)0.176091,(1/3)0.176091,0]\\approx[0.117394,0.058697,0]\".

Step 4. D2 weights

Length 2; TF \"[0,1/2,1/2]\". Vector \"\\approx[0,0.088046,0.088046]\".

Step 5. D3 weights

Length 3; TF \"[1/3,0,2/3]\". Vector \"\\approx[0.058697,0,0.117394]\".

Answer. D1≈[0.117394,0.058697,0]; D2≈[0,0.088046,0.088046]; D3≈[0.058697,0,0.117394].

TF: iss document mein kitna. IDF: poore corpus mein kitna rare. Dono multiply.

"},{"id":"u2-slide-combined","title":"Combined slide numerical: TF–IDF followed by cosine","html":"

Question. For D1=“Continuous learning is learning lifetime”, D2=“Continuous learning means lifetime learning”, D3=“Learning is a good practice”, find the TF–IDF cosine similarity of D1 and D2. Lowercase and remove {is,a,means}. Use normalized TF and log10 IDF.

Given

Vocabulary [continuous,good,learning,lifetime,practice]. D1 and D2 each have 4 retained tokens; D3 has 3.

To find

TF–IDF vectors and cosine(D1,D2).

Formula and why it applies

\"TF=C/N_d\", \"IDF=\\log_{10}(D/df)\"; \"\\cos(x,y)=(x\\cdot. First build the vectors; cosine cannot be applied directly to words.

Step 1. Count documents containing each term

TermdfIDF
continuous2log10(3/2)≈0.176091
good1log10(3)≈0.477121
learning3log10(1)=0
lifetime2≈0.176091
practice1≈0.477121

Step 2. Term frequencies

D1=D2=[1/4,0,2/4,1/4,0]. D3=[0,1/3,1/3,0,1/3].

Step 3. Multiply TF by IDF

Let \"a=\\tfrac14\\log_{10}(3/2)\\approx0.0440228\". Then D1=D2=[a,0,0,a,0]. D3≈[0,0.159040,0,0,0.159040].

Step 4. Dot product

\"x\\cdot.

Step 5. Vector lengths

\"\\|x\\|=\\|y\\|=\\sqrt{2a^2}\". Their product is \"2a^2\".

Step 6. Divide

\"\\cos(x,y)=2a^2/(2a^2)=1\".

Answer. Cosine(D1,D2)=1. Their BoW Euclidean distance is also 0 because their counts are identical.

The learning coordinate becomes zero because learning occurs in every document. D1–D3 cosine is 0 under this exact TF–IDF convention: their only originally shared word was learning, whose IDF is zero. This is a consequence of the model, not proof that the sentences have unrelated meanings.

Combined sum: pehle cleaned words → counts → TF-IDF → similarity. Beech ka stage skip mat kar.

"}]},{"id":"u2-similarity","title":"Similarity and distance: every numerical step","unit":2,"mode":"tee","scope":"Unit 2 · PDF pages 93–103","sections":[{"id":"u2-cosine","title":"Cosine: products, lengths and division","html":"

Cosine similarity compares the direction of two nonzero vectors. Multiply matching coordinates and add to obtain the dot product. To obtain each vector's norm, square its coordinates, add them and take a square root.

\"\\cos(x,y)=\\frac{\\sum_i

The vectors must use the same coordinate/vocabulary order. For nonnegative BoW or TF–IDF vectors, the result lies from 0 to 1; for general real-valued vectors, it can range from −1 to 1. A zero vector makes this formula undefined.

Question. Find cosine similarity for x=[1,1] and y=[1,0].

Given

Nonzero vectors, matching coordinate order.

To find

Find cosine similarity for x=[1,1] and y=[1,0].

Formula and why it applies

Use \"\\cos(x,y)=(x\\cdot because direction similarity is required.

Step 1. Multiply matching coordinates

1 × 1 = 1; 1 × 0 = 0.

Step 2. Add products

1 + 0 = 1.

Step 3. Length of x

\"\\sqrt{(1)^2+(1)^2}=\\sqrt{2}.\"

Step 4. Length of y

\"\\sqrt{(1)^2+(0)^2}=\\sqrt{1}.\"

Step 5. Divide

\"\\frac{1}{\\sqrt{2}\\sqrt{1}}\\approx

Answer. Cosine≈0.707107.

Question. Find cosine similarity for x=[1,2] and y=[2,4].

Given

Nonzero vectors, matching coordinate order.

To find

Find cosine similarity for x=[1,2] and y=[2,4].

Formula and why it applies

Use \"\\cos(x,y)=(x\\cdot because direction similarity is required.

Step 1. Multiply matching coordinates

1 × 2 = 2; 2 × 4 = 8.

Step 2. Add products

2 + 8 = 10.

Step 3. Length of x

\"\\sqrt{(1)^2+(2)^2}=\\sqrt{5}.\"

Step 4. Length of y

\"\\sqrt{(2)^2+(4)^2}=\\sqrt{20}.\"

Step 5. Divide

\"\\frac{10}{\\sqrt{5}\\sqrt{20}}\\approx

Answer. Cosine≈1.000000.

Question. Find cosine similarity for x=[1,2,0] and y=[0,1,2].

Given

Nonzero vectors, matching coordinate order.

To find

Find cosine similarity for x=[1,2,0] and y=[0,1,2].

Formula and why it applies

Use \"\\cos(x,y)=(x\\cdot because direction similarity is required.

Step 1. Multiply matching coordinates

1 × 0 = 0; 2 × 1 = 2; 0 × 2 = 0.

Step 2. Add products

0 + 2 + 0 = 2.

Step 3. Length of x

\"\\sqrt{(1)^2+(2)^2+(0)^2}=\\sqrt{5}.\"

Step 4. Length of y

\"\\sqrt{(0)^2+(1)^2+(2)^2}=\\sqrt{5}.\"

Step 5. Divide

\"\\frac{2}{\\sqrt{5}\\sqrt{5}}\\approx

Answer. Cosine≈0.400000.

Proportional vectors [1,2] and [2,4] have cosine 1 but nonzero Euclidean distance. Direction similarity does not require identical magnitudes.

Cosine: matching products add upar; dono lengths multiply neeche.

"},{"id":"u2-jaccard","title":"Jaccard: intersection, union and distance","html":"

Jaccard similarity is the fraction of shared distinct elements among all distinct elements in either set.

\"J(A,B)=\\frac{|A\\cap

Intersection means common elements; union means all distinct elements. Repetitions do not count again. Jaccard distance is \"1-J\", not the same quantity as similarity. Two empty sets require a declared convention because the fraction becomes 0/0.

Question. Find Jaccard similarity and distance for A={cat, dog} and B={dog, fish}.

Given

Sets contain unique elements.

To find

Find Jaccard similarity and distance for A={cat, dog} and B={dog, fish}.

Formula and why it applies

\"J=|A\\cap; \"d=1-J\".

Step 1. Find intersection

{dog}; count 1.

Step 2. Find union

{cat, dog, fish}; count 3.

Step 3. Divide

J=1/3≈0.333333.

Step 4. Subtract from one

Distance=1−0.333333≈0.666667.

Answer. Similarity≈0.333333; distance≈0.666667.

Question. Find Jaccard similarity and distance for A={red, blue} and B={red, blue}.

Given

Sets contain unique elements.

To find

Find Jaccard similarity and distance for A={red, blue} and B={red, blue}.

Formula and why it applies

\"J=|A\\cap; \"d=1-J\".

Step 1. Find intersection

{red, blue}; count 2.

Step 2. Find union

{red, blue}; count 2.

Step 3. Divide

J=2/2≈1.000000.

Step 4. Subtract from one

Distance=1−1.000000≈0.000000.

Answer. Similarity≈1.000000; distance≈0.000000.

Question. Find Jaccard similarity and distance for A={apple} and B={orange, pear}.

Given

Sets contain unique elements.

To find

Find Jaccard similarity and distance for A={apple} and B={orange, pear}.

Formula and why it applies

\"J=|A\\cap; \"d=1-J\".

Step 1. Find intersection

{}; count 0.

Step 2. Find union

{apple, orange, pear}; count 3.

Step 3. Divide

J=0/3≈0.000000.

Step 4. Subtract from one

Distance=1−0.000000≈1.000000.

Answer. Similarity≈0.000000; distance≈1.000000.

Jaccard = common unique ÷ total unique. Distance = 1 minus similarity.

"},{"id":"u2-distances","title":"Manhattan, Euclidean and Minkowski: three complete comparisons","html":"

Distance measures separation; smaller means closer. Absolute values prevent positive and negative differences from cancelling.

\"d_1=\\sum_i|x_i-y_i|\\quad\\text{Manhattan}\"
\"d_2=\\sqrt{\\sum_i(x_i-y_i)^2}\\quad\\text{Euclidean}\"
\"d_p=\\left(\\sum_i|x_i-y_i|^p\\right)^{1/p},\\quad

Manhattan is p=1; Euclidean is p=2. For p=3, cube absolute differences, add, then take a cube root.

Question. Find Manhattan, Euclidean and Minkowski p=3 distances for x=[1,2], y=[4,6].

Given

Vectors share coordinate order; negative inputs are allowed.

To find

Find Manhattan, Euclidean and Minkowski p=3 distances for x=[1,2], y=[4,6].

Formula and why it applies

Use \"d_1=\\sum|\\Delta_i|\", \"d_2=\\sqrt{\\sum|\\Delta_i|^2}\", \"d_3=\\sqrt[3]{\\sum|\\Delta_i|^3}\".

Step 1. Absolute differences

|1 − (4)| = 3; |2 − (6)| = 4.

Step 2. Manhattan

3 + 4 = 7.

Step 3. Square and add

3²=9; 4²=16. Sum=25.

Step 4. Euclidean root

\"\\sqrt{25}\\approx

Step 5. Cube and add

3³=27; 4³=64. Sum=91.

Step 6. Minkowski root

\"\\sqrt[3]{91}\\approx

Answer. Manhattan=7; Euclidean≈5.000000; p=3≈4.497941.

Question. Find Manhattan, Euclidean and Minkowski p=3 distances for x=[0,1,2], y=[2,1,0].

Given

Vectors share coordinate order; negative inputs are allowed.

To find

Find Manhattan, Euclidean and Minkowski p=3 distances for x=[0,1,2], y=[2,1,0].

Formula and why it applies

Use \"d_1=\\sum|\\Delta_i|\", \"d_2=\\sqrt{\\sum|\\Delta_i|^2}\", \"d_3=\\sqrt[3]{\\sum|\\Delta_i|^3}\".

Step 1. Absolute differences

|0 − (2)| = 2; |1 − (1)| = 0; |2 − (0)| = 2.

Step 2. Manhattan

2 + 0 + 2 = 4.

Step 3. Square and add

2²=4; 0²=0; 2²=4. Sum=8.

Step 4. Euclidean root

\"\\sqrt{8}\\approx

Step 5. Cube and add

2³=8; 0³=0; 2³=8. Sum=16.

Step 6. Minkowski root

\"\\sqrt[3]{16}\\approx

Answer. Manhattan=4; Euclidean≈2.828427; p=3≈2.519842.

Question. Find Manhattan, Euclidean and Minkowski p=3 distances for x=[-1,3], y=[2,-1].

Given

Vectors share coordinate order; negative inputs are allowed.

To find

Find Manhattan, Euclidean and Minkowski p=3 distances for x=[-1,3], y=[2,-1].

Formula and why it applies

Use \"d_1=\\sum|\\Delta_i|\", \"d_2=\\sqrt{\\sum|\\Delta_i|^2}\", \"d_3=\\sqrt[3]{\\sum|\\Delta_i|^3}\".

Step 1. Absolute differences

|-1 − (2)| = 3; |3 − (-1)| = 4.

Step 2. Manhattan

3 + 4 = 7.

Step 3. Square and add

3²=9; 4²=16. Sum=25.

Step 4. Euclidean root

\"\\sqrt{25}\\approx

Step 5. Cube and add

3³=27; 4³=64. Sum=91.

Step 6. Minkowski root

\"\\sqrt[3]{91}\\approx

Answer. Manhattan=7; Euclidean≈5.000000; p=3≈4.497941.

Manhattan: absolute-add. Euclidean: square-add-root. Minkowski: p-power-add-p-root.

"},{"id":"u2-choice","title":"Choose the representation before the metric","html":"
QuestionMeasureCloser means
Compare vector directionCosine similarityHigher
Compare term setsJaccard similarityHigher
Straight-line separationEuclidean distanceLower
Sum coordinate changesManhattan distanceLower
General p specifiedMinkowski distanceLower

BoW and TF–IDF may give different similarities. First show preprocessing, vocabulary and vectors; then use the metric. Arbitrary label IDs do not define meaningful word distances. Never compare vectors using different term orders. Keep intermediate precision until the final result.

Representation + metric dono question ke hisaab se. Formula pe seedha jump nahi.

"}]},{"id":"u3-corpora","title":"Corpora, lexical resources and data acquisition","unit":3,"mode":"tee","scope":"Unit 3 · PDF pages 1–61; pronouncing dictionary from current syllabus","sections":[{"id":"u3-corpus","title":"Corpus structure and annotations","html":"

A corpus is a collection of language data assembled for study or computational use. Its plural is corpora. A corpus can contain books, news, reviews or conversations. It need not be labelled. Annotation adds linguistic information such as POS tags, entities, parse structures or sense labels.

StructureMeaningUse
PlainCollection without category labelsWord counts, language modelling
CategorizedDocuments grouped by genre/topicClassification
Overlapping categoriesOne document has several categoriesMulti-topic news
TemporalDocuments indexed by timeLanguage change
AnnotatedLinguistic labels attachedTagging, parsing, WSD
ParallelAligned texts in different languagesTranslation

A tagset is the allowed label inventory. A treebank is a corpus annotated with syntax trees. Metadata records source, date, genre, language, annotation method and limitations. A common benchmark supports comparison under shared data and evaluation conditions.

Raw example: “Riya works in Pune.” POS annotation: Riya/PROPN works/VERB in/ADP Pune/PROPN. NER annotation: Riya/PERSON, Pune/LOCATION. Each annotation answers a different task.

A good corpus is relevant to its domain, sufficiently representative, consistently annotated and appropriately split. More data alone do not repair duplicated examples, incorrect labels or a mismatch between training and real use.

Corpus=text collection. Annotation=text ke saath labels. Metadata=collection ki information.

"},{"id":"u3-types","title":"Gutenberg, web/chat, Brown, Reuters and Inaugural","html":"
ResourceContentsTypical use
Gutenberg selectionLiterary/book textsLong-form language and vocabulary
Web textInformal domain-specific textStudy usage beyond edited books
Chat corpusConversational postsDialogue, informal forms
BrownEnglish texts across genresGenre comparison
ReutersNews with topic labelsText classification; overlapping topics
InauguralSpeeches indexed by timeVocabulary change across years
WordlistWords/names/stopwordsLexical filtering
Pronouncing dictionaryWords and phonetic formsSpeech, stress, rhyme

Gutenberg versus web/chat: book language does not represent every spelling or conversational pattern in messages. Brown versus Reuters: Brown emphasizes genres; Reuters news can have overlapping topic labels. Inaugural addresses: dates provide a time dimension.

Corpus sizes depend on version, subset and tokenization. Explain their purpose and structure rather than assuming a historical count is a permanent property.

Gutenberg=books; web/chat=informal; Brown=genres; Reuters=news topics; Inaugural=time.

"},{"id":"u3-access","title":"Raw text, tokens and sentences are different views","html":"

Corpus readers expose several views. raw returns a character string, words returns token occurrences, and sents returns a list of sentences where each sentence contains tokens.

Conceptual methodReturned viewIts length counts
fileids()Document IDsFiles
raw(fileid)Character stringCharacters
words(fileid)Token listToken occurrences
sents(fileid)Sentence listsSentences
categories()Genre/topic labelsCategories
tagged_words(fileid)Word–tag pairsAnnotated tokens

Question. For sentence lists [the,cat,sleeps,.] and [the,dog,runs,.], count all tokens, sentences and lowercase alphabetic word types.

Given

Punctuation is included in token count but excluded from word types.

To find

For sentence lists [the,cat,sleeps,.] and [the,dog,runs,.], count all tokens, sentences and lowercase alphabetic word types.

Formula and why it applies

Count every token occurrence, then distinct alphabetic forms. Different counting rules give different totals.

Step 1. Token count

4+4=8.

Step 2. Sentence count

Two outer lists → 2.

Step 3. Types

{the,cat,sleeps,dog,runs} → 5.

Answer. 8 tokens, 2 sentences, 5 word types.

For “Cats run. Dogs sleep.”, a reader may return six tokens including two periods, but only four alphabetic word tokens. State the convention before calculating TTR or probabilities.

raw=characters, words=tokens, sents=sentences. Teen alag counts.

"},{"id":"u3-resources","title":"Wordlists, pronunciation and semantic resources","html":"

A wordlist stores lexical entries without necessarily giving usage context. A stopword list gives task-specific filtering terms. Names lists can support lexical analysis but cannot reliably infer a real person's identity or gender.

A pronouncing dictionary maps a written word to one or more phonetic symbol sequences. CMU-style dictionaries use ARPAbet; vowel stress suffixes distinguish unstressed, primary-stressed and secondary-stressed vowels. A spelling can have more than one pronunciation, as “read” does.

QuestionResourceReason
Which words rhyme with night?Pronouncing dictionaryCompare sounds, not just spelling
Which terms should be filtered?Specified stopword listApply task's filtering convention
What broader concept includes car?WordNetFollow hypernym relation
How frequently does car occur?Text corpusCount actual usage occurrences

WordNet groups word senses in synsets, supplies glosses and connects meanings by semantic relations. Its full treatment is in Unit 5. A corpus supplies usage evidence; a lexical resource supplies organized knowledge. A system can combine both.

Sound→pronouncing dictionary. Meaning→WordNet. Occurrence→corpus.

"},{"id":"u3-acquisition","title":"Acquiring data and preserving labels during augmentation","html":"

Data acquisition obtains language examples relevant to the task from existing data, public resources or appropriate new collection. Product instrumentation can gather useful feedback. Supervised learning additionally needs an annotation process when labels are absent.

Data augmentation creates training variants intended to preserve the correct label. It is not simply changing words arbitrarily.

MethodProcedureCaution
Synonym replacementUse suitable synonyms for selected wordsChoose the right sense
Back translationTranslate to another language and backMay change meanings/entities
TF–IDF-informed replacementProtect informative words while varying othersDo not remove label-bearing clues
Bigram flippingSwap adjacent wordsCan break grammar or meaning
Entity replacementReplace by same-type entityMust preserve the task label
Noise injectionSimulate typos/keyboard errorsMatch expected noise
Active learningSelect informative cases for human labelsThis acquires labels; it is not synthetic text

Three label decisions

“The film is good”→“The film is excellent” may preserve a positive sentiment label. “The film is not good”→“The film is good” reverses the label and is not valid label-preserving augmentation. “Flights to Pune”→“Flights to Mumbai” may preserve a travel-topic label but changes the destination label.

Split train/test data before creating related augmented variants. Otherwise near-duplicates can leak between sets and produce misleading evaluation. Keep a representative evaluation set of genuine examples.

Sentence badal sakta hai; required correct label nahi. Label badla toh augmentation invalid.

"},{"id":"u3-links","title":"How corpus evidence powers later units","html":"

Word-sequence counts support n-gram language models and smoothing. POS-annotated sentences support supervised taggers. Sense-labelled contexts support Naive Bayes WSD. Word-context counts support distributional semantics and GloVe. Parallel aligned sentences support translation.

For a question asking the importance of corpora, give three points: empirical language evidence, examples for training/evaluation, and reproducible comparison. For an annotated-corpus question, define the added labels and give a concrete example. For resource comparisons, distinguish running text, lexical lists, pronunciation entries and semantic relations.

Same text se task ke hisaab se counts, tags, senses ya context patterns nikaalte hain.

"}]},{"id":"u3-regex","title":"Regular expressions: read, build and test text patterns","unit":3,"mode":"tee","scope":"TEE · current Unit 3 syllabus and supplied Regular Expressions handout; outside confirmed M2 page cutoff","sections":[{"id":"regex-language","title":"Characters, classes, anchors and repetition","html":"

A regular expression (regex) is a pattern language for specifying classes of text strings. A regex engine tests or finds strings matching a pattern. It can extract numbers, locate words, clean text and implement rule-based NLP features. A match is a string satisfying the pattern; it is not an interpretation of the string's meaning.

Start with a literal: cat matches those three consecutive letters. Without boundary constraints it also matches the cat part inside “catalog” or “wildcat”. Add restrictions only when the question asks for them.

PatternMeaningExample
[aeiou]Exactly one listed lowercase vowela, e; not b
[A-Z]One uppercase ASCII letterA through Z
[^0-9]One character not in the digit rangea or a space; not 5
\\dOne digit under the regex engine's character rulesUse [0-9] if only ASCII digits are intended
\\wWord character, usually letters/digits/underscoreUnicode details depend on engine
\\sWhitespace characterSpace, tab, newline
\\D, \\W, \\SComplement classesNon-digit, non-word, non-whitespace
.One arbitrary character except newline by defaultDOTALL flag can include newlines
\\.Literal full stopEscaping removes the wildcard meaning
^Start of string/line depending on mode^The
$End anchor under the engine's rulesSpain$
\\bBoundary between word and non-word positionsWhole-word matching
*Zero or more repetitions of previous itemab* matches a, ab, abb
+One or moreab+ requires at least one b
?Zero or onecolou?r allows color and colour
{m}, {m,n}, {m,}Exact, bounded or minimum repetitions[0-9]{3}
(...)Group a sequence(ab)+ repeats the pair
|Choose an alternativecat|dog

Critical distinction: a character class chooses one character, whereas alternatives can choose whole strings. [cat] matches one c, a or t; (cat|dog) matches either complete word.

A quantifier acts on the immediately preceding atom/group. abc* means ab followed by any number of c; (abc)* repeats the entire abc group.

Regex left-to-right padho: kaunsa character, kitni baar, aur start/end/word boundary kya hai?

"},{"id":"regex-functions","title":"Search, match, fullmatch, findall, substitution and flags","html":"
Python operationWhat it doesTypical use
re.search(pattern,text)Finds the first match anywhereLocate a clue inside a sentence
re.match(pattern,text)Tests only from the startCheck a prefix
re.fullmatch(pattern,text)Requires the complete string to matchValidate an exact format
re.findall(pattern,text)Returns non-overlapping matchesExtract all numbers
re.finditer(pattern,text)Returns match objects with spansLocate match positions
re.sub(pattern,replacement,text)Replaces matchesNormalize repeated whitespace
re.split(pattern,text)Splits at matchesCustom token boundaries

Python raw strings such as r\"\\bcat\\b\" keep the backslashes for the regex engine. In an ordinary Python string, \\b is a backspace escape, so using a raw string prevents a common mistake. Capturing groups can change what findall returns; use a noncapturing group (?:...) when grouping is needed but capturing is not.

Flags and engine differences

IGNORECASE / i ignores case. MULTILINE / m lets ^ and $ refer to line boundaries. DOTALL / s lets dot match newline. JavaScript's g flag requests repeated/global matching; Python has no separate global flag for findall, which already finds successive non-overlapping matches. Do not mix JavaScript slash delimiters with Python's string syntax.

Greedy versus lazy quantifiers

A greedy quantifier tries to consume as much as possible while permitting the whole match; a lazy quantifier tries as little as possible. On <b>one</b><b>two</b>, <.*> can match the entire string from the first < to last >. The lazy <.*?> returns shorter tag-shaped matches during repeated matching. Regex is useful for restricted patterns; robust arbitrary HTML processing should use an HTML parser.

search=kahin bhi; match=start se; fullmatch=poora text; findall=sab matches. Greedy zyada, lazy kam.

"},{"id":"regex-worked","title":"Six full regex questions and solutions","html":"

Question. From “cat catalog wildcat Cat”, extract only the lowercase whole word cat using Python regex.

Given

Case-sensitive matching; whole words only.

To find

A pattern and its matches.

Rule and why it applies

Use \\b boundaries around the literal word. This prevents substring matches inside longer word tokens.

Step 1. Write the literal

cat alone would also match catalog and wildcat.

Step 2. Add both word boundaries

r\"\\bcat\\b\".

Step 3. Check candidates

cat matches. catalog fails at the ending boundary. wildcat fails at the starting boundary. Cat fails because matching is case sensitive.

Step 4. Extract

re.findall(r\"\\bcat\\b\", text) returns [\"cat\"].

Answer. Pattern \\bcat\\b; one match: cat. With IGNORECASE, Cat would also match.

Question. Write a regex that validates exactly ten ASCII digits, then test 9876543210, 98765 and A9876543210.

Given

No country-prefix restriction; digits only; entire input must match.

To find

A validation pattern and each result.

Rule and why it applies

[0-9] permits one ASCII digit; {10} repeats it exactly ten times; fullmatch prevents extra characters.

Step 1. Choose the character class

[0-9].

Step 2. Specify exact length

[0-9]{10}.

Step 3. Require the whole input

re.fullmatch(r\"[0-9]{10}\", text). In a simple single-line exam pattern, ^[0-9]{10}$ expresses the intended format.

Step 4. Test all examples

9876543210: 10 digits → valid. 98765: only 5 → invalid. A9876543210: contains an extra letter → invalid.

Answer. Use fullmatch with [0-9]{10}. Only 9876543210 passes. This checks format, not whether a phone number exists.

Question. Extract date-shaped strings in DD/MM/YYYY format from “On 05/10/2026 and 7/10/2026 we met.” Require two digits for day and month and four for year.

Given

Format recognition only; no calendar-validity test.

To find

Pattern and extracted strings.

Rule and why it applies

Use fixed repetition counts separated by literal slashes, and boundaries around the full pattern.

Step 1. Day part

[0-9]{2}.

Step 2. Month and year

[0-9]{2}/[0-9]{2}/[0-9]{4}.

Step 3. Add boundaries

r\"\\b[0-9]{2}/[0-9]{2}/[0-9]{4}\\b\".

Step 4. Evaluate the two candidates

05/10/2026 has 2/2/4 digits and matches. 7/10/2026 has a one-digit day and fails.

Answer. Only 05/10/2026 is extracted. A date such as 99/99/2026 has the right shape but needs a calendar parser to reject it semantically.

Question. Use a regex to check a simple email shape: one or more username characters from letters, digits, dot, underscore, percent, plus or hyphen; @; a domain of letters, digits, dots or hyphens; a literal dot; a letter-only suffix of length at least two.

Given

This is the simplified handout format, not full standards-compliant validation.

To find

The pattern and an explanation of every part.

Rule and why it applies

Build the requested components and use fullmatch so no extra prefix or suffix is accepted.

Step 1. Username

[A-Za-z0-9._%+-]+: one or more allowed username characters. The hyphen is last, so it is literal.

Step 2. Separator and domain

@[A-Za-z0-9.-]+: literal @ followed by domain characters.

Step 3. Suffix

\\.[A-Za-z]{2,}: escaped dot, then at least two letters.

Step 4. Combine and test

re.fullmatch(r\"[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\\.[A-Za-z]{2,}\", text). riya+nlp@example.com matches; riya.example.com lacks @ and fails.

Answer. The combined pattern meets the stated classroom shape. It neither proves address existence nor enforces every real email/domain constraint.

Question. For text “The rain in Spain”, write patterns that test starts with The, ends with Spain, and satisfies both.

Given

Single-line, case-sensitive text.

To find

Three patterns and outcomes.

Rule and why it applies

Use ^ for beginning, $ for ending, and .* for the intervening sequence in this simple one-line task.

Step 1. Starts with

^The succeeds.

Step 2. Ends with

Spain$ succeeds.

Step 3. Both

^The.*Spain$ succeeds because the text begins with The and ends with Spain.

Step 4. Why dot-star?

. consumes one intervening character; * allows zero or more, including spaces here.

Answer. All three checks succeed. Without anchors a substring match alone would not enforce the required position.

Question. Construct a case-sensitive pattern for full filenames ending in .jpg or .png and test photo.jpg, chart.png and photo.jpg.exe.

Given

Filename must have at least one character before the dot. Treat the filename as a single line.

To find

Pattern and three results.

Rule and why it applies

Use a noncapturing alternative for the two complete suffixes and fullmatch to disallow trailing text.

Step 1. Filename stem

.+ requires one or more characters.

Step 2. Literal dot

\\..

Step 3. Choose suffix

(?:jpg|png) chooses jpg or png.

Step 4. Full match and test

re.fullmatch(r\".+\\.(?:jpg|png)\", filename). photo.jpg passes; chart.png passes; photo.jpg.exe fails.

Answer. Full pattern .+\\.(?:jpg|png); use IGNORECASE only if uppercase extensions should also be accepted.

Pattern likhke ek valid, ek short/wrong, ek extra-character case zaroor mentally run karo.

"},{"id":"regex-reference","title":"Pattern reference for the handout's remaining tasks","html":"
TaskPattern / operationRead it correctly
Extract hashtags#[A-Za-z0-9_]+# followed by one or more permitted tag characters
Extract integer sequences[0-9]+Does not include decimal dots or signs
Extract words ending ing\\b[A-Za-z]*ing\\bZero or more prefix letters, then ing
Currency amounts, simple integers[$₹][0-9]+One currency symbol followed by digits; decimals need extension
Words containing exact uppercase AI\\b[A-Za-z]*AI[A-Za-z]*\\bCase-sensitive; letters before/after may be absent
Capital-initial alphabetic words\\b[A-Z][A-Za-z]*\\bOne capital then zero or more letters
Extract mentions@[A-Za-z0-9_]+Simple username rule, not any platform's complete policy
Exactly five-letter words\\b[A-Za-z]{5}\\bWord boundaries plus exact length
Starts N and ends P\\bN[A-Za-z]*P\\bNP itself is allowed; require + to demand middle letters
Alphabetic words only\\b[A-Za-z]+\\bASCII letter-only token matches
Basic http/https URL extractionhttps?://[^\\s<>]+Matches a URL-shaped span; trailing punctuation may need removal
Collapse whitespacere.sub(r\"\\s+\", \" \", text)Replace each whitespace run with one space

IPv4 addresses: shape versus numeric validity

(?:[0-9]{1,3}\\.){3}[0-9]{1,3} recognizes four dotted digit groups, but permits 999.999.999.999. For numeric octets 0–255 without leading zeros, define one octet as (?:25[0-5]|2[0-4][0-9]|1[0-9]{2}|[1-9]?[0-9]), join four with escaped dots, and require fullmatch. An IP-address parser is clearer for production validation.

Limits: regex recognizes surface patterns; it cannot infer the intended sense of bank, establish a person's identity from capitalization, or decide arbitrary grammatical meaning. A date/email/URL-shaped match is not automatically a valid or real-world existing item.

Regex shape verify karta hai; real meaning/existence ke liye extra logic lagta hai.

"}]},{"id":"m2-language-models","title":"N-grams and language models","unit":3,"scope":"Unit 3, pages 62–108; essential probability revision","sections":[{"id":"words-counts-probability","title":"Words, counts and probabilities: the notation you need","html":"

A token is one occurrence of a word. A type is one distinct word. A corpus is a collection of texts used for analysis or training. In “cats chase cats”, there are three word tokens but only two word types: cats and chase. A language model learns regularities from such a corpus to assign probabilities to word sequences. It does not simply count whether a sentence is grammatically correct.

SymbolRead it asConcrete meaning
C(cat)count of catNumber of times cat occurs.
C(the, cat)count of the followed by catCount adjacent ordered pairs; cat the is different.
P(cat)probability of catA number from 0 to 1.
P(cat | the)probability of cat given theAmong contexts ending in the, how often cat follows.
Vvocabulary sizeNumber of distinct outcomes the model may predict.
Nnumber of tokens or predictionsIts exact meaning must be stated for the calculation.

The vertical bar | means “given”. The condition is on its right. In P(cat | the), the model has already seen “the” and is predicting “cat”. This direction matters: P(cat | the) and P(the | cat) usually have different denominators.

Question. A corpus contains 20 word tokens. The word cat occurs 5 times. Find the maximum-likelihood unigram probability of cat.

Given

Total tokens N = 20; count C(cat) = 5.

To find

P(cat).

Formula and why it applies

\"P(w)=\\frac{C(w)}{N}\"

Relative frequency estimates the probability of drawing this word from the corpus. The denominator includes every token, not just distinct words.

Solution

Step 1. Select the matching count

The required outcome is cat, which occurs 5 times.

Step 2. Divide by the total number of trials
\"P(\\text{cat})=\\frac{5}{20}=0.25\"
Step 3. Interpret the result

Under this unigram model, cat receives probability 0.25, or 25%.

Final answer. P(cat) = 0.25.

Yaad rakho: Jo chahiye uska count upar; total possible trials neeche.

Question. The word the has 10 outgoing occurrences in a corpus. It is followed by cat on 4 occasions and dog on 6. Find P(cat | the) and P(dog | the).

Given

C(the, cat) = 4; C(the, dog) = 6; C(the) = 10 outgoing contexts.

To find

Both next-word probabilities.

Formula and why it applies

\"P(w\\mid

The condition h = the restricts attention to its 10 continuations. We do not divide by the total corpus size.

Solution

Step 1. Use the cat continuation
\"P(\\text{cat}\\mid\\text{the})=\\frac{4}{10}=0.4\"
Step 2. Use the dog continuation
\"P(\\text{dog}\\mid\\text{the})=\\frac{6}{10}=0.6\"
Step 3. Check the complete row
\"0.4+0.6=1\"
Final answer. P(cat | the) = 0.4 and P(dog | the) = 0.6.

Yaad rakho: Given waala word denominator choose karata hai.

Question. A collection has 8 sentences; 6 start with I and 2 start with we. Estimate each starting-word probability.

Given

Each sentence receives a start marker <s>. C(<s>) = 8; C(<s>, I) = 6; C(<s>, we) = 2.

To find

P(I | <s>) and P(we | <s>).

Formula and why it applies

\"P(w_1\\mid\\langle

The start marker supplies the context for the first word; one start context is available per sentence.

Solution

Step 1. Calculate the first option
\"P(I\\mid\\langle
Step 2. Calculate the second option
\"P(we\\mid\\langle
Final answer. The estimated probabilities are 0.75 and 0.25.
"},{"id":"n-grams-construction","title":"Unigrams, bigrams and trigrams: construct them correctly","html":"

Definition. An n-gram is a contiguous sequence of n items. For word n-grams, the items are consecutive word tokens. A unigram contains one word; a bigram contains two; a trigram contains three. Character n-grams use characters instead, so always identify the unit being counted.

Tokens: we | study | natural | language
→
Move a window one token at a time
→
Bigrams: we study / study natural / natural language

For m tokens without padding, the number of n-gram positions is max(0, m−n+1). Repeated positions may contain the same n-gram type. A bag-of-n-grams representation counts each n-gram type and loses the overall sentence order beyond the local order inside each n-gram.

\"\\text{Number

Question. List all word unigrams, bigrams and trigrams for “the prime minister of our country”. Do not add boundary tokens.

Given

Six tokens: the, prime, minister, of, our, country. n = 1, 2 and 3.

To find

All three lists and their counts.

Formula and why it applies

\"\\#\\text{positions}=m-n+1\"

Each window must contain consecutive words and move one position. No wraparound or invented final word is allowed.

Solution

Step 1. Take one word at a time

Unigrams: (the), (prime), (minister), (of), (our), (country). Count = 6−1+1 = 6.

Step 2. Take two consecutive words

Bigrams: (the, prime), (prime, minister), (minister, of), (of, our), (our, country). Count = 6−2+1 = 5.

Step 3. Take three consecutive words

Trigrams: (the, prime, minister), (prime, minister, of), (minister, of, our), (of, our, country). Count = 6−3+1 = 4.

Final answer. 6 unigrams, 5 bigrams and 4 trigrams.

Question. Construct bigrams for “I like tea” using one start and one end marker.

Given

Original tokens: I, like, tea. Add <s> and </s> once each.

To find

The boundary-aware bigram sequence.

Formula and why it applies

\"\\#\\text{bigrams}=(m+2)-1=m+1\"

The start context predicts the first word, and the last word predicts that the sentence ends.

Solution

Step 1. Write the padded sequence

<s> I like tea </s>

Step 2. Move a two-token window

(<s>, I), (I, like), (like, tea), (tea, </s>).

Step 3. Count the windows
\"3+1=4\"
Final answer. There are four boundary-aware bigrams.

Question. Construct trigrams for “I like tea” using two start markers and one end marker.

Given

Original sentence has three words. For a trigram model each predicted token needs two prior context tokens.

To find

All trigram prediction events.

Formula and why it applies

\"P(w_i\\mid

Two start markers provide a complete context even for the first word.

Solution

Step 1. Pad with two starts

<s> <s> I like tea </s>

Step 2. Slide a three-token window

(<s>, <s>, I), (<s>, I, like), (I, like, tea), (like, tea, </s>).

Step 3. Count predicted events

The model predicts I, like, tea and the end marker: four predictions.

Final answer. Four trigrams. Start markers are context; they are not ordinary predicted words.

Question. Give the bag-of-bigrams counts for “a b a b” without padding.

Given

Four tokens in their given order.

To find

Bigram positions, distinct bigram types and their counts.

Formula and why it applies

\"\\#\\text{positions}=4-2+1=3\"

A bag stores frequencies; repeated bigrams must not be discarded.

Solution

Step 1. Enumerate all positions

(a,b), (b,a), (a,b).

Step 2. Group identical pairs
Bigram typeCount
a b2
b a1
Final answer. Three bigram positions but two distinct bigram types.

Yaad rakho: n-gram mein n words; context mein n−1 words.

"},{"id":"collocations-and-lm","title":"Collocations and the purpose of a language model","html":"

A collocation is a conventional word combination whose words occur together more strongly than ordinary chance or independent frequencies would suggest. Examples include “strong tea”, “heavy rain” and “red wine”. A bigram is merely two consecutive items; it need not be a collocation. “of the” can be frequent because both words are frequent, while a collocation expresses a particularly strong association.

TermWhat it representsExample
BigramAny adjacent two-token sequencethe book
CollocationA characteristic strongly associated expressionheavy rain
Language modelA distribution used to score or predict sequencesP(rain | heavy)

Definition. A statistical language model assigns a probability to a sequence of words, or equivalently predicts the next word from preceding context. It is useful for autocomplete, speech recognition, machine translation, spelling correction, assistive communication and ranking candidate text. High probability reflects patterns learned from its training distribution; it is not a guarantee of truth or grammatical correctness.

Previous words: please turn your homework
→
Candidate: in / over / elephant
→
Language model gives each candidate a probability
→
Application selects or ranks candidates

Noisy-channel connection. Suppose x is the observed noisy text or speech and y is a candidate intended message. The noisy-channel approach chooses the y that maximizes P(x | y)P(y). The channel model P(x | y) asks how y could produce x; the language model P(y) asks whether y is a plausible message. P(x) is the same for all candidate y and therefore can be omitted when ranking.

\"\\hat

Question. A spelling-correction system considers “flowers” and “flours”. For the observed input, the channel likelihoods are 0.2 and 0.4, and the respective language-model probabilities in context are 0.08 and 0.01. Which candidate does the noisy-channel rule choose?

Given

Candidate flowers: P(x|flowers)=0.2, P(flowers)=0.08. Candidate flours: P(x|flours)=0.4, P(flours)=0.01.

To find

The larger unnormalized candidate score.

Formula and why it applies

\"\\operatorname{score}(y)=P(x\\mid

Both the error mechanism and the language context matter. The same denominator would normalize both candidates.

Solution

Step 1. Score flowers
\"0.2\\times0.08=0.016\"
Step 2. Score flours
\"0.4\\times0.01=0.004\"
Step 3. Compare
\"0.016>0.004\"
Final answer. Choose flowers. The scores are not yet posterior probabilities.

Yaad rakho: Har word-pair bigram hai; har frequent pair meaningful collocation nahi.

"},{"id":"chain-rule-markov","title":"Probability chain rule and the Markov assumption","html":"

The probability chain rule decomposes the probability of a complete sequence into successive conditional probabilities. It is different from the derivative chain rule used in deep learning. Here we multiply probabilities, not derivatives.

\"P(w_1,w_2,w_3)=P(w_1)P(w_2\\mid

Read left to right: probability of the first word; then probability of the second after the first; then probability of the third after both previous words. For a longer sentence, continue the same pattern. The product symbol ∏ simply means “multiply all of these terms”.

\"P(w_1,\\ldots,w_m)=\\prod_{i=1}^{m}P(w_i\\mid

The Markov assumption approximates long history by a limited recent context. A unigram model ignores preceding words. A bigram model keeps the previous word. A trigram model keeps the previous two words. This reduces the number of counts needed, but loses longer-range dependencies.

ModelPrediction ruleFor the next word country after the prime minister of our
UnigramP(wᵢ)P(country)
BigramP(wᵢ | wᵢ₋₁)P(country | our)
TrigramP(wᵢ | wᵢ₋₂,wᵢ₋₁)P(country | of, our)
Full chain ruleP(wᵢ | all previous words)P(country | the, prime, minister, of, our)

Question. For the three-word sentence “I like tea”, P(I)=0.2, P(like|I)=0.5 and P(tea|I,like)=0.4. Find its probability using the full chain rule.

Given

Three conditional factors are given, including the full two-word history for tea. No end probability is requested.

To find

P(I, like, tea).

Formula and why it applies

\"P(I,like,tea)=P(I)P(like\\mid

The chain rule supplies one factor per word in the specified sequence.

Solution

Step 1. Insert the three supplied factors
\"P=0.2\\times0.5\\times0.4\"
Step 2. Multiply the first two
\"0.2\\times0.5=0.1\"
Step 3. Multiply by the third
\"0.1\\times0.4=0.04\"
Final answer. The sentence probability is 0.04.

Question. A bigram model gives P(I|<s>)=0.5, P(like|I)=0.6, P(tea|like)=0.25 and P(</s>|tea)=0.8. Find the full sentence probability.

Given

All four conditional probabilities, including the end event, are supplied.

To find

P(I like tea </s> | <s>).

Formula and why it applies

\"P=P(I\\mid\\langle

Under the bigram assumption, every factor uses one preceding token.

Solution

Step 1. Insert the factors in sentence order
\"P=0.5\\times0.6\\times0.25\\times0.8\"
Step 2. Start and second word
\"0.5\\times0.6=0.3\"
Step 3. Third word
\"0.3\\times0.25=0.075\"
Step 4. End the sentence
\"0.075\\times0.8=0.06\"
Final answer. The complete sentence probability is 0.06.

Yaad rakho: Sequence probability = saare required transitions ka product.

Typical 2-mark answer. An n-gram language model predicts a word from its previous n−1 words using a Markov approximation. For example, a bigram model uses P(wᵢ | wᵢ₋₁). It estimates these probabilities from corpus counts and multiplies the appropriate factors to score a sentence.

"}]},{"id":"m2-count-to-probability","title":"From a corpus to sentence probabilities","unit":3,"scope":"Unit 3, pages 97–120 and appendix","sections":[{"id":"unigram-estimation","title":"Unigram estimation and complete worked examples","html":"

Maximum likelihood estimation (MLE) chooses probabilities equal to observed relative frequencies. In a unigram model, every word is predicted independently of its surrounding words. If a word occurs c times in N training tokens, its estimate is c/N. To score m specified words, multiply their m unigram probabilities.

\"P_{\\mathrm{MLE}}(w)=\\frac{C(w)}{N}\\qquad

Choose a tokenization convention before counting. Lowercasing, punctuation handling and boundary markers can change the vocabulary and counts. Never count words with one convention and form the denominator with another. A unigram model gives the same product to rearrangements of the same word multiset; that is one of its limitations.

Question. Train an unpadded, case-insensitive unigram model on “I like tea” and “I like coffee”. Find the probability of “I like tea”.

Given

Training sentences: I like tea; I like coffee. Use lowercase words; exclude punctuation and sentence boundaries.

To find

The vocabulary, word probabilities and test-sentence probability.

Formula and why it applies

\"P(w)=C(w)/N,\\qquad

The unigram approximation ignores order and uses total token count for every denominator.

Solution

Step 1. Count every training token

Each sentence contains three words, so N = 3+3 = 6.

Step 2. Count types and frequencies
WordCountMLE probability
i22/6 = 1/3
like22/6 = 1/3
tea11/6
coffee11/6

Vocabulary size V = 4.

Step 3. Write one factor per test word
\"P=\\frac{2}{6}\\times\\frac{2}{6}\\times\\frac{1}{6}\"
Step 4. Multiply the fractions
\"P=\\frac{4}{216}=\\frac{1}{54}\\approx0.018519\"
Final answer. The unigram sentence probability is approximately 0.018519.

Question. A corpus contains counts a: 4, b: 3 and c: 1. Find the unigram probability of “a b a”.

Given

C(a)=4, C(b)=3, C(c)=1. No boundary markers.

To find

P(a b a).

Formula and why it applies

\"P(a\\,b\\,a)=P(a)P(b)P(a)\"

A repeated test word contributes a probability factor at every occurrence; do not include it just once.

Solution

Step 1. Find the total
\"N=4+3+1=8\"
Step 2. Estimate the required probabilities
\"P(a)=4/8=1/2,\\qquad
Step 3. Use both occurrences of a
\"P=\\frac12\\times\\frac38\\times\\frac12\"
Step 4. Simplify
\"P=\\frac3{32}=0.09375\"
Final answer. P(a b a) = 0.09375.

Yaad rakho: Repeated word ka probability bhi repeat multiply hoga.

Question. An unsmoothed unigram model is trained on “red blue red”. Find the probability of “red green”.

Given

N=3. Counts: red=2, blue=1, green=0. The test contains red and green.

To find

The unsmoothed test probability and the reason for its value.

Formula and why it applies

\"P(red\\

MLE assigns zero to an unseen outcome. Smoothing is needed if an unseen in-vocabulary item must receive probability.

Solution

Step 1. Estimate red
\"P(red)=2/3\"
Step 2. Estimate the unseen word
\"P(green)=0/3=0\"
Step 3. Multiply
\"(2/3)\\times0=0\"
Final answer. Unsmoothed probability = 0. This reflects missing training evidence, not proof that the phrase is impossible.
"},{"id":"bigram-count-table","title":"Bigram counts: build the table before multiplying","html":"

A bigram probability table uses the previous word as its row and the next word as its column. Divide each cell count by its row's total outgoing count. With one end marker on every sentence, each ordinary training word has a continuation, so its outgoing count equals its token frequency. Do not create a bigram that crosses from one sentence into the next.

\"P(w\\mid
Add boundaries to each sentence separately
→
List adjacent pairs
→
Count repeated pairs
→
Divide each row by its total
→
Follow the test sentence through the table

Question. Train a bigram model on “I like tea”, “I like coffee” and “you like tea”. Use one start and one end marker. Find the probability of “I like tea”.

Given

Corpus: <s> I like tea </s>; <s> I like coffee </s>; <s> you like tea </s>. Preserve these tokens.

To find

All nonzero bigram estimates and the test probability.

Formula and why it applies

\"P(w\\mid

Every sentence starts and ends independently. Four probability factors are required for the three test words plus end.

Solution

Step 1. List the training pairs

Sentence 1: (<s>,I), (I,like), (like,tea), (tea,</s>).
Sentence 2: (<s>,I), (I,like), (like,coffee), (coffee,</s>).
Sentence 3: (<s>,you), (you,like), (like,tea), (tea,</s>).

Step 2. Create the nonzero count table
History → nextPair countHistory countProbability
<s> → I232/3
<s> → you131/3
I → like221
you → like111
like → tea232/3
like → coffee131/3
tea → </s>221
coffee → </s>111
Step 3. Write the test path
\"P=P(I\\mid\\langle
Step 4. Substitute each matching row
\"P=\\frac23\\times1\\times\\frac23\\times1\"
Step 5. Multiply
\"P=\\frac49\\approx0.444444\"
Final answer. P(I like tea) = 4/9 ≈ 0.444444.

Question. Using the same three-sentence corpus given here—“I like tea”, “I like coffee”, “you like tea”—find the bigram probabilities of “you like coffee” and “tea like I”. Include boundaries.

Given

Corpus with one start/end marker per sentence. Counts: start→you=1/3; you→like=1; like→coffee=1/3; coffee→end=1. No training sentence starts with tea.

To find

The two test probabilities.

Formula and why it applies

\"P(\\text{sentence})=\\prod

A model can assign a nonzero probability to a sentence it never saw if all of its local bigrams were seen.

Solution

Step 1. Score you like coffee
\"P=\\frac13\\times1\\times\\frac13\\times1=\\frac19\\approx0.111111\"
Step 2. Inspect tea like I
\"P(tea\\mid\\langle
Step 3. Apply the zero factor

The full product is zero, regardless of the remaining factors.

Final answer. P(you like coffee) = 1/9; P(tea like I) = 0. A new sentence is not necessarily a zero-probability sentence.

Question. Obtain the unsmoothed bigram probability of “students are from Vellore” from this corpus: “I am from Vellore”; “I am a teacher”; “students are good and are from various cities”; “students from Vellore do engineering”. Use sentence boundaries.

Given

Four training sentences, each padded with one <s> and </s>. Test sequence: <s> students are from Vellore </s>.

To find

The five required conditional probabilities and their product.

Formula and why it applies

\"P(w\\mid

Count all occurrences in the entire corpus. In particular, are occurs twice inside the third sentence, and Vellore is followed by end once and do once.

Solution

Step 1. Count the histories
HistoryOccurrences with continuation
<s>4
students2
are2
from3
Vellore2
Step 2. Count the required pairs
Required pairCountProbability
<s> students22/4
students are11/2
are from11/2
from Vellore22/3
Vellore </s>11/2
Step 3. Write the complete product
\"P=\\frac24\\times\\frac12\\times\\frac12\\times\\frac23\\times\\frac12\"
Step 4. Multiply progressively
\"\\frac24\\times\\frac12=\\frac14,\\quad
\"\\frac18\\times\\frac23=\\frac1{12},\\quad
Final answer. P = 1/24 ≈ 0.0416667.

Yaad rakho: Same sentence mein word do baar aaye toh count bhi do.

"},{"id":"trigram-estimation","title":"Trigram estimates: two-word history, not two denominators","html":"

A trigram model predicts the next word from the two preceding words. The numerator is the count of a triple. The denominator is the count of its two-word history with an available next token. Do not multiply two independent bigram probabilities and call that a trigram probability.

\"P(w_i\\mid

Question. A corpus contains 12 occurrences of “prime minister”. Eight are followed by “of”, three by “said”, and one by “visited”. Estimate the three trigram probabilities.

Given

History h = (prime, minister). Count of this history = 12. Triple counts for of, said, visited are 8, 3, 1.

To find

The conditional distribution of the next word.

Formula and why it applies

\"P(w\\mid

All three candidates share one two-word history, so they share denominator 12.

Solution

Step 1. Calculate of
\"P(of\\mid
Step 2. Calculate said
\"P(said\\mid
Step 3. Calculate visited
\"P(visited\\mid
Step 4. Confirm normalization
\"8/12+3/12+1/12=12/12=1\"
Final answer. The probabilities are 2/3, 1/4 and 1/12.

Question. Train a trigram model on “I like tea”, “I like coffee” and “you like tea”. Use two start markers and one end marker. Score “I like tea”.

Given

Three sentences. Padded first sentence is <s> <s> I like tea </s>; pad the other two in the same way.

To find

Four trigram prediction probabilities and the sentence probability.

Formula and why it applies

\"P=\\prod_i

Each prediction uses a two-token context; the two start markers supply the first history.

Solution

Step 1. Count the first-word event

The history (<s>,<s>) occurs three times, and two are followed by I.

\"P(I\\mid\\langle
Step 2. Count the second-word event

(<s>,I) occurs twice; both are followed by like.

\"P(like\\mid\\langle
Step 3. Count the third-word event

(I,like) occurs twice; one is followed by tea.

\"P(tea\\mid
Step 4. Count the end event

(like,tea) occurs twice; both are followed by end.

\"P(\\langle/s\\rangle\\mid
Step 5. Multiply all four
\"P=\\frac23\\times1\\times\\frac12\\times1=\\frac13\"
Final answer. Trigram P(I like tea) = 1/3. It differs from the bigram estimate because the context and denominator changed.

Question. A trigram model supplies P(we|<s>,<s>)=0.4, P(study|<s>,we)=0.5, P(NLP|we,study)=0.8 and P(</s>|study,NLP)=0.25. Score “we study NLP”.

Given

Four trigram conditional factors. The end marker is included.

To find

The complete sentence probability.

Formula and why it applies

\"P=P(we\\mid

Use each supplied two-word history exactly; do not swap its word order.

Solution

Step 1. Insert the values
\"P=0.4\\times0.5\\times0.8\\times0.25\"
Step 2. Multiply the first pair
\"0.4\\times0.5=0.2\"
Step 3. Include NLP
\"0.2\\times0.8=0.16\"
Step 4. Include the end
\"0.16\\times0.25=0.04\"
Final answer. P = 0.04.

Yaad rakho: Trigram: teen-word count upar, pehle do-word count neeche.

Why larger n is not automatically better. More context can capture local order more precisely, but each longer history occurs less often. Parameter count, storage and sparsity grow. A model trained on Shakespeare may assign poor probabilities to financial news because training and test distributions differ. Keep a held-out test set; do not evaluate only on memorized training sentences.

"},{"id":"class-count-table-questions","title":"Class-style count-table questions and partial-table traps","html":"

Question. Train a bigram model on “I am Sam”, “Sam I am”, and “I do not like green eggs and ham”. Include sentence boundaries. Find P(I|<s>), P(Sam|<s>), P(am|I), P(do|I), P(Sam|am) and P(</s>|Sam).

Given

Three sentences; one <s> and </s> per sentence; preserve capital I and Sam.

To find

Six bigram estimates.

Formula and why it applies

\"P(w\\mid

Count all matching histories, including repeated words in different sentence positions.

Solution

Step 1. Count the histories

START occurs 3 times; I occurs 3 times; am occurs 2 times; Sam occurs 2 times.

Step 2. Count the requested pairs
PairPair countHistory count
START I23
START Sam13
I am23
I do13
am Sam12
Sam END12
Step 3. Divide pair by history
Requested probabilityResult
P(I|START)2/3
P(Sam|START)1/3
P(am|I)2/3
P(do|I)1/3
P(Sam|am)1/2
P(END|Sam)1/2
Final answer. The six estimates are 2/3,1/3,2/3,1/3,1/2,1/2 in the requested order.

Question. A bigram table gives P(i|<s>)=0.25, P(want|i)=0.33, P(english|want)=0.0011, P(food|english)=0.5 and P(</s>|food)=0.68. Score “i want english food” using these supplied rounded values.

Given

All five transition probabilities are explicitly given. Include the end marker.

To find

The sentence probability from the supplied table.

Formula and why it applies

\"P=P(i\\mid

A table-based question uses the provided values; slight differences can arise if exact corpus fractions are used instead.

Solution

Step 1. First two events
\"0.25(0.33)=0.0825\"
Step 2. Add english
\"0.0825(0.0011)=0.00009075\"
Step 3. Add food
\"0.00009075(0.5)=0.000045375\"
Step 4. End the sentence
\"0.000045375(0.68)=0.000030855\"
Final answer. P≈0.000030855 using the supplied rounded probabilities.

Question. A corpus has vocabulary size 1446, C(i)=2533 and C(i,want)=827. Compute the unsmoothed and add-one-smoothed P(want|i). The displayed bigram table contains only eight vocabulary columns.

Given

Full history count C(i)=2533; pair count 827; complete vocabulary size V=1446. The displayed table is only a subset.

To find

Both conditional probabilities.

Formula and why it applies

\"P(want\\mid

Use the supplied full history total. Summing only the eight displayed columns would wrongly ignore all other possible continuations.

Solution

Step 1. Calculate the unsmoothed ratio
\"P=827/2533\\approx0.326490\"
Step 2. Add one to the pair count
\"827+1=828\"
Step 3. Add V to the history count
\"2533+1446=3979\"
Step 4. Divide
\"P_L=828/3979\\approx0.208092\"
Final answer. Unsmoothed≈0.326490; smoothed≈0.208092.

Yaad rakho: Partial table ka visible row-sum, full history total nahi hota.

"}]},{"id":"m2-evaluation-smoothing","title":"Perplexity, log probabilities and smoothing","unit":3,"scope":"Unit 3, pages 121–156","sections":[{"id":"evaluation-perplexity","title":"Perplexity: meaning, formula and four complete examples","html":"

Intrinsic evaluation evaluates a model directly on a held-out test corpus, for example using perplexity. Extrinsic evaluation places it inside a downstream application, such as a speech recognizer or translation system, and measures the application's performance. Extrinsic testing is closer to actual use but often slower and more expensive. A lower perplexity is not a guarantee of improvement on every downstream task.

Perplexity is the inverse probability of a test sequence, normalized by the number of prediction events. Intuitively, it is the model's effective average number of equally plausible choices. A model that always gives the correct next word probability 1/4 has perplexity 4. Lower is better when the models are tested on the same data, vocabulary, tokenization and boundary convention.

\"\\operatorname{PP}(W)=P(W)^{-1/N}=\\sqrt[N]{\\frac1{P(W)}}\"
SymbolMeaning
WThe complete test sequence.
P(W)Product of the probabilities assigned to the actual next tokens.
NNumber of scored prediction events under the stated convention.
Exponent −1/NTake the reciprocal, then the Nth root.

For calculator entry, use parentheses: P raised to (−1 ÷ N), or (1 ÷ P) raised to (1 ÷ N). Retain several digits in intermediate products. The ordinary start marker is usually given context, not a predicted event; the end marker counts if its probability is included.

Question. A model assigns probability 0.0625 to a test sequence containing four scored tokens. Compute its perplexity.

Given

P(W)=0.0625=1/16; N=4.

To find

PP(W).

Formula and why it applies

\"\\operatorname{PP}(W)=\\left(\\frac1{P(W)}\\right)^{1/N}\"

The root removes the effect of sequence length; the reciprocal makes lower surprise correspond to lower perplexity.

Solution

Step 1. Take the reciprocal
\"1/P(W)=1/0.0625=16\"
Step 2. Use the fourth root
\"\\operatorname{PP}=16^{1/4}\"
Step 3. Evaluate
\"2^4=16\\quad\\Rightarrow\\quad\\operatorname{PP}=2\"
Final answer. Perplexity = 2.

Question. Two language models score the same three-token test sequence. Model A gives next-token probabilities 0.5, 0.5, 0.5. Model B gives 0.5, 0.25, 0.25. Compute their perplexities and select the better intrinsic result.

Given

Same sequence and N=3 for both models. A factors: 0.5,0.5,0.5. B factors: 0.5,0.25,0.25.

To find

Both sequence probabilities and perplexities.

Formula and why it applies

\"P(W)=\\prod_i

A fair comparison uses identical test tokens and scoring conventions.

Solution

Step 1. Calculate A's probability
\"P_A=0.5\\times0.5\\times0.5=0.125=1/8\"
Step 2. Calculate A's perplexity
\"\\operatorname{PP}_A=8^{1/3}=2\"
Step 3. Calculate B's probability
\"P_B=0.5\\times0.25\\times0.25=0.03125=1/32\"
Step 4. Calculate B's perplexity
\"\\operatorname{PP}_B=32^{1/3}\\approx3.174802\"
Step 5. Compare

2 is lower than 3.174802; A places more probability on the observed sequence.

Final answer. Model A is better by perplexity on this test: 2 versus about 3.174802.

Question. The bigram training corpus is “This is a dog”, “This is a cat”, “I love my cat”, “This is my name”. Score “This is my cat”, including an end marker, and compute perplexity using five predicted events.

Given

Each training sentence has one start and end marker. Test predictions: This, is, my, cat, </s>. Thus N=5.

To find

Bigram sentence probability and perplexity.

Formula and why it applies

\"P(w\\mid

Start is given context. Four words plus the end marker produce five probability factors.

Solution

Step 1. Count the required transitions
TransitionPair count / history count
<s>→This3/4
This→is3/3
is→my1/3
my→cat1/2
cat→</s>2/2
Step 2. Form the product
\"P=\\frac34\\times1\\times\\frac13\\times\\frac12\\times1=\\frac18\"
Step 3. Take the reciprocal
\"1/P=8\"
Step 4. Take the fifth root
\"\\operatorname{PP}=8^{1/5}\\approx1.515717\"
Final answer. P(W)=1/8 and perplexity≈1.515717 using five predicted events.

If the question explicitly states N = 6: use that supplied convention and report PP = 81/6 ≈ 1.414214. The class slide counts both boundary markers in N. Write the chosen N beside the formula; do not silently switch between 5 and 6. Also, This occurs three times and is follows it three times, so P(is|This)=3/3, not 3/4.

Question. A uniform model assigns probability 1/10 to each actual next word in a five-token test. Find its perplexity.

Given

Five scored tokens, each with assigned probability 0.1. N=5.

To find

The sentence probability and perplexity.

Formula and why it applies

\"P(W)=(1/10)^5,\\qquad

Uniform equal probabilities make the effective number of choices especially clear.

Solution

Step 1. Multiply the five factors
\"P(W)=0.1^5=0.00001=10^{-5}\"
Step 2. Invert
\"1/P(W)=10^5\"
Step 3. Take the fifth root
\"\\operatorname{PP}=(10^5)^{1/5}=10\"
Final answer. Perplexity = 10.

Yaad rakho: Probability high → surprise low → perplexity low.

"},{"id":"log-probabilities","title":"Log probabilities: replace long products with sums","html":"

Probabilities are at most 1, so multiplying many of them produces very small numbers. Computers can round extremely small positive products to zero: numerical underflow. Log probabilities convert multiplication into addition and are more stable for long sequences. Natural logs use base e; log₂ uses base 2. Choose one base consistently.

\"\\ln
\"P(W)=e^{\\ln

The log of a probability below 1 is negative. A value closer to zero means a larger probability: ln(0.5) ≈ −0.693 is larger than ln(0.1) ≈ −2.303. A zero probability has log −∞; taking logs does not solve the unseen-event problem. Smoothing handles that separate problem.

Question. The next-word probabilities are 0.5, 0.25 and 0.5. Calculate the natural log probability, ordinary probability and perplexity.

Given

p₁=0.5; p₂=0.25; p₃=0.5; N=3.

To find

ln P(W), P(W), and PP.

Formula and why it applies

\"\\ln

Logarithms preserve products as sums and the exponent converts the result back.

Solution

Step 1. Compute individual logs
\"\\ln0.5\\approx-0.693147,\\quad\\ln0.25\\approx-1.386294\"
Step 2. Add all three terms
\"\\ln
Step 3. Recover the probability
\"P=e^{-2.772589}\\approx0.0625\"
Step 4. Find average negative log probability
\"-\\ln
Step 5. Exponentiate
\"\\operatorname{PP}=e^{0.924196}\\approx2.519842\"
Final answer. ln P≈−2.772589, P=0.0625, PP≈2.519842.

Question. A model has a total natural log probability −12 over six scored tokens. Calculate its perplexity.

Given

ln P(W)=−12; N=6.

To find

PP without forming the tiny ordinary probability.

Formula and why it applies

\"\\operatorname{PP}=\\exp(-\\ln

The logarithmic formula directly uses the supplied log score.

Solution

Step 1. Negate the log score
\"-\\ln
Step 2. Divide by token count
\"12/6=2\"
Step 3. Apply exponential
\"\\operatorname{PP}=e^2\\approx7.389056\"
Final answer. Perplexity≈7.389056.

Question. A language model's average negative log₂ probability is 3 bits per token. Find its perplexity.

Given

Cross-entropy H=3 bits/token. Log base is 2.

To find

PP.

Formula and why it applies

\"\\operatorname{PP}=2^H\"

The inverse operation of log₂ is raising 2 to that power; use e only for natural logs.

Solution

Step 1. Identify the log base

Bits mean base 2.

Step 2. Raise the base to H
\"\\operatorname{PP}=2^3=8\"
Final answer. Perplexity = 8.

Yaad rakho: ln use kiya toh e se wapas; log₂ use kiya toh 2 se wapas.

"},{"id":"laplace-basics","title":"Laplace smoothing: why both numerator and denominator change","html":"

Smoothing reallocates some probability mass to events that received zero or unreliable estimates from finite training data. Laplace, or add-one, smoothing adds one pseudo-count to every possible next outcome. If there are V possible outcomes, the total added count is V, so adding 1 only to the numerator would produce probabilities that do not sum to 1.

\"P_{\\mathrm{Laplace}}(w)=\\frac{C(w)+1}{N+V}\"
\"P_{\\mathrm{Laplace}}(w\\mid

For unigram smoothing, N is the total training token count. For bigram smoothing, C(h) is the count of the particular history. In both cases V is the number of outcomes being predicted, not the number of training sentences and not the number of distinct bigrams. When end is a possible prediction, include it in V; the start marker is usually context only.

Observed counts: 3, 1, 0
→
Add one to every outcome: 4, 2, 1
→
New total: 7
→
Smoothed probabilities: 4/7, 2/7, 1/7

Question. A fixed unigram vocabulary is {a,b,c}. The training counts are 3,1,0. Compute unsmoothed and Laplace-smoothed probabilities.

Given

C(a)=3, C(b)=1, C(c)=0; N=4; V=3.

To find

The complete distributions.

Formula and why it applies

\"P(w)=C(w)/N,\\qquad

The zero-count c is still in the fixed vocabulary, so it must receive one pseudo-count and contribute to V.

Solution

Step 1. Compute unsmoothed values
WordUnsmoothed
a3/4 = 0.75
b1/4 = 0.25
c0/4 = 0
Step 2. Calculate the new denominator
\"N+V=4+3=7\"
Step 3. Add one to each numerator
\"C(a)+1=4,\\quad
Step 4. Divide
WordSmoothed
a4/7 ≈ 0.571429
b2/7 ≈ 0.285714
c1/7 ≈ 0.142857
Step 5. Confirm the total
\"4/7+2/7+1/7=1\"
Final answer. Smoothed probabilities: a=4/7, b=2/7, c=1/7. Common a loses mass; unseen c gains mass.

Question. After history I, a corpus records like three times, study once and sleep zero times. The allowed next vocabulary is {like,study,sleep}. Compute all Laplace-smoothed bigram probabilities.

Given

C(I,like)=3; C(I,study)=1; C(I,sleep)=0. History count C(I)=4. V=3.

To find

P_L(like|I), P_L(study|I), P_L(sleep|I).

Formula and why it applies

\"P_L(w\\mid

This is smoothing within the I row; use its total 4, not the global corpus token count.

Solution

Step 1. Form the common denominator
\"C(I)+V=4+3=7\"
Step 2. Calculate like
\"P_L(like\\mid
Step 3. Calculate study
\"P_L(study\\mid
Step 4. Calculate sleep
\"P_L(sleep\\mid
Final answer. The row is (4/7, 2/7, 1/7). An unseen bigram is no longer forced to zero.

Question. A fixed vocabulary has five possible next tokens. A history h is unseen in training. What is its add-one probability for each next token?

Given

C(h)=0; C(h,w)=0 for each outcome; V=5.

To find

P_L(w|h) for each token.

Formula and why it applies

\"P_L(w\\mid

All outcomes receive equal pseudo-counts because the training data gives no preference for this history.

Solution

Step 1. Add the pseudo-count
\"0+1=1\"
Step 2. Form the denominator
\"0+5=5\"
Step 3. Divide and sum
\"P_L(w\\mid
Final answer. Every outcome gets 0.2 under this fixed-vocabulary model.

Vocabulary warning. Add-one does not automatically define probabilities for every word in the world. Unknown words require a declared vocabulary/UNK convention. If a question supplies V, use that stated V rather than inventing additional tokens.

Yaad rakho: Add-one: upar +1, neeche +V; bigram mein neeche history ka count.

"},{"id":"smoothed-sentence","title":"A complete unseen-bigram question from corpus to final answer","html":"

Question. Consider the corpus “I love to code in Python. I love to learn new things. I code every day.” Obtain the bigram probabilities needed for “I love Python”. Explain the zero-probability issue, then use Laplace smoothing to score it. Treat each sentence separately, preserve case, remove punctuation, and include an end token among predicted outcomes.

Given

Three sentences padded with one start/end marker. Distinct words: I, love, to, code, in, Python, learn, new, things, every, day. With </s>, V=12. Start is context only.

To find

Unsmoothed and smoothed P(I love Python </s> | <s>).

Formula and why it applies

\"P(w\\mid

The test contains an unseen transition love→Python. Smoothing assigns probability to that transition and renormalizes every required history row.

Solution

Step 1. Count the contexts
HistoryCount
<s>3
I3
love2
Python1
Step 2. Count each required pair
PairCount
<s> I3
I love2
love Python0
Python </s>1
Step 3. Compute the unsmoothed product
\"P=\\frac33\\times\\frac23\\times\\frac02\\times\\frac11=0\"

Both words love and Python were seen, but their adjacent pair was not. This is an unseen bigram, not an unknown-word case.

Step 4. Smooth the first event
\"P_L(I\\mid\\langle
Step 5. Smooth the second event
\"P_L(love\\mid
Step 6. Smooth the unseen event
\"P_L(Python\\mid
Step 7. Smooth the end event
\"P_L(\\langle/s\\rangle\\mid
Step 8. Multiply the smoothed factors
\"P_L=\\frac4{15}\\times\\frac3{15}\\times\\frac1{14}\\times\\frac2{13}\"
\"P_L=\\frac{24}{40950}=\\frac4{6825}\\approx0.000586081\"
Final answer. Unsmoothed P=0. With the stated V=12 and boundaries, Laplace-smoothed P≈0.000586081.

If an exam states a different convention—for example excluding boundary tokens from the entire calculation—recompute both V and the set of probability factors consistently. Do not combine a boundary-aware numerator with a boundary-free denominator.

"},{"id":"adjusted-count-discount","title":"Adjusted counts and relative discounting","html":"

The slides also express smoothing through an adjusted count c*. Instead of writing the smoothed probability with denominator N+V, they rescale its numerator so the denominator can remain N. Algebraically these are equivalent:

\"c^*=(c+1)\\frac{N}{N+V},\\qquad
\"d=\\frac{c^*}{c}\\quad(c>0)\"

The relative discount d compares adjusted and original counts. It is undefined when c=0 because division by zero is undefined. Add-one smoothing moves the distribution toward uniformity: frequent events are reduced, but some rare seen events may increase. It is not correct to claim that every nonzero count must decrease under Laplace smoothing.

Question. For N=20 and V=5, a word has original count c=10. Find its adjusted count, smoothed probability and relative discount.

Given

N=20; V=5; c=10.

To find

c*, P_L and d.

Formula and why it applies

\"c^*=(c+1)N/(N+V),\\quad

These expressions rewrite add-one smoothing without changing its probability.

Solution

Step 1. Form the new total
\"N+V=20+5=25\"
Step 2. Find the adjusted count
\"c^*=(10+1)\\frac{20}{25}=11(0.8)=8.8\"
Step 3. Find the probability
\"P_L=8.8/20=0.44\"
Step 4. Find the relative discount
\"d=8.8/10=0.88\"
Final answer. c*=8.8, P_L=0.44, d=0.88. The original probability 0.5 is reduced.

Question. Under N=20, V=5, a rare word occurs once. Compute its adjusted count and compare the original and smoothed probabilities.

Given

N=20; V=5; c=1.

To find

c*, original P and smoothed P_L.

Formula and why it applies

\"c^*=(c+1)N/(N+V)\"

The same smoothing rule may increase a rare event's probability.

Solution

Step 1. Original probability
\"P=1/20=0.05\"
Step 2. Adjusted count
\"c^*=(1+1)(20/25)=1.6\"
Step 3. Smoothed probability
\"P_L=1.6/20=0.08\"
Final answer. c*=1.6 and P_L=0.08, which exceeds the original 0.05.

Question. For N=20 and V=5, an in-vocabulary word has count zero. Find its adjusted count and smoothed probability. Can a relative discount be calculated?

Given

c=0; N=20; V=5.

To find

c*, P_L and whether d exists.

Formula and why it applies

\"c^*=(c+1)N/(N+V),\\qquad

The pseudo-count creates positive mass for the unseen vocabulary item.

Solution

Step 1. Adjusted count
\"c^*=(0+1)(20/25)=0.8\"
Step 2. Smoothed probability
\"P_L=0.8/20=0.04\"
Step 3. Relative discount

d would be 0.8/0. That is undefined, so do not report a finite discount.

Final answer. c*=0.8 and P_L=0.04; relative discount is undefined for c=0.
"},{"id":"class-perplexity-characters","title":"Character-model perplexity: keep exact probabilities","html":"

Question. A unigram character model gives P(A)=0.25, P(B)=0.50, P(C)=0.25. Find the perplexity of AAA and ABC. Then find both perplexities when the three characters are equally likely.

Given

Each test string contains 3 scored characters, so N=3. Initial distribution 0.25,0.50,0.25; comparison distribution 1/3,1/3,1/3.

To find

Both sequence probabilities and perplexities under each distribution.

Formula and why it applies

\"P(c_1c_2c_3)=P(c_1)P(c_2)P(c_3),\\qquad

Characters, not words, are the model's prediction units in this question. Use exact 1/3 for uniform probabilities, rather than rounded 0.33.

Solution

Step 1. Score AAA
\"P(AAA)=0.25^3=\\frac1{64}=0.015625\"
Step 2. Find AAA perplexity
\"PP(AAA)=64^{1/3}=4\"
Step 3. Score ABC
\"P(ABC)=0.25(0.5)(0.25)=\\frac1{32}=0.03125\"
Step 4. Find ABC perplexity
\"PP(ABC)=32^{1/3}\\approx3.174802\"
Step 5. Change to an exactly uniform model
\"P(AAA)=P(ABC)=(1/3)^3=1/27\"
Step 6. Find both uniform-model perplexities
\"PP=27^{1/3}=3\"
Final answer. Original model: AAA→4, ABC→3.174802. Uniform model: both→3. These values follow the stated formulas without intermediate rounding.
"}]},{"id":"u3-smoothing-extensions","title":"Faculty topics: interpolation and Stupid Backoff","unit":3,"mode":"tee","order":100,"scope":"Unit 3 PDF p147 names these methods. Neither supplied TEE paper asks their numerical calculation; the worked values below are authored examples.","sections":[{"id":"interpolation","title":"N-gram interpolation: weighted evidence from several histories","html":"

Faculty topic; not an observed numerical PYQ. Unit 3 PDF p147 names these methods. Neither supplied TEE paper asks their numerical calculation; the worked values below are authored examples.

Interpolation combines several context lengths for every prediction. A trigram estimate is specific but may have little evidence. Bigram and unigram estimates use less context and can support unseen longer sequences. A weighted mixture combines their information.

\"P_{\\mathrm{mix}}(w\\mid
\"\\lambda_1,\\lambda_2,\\lambda_3\\ge0,\\qquad\\lambda_1+\\lambda_2+\\lambda_3=1.\"

Each component must be a normalized distribution over the same vocabulary. Then summing the mixture over all possible words gives λ₁+λ₂+λ₃=1. A zero higher-order count can be rescued by positive lower-order evidence. Unlike simple backoff, interpolation can combine all levels even when the trigram was seen.

Question. For predicting tea after I like, suppose trigram, bigram and unigram probabilities are 0, 0.2 and 0.1. Use weights λ₃=0.5, λ₂=0.3 and λ₁=0.2 to calculate the interpolated probability. Repeat for coffee with probabilities 0.4, 0.1 and 0.05. [Authored worked example]

Answer

Given

The three distributions use one vocabulary. Nonnegative weights sum to 1. These are supplied probabilities; no count estimation is needed.

To find

Pmix(tea | I,like) and Pmix(coffee | I,like).

Formula and why it applies

\"P_{mix}=\\lambda_3P_3+\\lambda_2P_2+\\lambda_1P_1\"

We add weighted alternatives from models, rather than multiply sequential word events.

Solution

Step 1. Check the weights
\"0.5+0.3+0.2=1.\"
Step 2. Calculate the three tea contributions
\"0.5(0)=0,\\qquad0.3(0.2)=0.06,\\qquad0.2(0.1)=0.02.\"
Step 3. Add the tea contributions
\"P_{mix}(tea)=0+0.06+0.02=0.08.\"
Step 4. Calculate the coffee contributions
\"0.5(0.4)=0.20,\\qquad0.3(0.1)=0.03,\\qquad0.2(0.05)=0.01.\"
Step 5. Add and compare
\"P_{mix}(coffee)=0.20+0.03+0.01=0.24.\"

Coffee gets three times tea’s probability in this model. Other vocabulary outcomes carry the remaining mass; these two displayed entries are not the whole distribution.

Final answer. Tea=0.08 and coffee=0.24. A zero trigram probability does not force a zero mixture.

Yaad rakho: Interpolation = sab context-lengths ka weighted mix. Word-sequence product aur model-mixture sum alag operations hain.

"},{"id":"stupid-backoff","title":"Backoff and the important score-versus-probability distinction","html":"

Faculty topic; not an observed numerical PYQ. Unit 3 PDF p147 names these methods. Neither supplied TEE paper asks their numerical calculation; the worked values below are authored examples.

Backoff uses a shorter history when the longer-history estimate lacks evidence. A properly normalized backoff model discounts some seen-event probability and allocates the remaining mass across unseen continuations using lower-order probabilities and normalization factors.

Stupid Backoff is a specifically named efficient scoring method in the slides’ list. It uses an observed n-gram relative frequency when available; otherwise it multiplies the shorter-history score by a fixed penalty, commonly 0.4. Its output is generally an unnormalized score, not a conditional probability distribution. Do not insert raw Stupid Backoff scores into standard probability-based perplexity and call the result a normalized-model perplexity.

\"S(w\\mid

Here h′ drops the oldest history word; α is the backoff penalty. At the unigram base, use C(w)/N. The letter α here is a penalty, not the forward-HMM variable introduced later.

Question. A bigram scorer has complete vocabulary {a,b}. After h, C(h,a)=2 and C(h,b)=0. The unigram probabilities are P(a)=P(b)=0.5. Apply Stupid Backoff with penalty 0.4 and show whether the scores form a normalized row. [Authored worked example]

Answer

Given

C(h)=2. The a bigram is observed; the b bigram is unseen. Unigram probabilities are supplied.

To find

S(a|h), S(b|h), their sum, and whether they are probabilities.

Formula and why it applies

Use relative frequency for an observed pair and α times the unigram base for an unseen pair. The method does not discount a’s observed score in this example.

Solution

Step 1. Score the observed pair
\"S(a\\mid
Step 2. Back off for the unseen pair
\"S(b\\mid
Step 3. Add the complete row
\"1+0.2=1.2\\ne1.\"
Step 4. Interpret the failure of normalization

The scores can rank a above b, but the raw row cannot be called a probability distribution. Merely labelling the method “smoothing” does not change this.

Final answer. Scores are 1 and 0.2; their sum is 1.2. They are useful ranking scores under this rule, not normalized probabilities.
MethodOperationNormalization
Add-one/add-kAdjust all allowed outcome counts and the totalNormalized with a fixed vocabulary
InterpolationMix normalized distributions using nonnegative weights summing to 1Normalized
Normalized backoffDiscount seen events and distribute remaining mass over unseen eventsNormalized when its factors are correctly computed
Stupid BackoffKeep seen relative frequencies; penalize shorter-history scores for unseen n-gramsGenerally not normalized

Scope: Unit 3 slide 147 lists Stupid Backoff and n-gram interpolation; the class calculations emphasize Laplace. These short worked extensions explain the listed methods without claiming extra official exam weight.

"}]},{"id":"m2-pos-tagging","title":"POS tagging and its learning approaches","unit":4,"scope":"Unit 4, pages 3–46","sections":[{"id":"word-classes-pos","title":"Parts of speech and contextual ambiguity","html":"

Part-of-speech (POS) tagging assigns a grammatical category to every word token in context, such as noun, verb, adjective or determiner. It is a sequence-labeling task: the input is a sequence of words, and the output is a tag for each word. A tag describes the word's role in this sentence, not its meaning in every possible sentence.

TagRoleExample in context
NN / NNSSingular / plural common nouna book / two books
NNPProper nounMary, Mumbai
VB / VBD / VBZBase / past / third-person singular verbread / read / reads
JJAdjectivethe red book
RBAdverbruns quickly
DTDeterminerthe, a, this
INPreposition or subordinating conjunctionon the table
PRPPersonal pronounshe, they
MDModal auxiliarycan, will, should

Open classes readily accept new content words: nouns, verbs, adjectives and adverbs. Examples include chatbot and livestream. Closed classes contain comparatively stable grammatical/function words: determiners, pronouns, prepositions and conjunctions. These are tendencies, not a claim that language never changes.

Input: Book that flight
→
Context: an imperative request
→
Output: Book/VB that/DT flight/NN

Compare “hand me that book”: book is a noun here. Similarly, “I can swim” has can/MD, while “the can is empty” has can/NN. The spelling stays the same; neighboring words and sentence structure change the correct tag. This is POS ambiguity.

SentenceTagged interpretationReason
They watch birds.They/PRP watch/VBP birds/NNSwatch is the action.
The watch stopped.The/DT watch/NN stopped/VBDwatch is the object that stopped.
She has a light bag.She/PRP has/VBZ a/DT light/JJ bag/NNlight describes bag.
Light the lamp.Light/VB the/DT lamp/NNLight is an imperative verb.

POS tags support parsing, named entity recognition, question answering, sentiment analysis and word-sense disambiguation. They do not by themselves identify every entity or fully resolve meaning.

Yaad rakho: Word ka naam same ho sakta hai; sentence mein uska kaam tag decide karta hai.

"},{"id":"tagger-methods","title":"Lexical, rule-based, stochastic, Brill and neural taggers","html":"

A lexical baseline gives each known word its most frequent training tag. It is simple and often useful, but ignores context. A rule-based tagger uses a lexicon of possible tags plus manually designed contextual or morphological rules to resolve ambiguity. For example, after a determiner, a noun reading of “can” may be preferred in “the can”.

Rules are usually heuristics with exceptions. “A red car” shows that the word immediately after an article is not always a noun. The suffix -ed does not make “red” a past-tense verb. A robust rule system uses multiple linguistic constraints instead of treating such shortcuts as universal laws.

MethodHow it chooses tagsStrengthLimitation
Most frequent / lexicalMost common training tag for the wordSimple baselineMisses contextual changes and unknown words
Rule-basedLexicon plus handwritten rulesReadable, controllable linguistic decisionsExpert effort; rules may conflict or need updates
StochasticScores tag sequences using estimated probabilitiesUses corpus evidence and contextSparse/unseen events; depends on training domain
Transformation-based / BrillStart with baseline, then apply learned correction rulesReadable learned correctionsNeeds tagged training data and suitable rule templates
NeuralLearns contextual representations and sequence predictionsCan exploit richer contextNeeds training resources and data; less directly interpretable

Transformation-based error-driven learning (Brill tagging) combines a simple starting assignment with learned correction rules. First tag the training corpus with a baseline. Compare against the gold tags. Generate candidate correction rules from templates, such as “change MD to NN after DT”. Apply the rule that fixes the most errors net of newly introduced errors; repeat until further improvement is too small. At test time apply the learned ordered rule list.

Baseline: The/DT can/MD was/VBD crushed/VBN
→
Context rule: change can from MD to NN after DT
→
Result: The/DT can/NN was/VBD crushed/VBN

Rule-based versus Brill: a purely rule-based system depends on linguists specifying its rules; a Brill tagger learns which transformations improve a labeled corpus. A stochastic tagger such as an HMM finds a high-probability tag sequence using transition and emission probabilities. CRFs and neural sequence taggers are other learning approaches; they do not have to make the same HMM independence assumptions.

Exam-ready answer. POS tagging assigns a grammatical tag to each word in context. A rule-based tagger uses lexicons and linguistic rules, whereas a stochastic tagger chooses tags through probabilities estimated from a corpus. For example, can is MD in “I can swim” but NN in “the can”; context resolves the ambiguity.

"},{"id":"generative-discriminative","title":"Generative versus discriminative models and Bayes' rule","html":"

Let x be the observed words and y be the desired tag sequence. A generative model describes their joint distribution P(x,y), often as P(y)P(x|y). A discriminative model learns P(y|x) directly or learns a decision boundary that separates possible labels. Both kinds can be trained with labeled data; “generative” does not mean “always unsupervised”.

\"P(y\\mid
QuantityMeaningPOS interpretation
P(y)PriorHow plausible is this tag or tag sequence before seeing the words?
P(x|y)LikelihoodHow plausible are these words under that tag sequence?
P(x)EvidenceTotal probability of the observed words across possible labels.
P(y|x)PosteriorProbability of the tags after seeing the words.
AspectGenerativeDiscriminative
ModelsJoint P(x,y), or prior × likelihoodConditional P(y|x) or decision boundary
Classification examplesNaive BayesLogistic regression, SVM
Sequence-labeling examplesHMMLinear-chain CRF
IntuitionModel how a label could generate the observationModel which label fits the observation

Question. A token has two possible tags, noun N and verb V. P(N)=0.6, P(V)=0.4, P(book|N)=0.2 and P(book|V)=0.5. Calculate P(N|book) and P(V|book).

Given

Two tags are exhaustive. Priors: 0.6,0.4. Word likelihoods: 0.2,0.5.

To find

Normalized posterior probabilities.

Formula and why it applies

\"P(t\\mid

Bayes' rule combines prior and likelihood. Normalization is needed to convert scores into posterior probabilities.

Solution

Step 1. Calculate the noun score
\"0.2\\times0.6=0.12\"
Step 2. Calculate the verb score
\"0.5\\times0.4=0.20\"
Step 3. Find the evidence
\"P(book)=0.12+0.20=0.32\"
Step 4. Normalize the noun score
\"P(N\\mid
Step 5. Normalize the verb score
\"P(V\\mid
Final answer. Verb is more probable in this simplified model: 0.625 versus 0.375.

Question. Two candidate tags have prior-likelihood products 0.06 and 0.14. Find their normalized probabilities.

Given

Unnormalized scores s₁=0.06 and s₂=0.14; these are the only candidates.

To find

The two posterior probabilities.

Formula and why it applies

\"P_i=s_i/(s_1+s_2)\"

A common denominator converts the candidate scores to a distribution summing to one.

Solution

Step 1. Add scores
\"0.06+0.14=0.20\"
Step 2. Normalize first
\"0.06/0.20=0.3\"
Step 3. Normalize second
\"0.14/0.20=0.7\"
Final answer. Posterior probabilities are 0.3 and 0.7.

Yaad rakho: Generative: tag se word ka model. Discriminative: word se tag ka decision.

"}]},{"id":"m2-hmm","title":"Markov chains and Hidden Markov Models","unit":4,"scope":"Unit 4, pages 47–73","sections":[{"id":"markov-chain","title":"Markov chains: read a transition table and multiply a path","html":"

A Markov chain is a probabilistic model of a sequence of states. Under the first-order Markov assumption, the next state's probability depends on the current state rather than the complete previous history. The initial probability tells us how likely each first state is. A transition probability tells us how likely a move from one state to another is.

\"P(s_t\\mid
\"P(s_1,\\ldots,s_T)=P(s_1)\\prod_{t=2}^{T}P(s_t\\mid

For T states in a path, use one initial factor and T−1 transition factors. A row-oriented transition matrix must have each complete row sum to 1. Always read the row/column labels; some exam diagrams use a transposed arrangement.

Current → nextRainDry
Rain0.30.7
Dry0.20.8

Initial probabilities: P(Rain)=0.4, P(Dry)=0.6. Rows describe the current state; columns describe the next state.

Question. For the Rain/Dry Markov chain with initial P(Rain)=0.4 and P(Dry)=0.6, and transitions Rain→Rain=0.3, Rain→Dry=0.7, Dry→Rain=0.2, Dry→Dry=0.8, find the probability of Dry, Dry, Rain, Rain.

Given

Current → nextRainDry
Rain0.30.7
Dry0.20.8

Initial probabilities: P(Rain)=0.4, P(Dry)=0.6. Rows describe the current state; columns describe the next state.

To find

P(Dry,Dry,Rain,Rain).

Formula and why it applies

\"P(D,D,R,R)=P(D)P(D\\mid

Each step conditions on the immediately preceding state. The sequence contains four states, so it needs one initial plus three transition factors.

Solution

Step 1. Start in Dry
\"P(D)=0.6\"
Step 2. Move Dry to Dry
\"0.6\\times0.8=0.48\"
Step 3. Move Dry to Rain
\"0.48\\times0.2=0.096\"
Step 4. Move Rain to Rain
\"0.096\\times0.3=0.0288\"
Final answer. Path probability = 0.0288.

Question. For initial probabilities Rain=0.4, Dry=0.6 and transition rows Rain=(0.3,0.7), Dry=(0.2,0.8) in column order Rain,Dry, find the probability of Rain, Dry, Rain.

Given

Current → nextRainDry
Rain0.30.7
Dry0.20.8

Initial probabilities: P(Rain)=0.4, P(Dry)=0.6. Rows describe the current state; columns describe the next state.

To find

P(Rain,Dry,Rain).

Formula and why it applies

\"P(R,D,R)=P(R)P(D\\mid

Do not replace Dry→Rain with Rain→Dry: the two directional probabilities differ.

Solution

Step 1. Initial Rain
\"P(R)=0.4\"
Step 2. First transition
\"P(D\\mid
Step 3. Second transition
\"P(R\\mid
Final answer. Path probability = 0.056.

Question. For the same fully specified chain—initial Rain=0.4, Dry=0.6; Rain→Rain=0.3, Rain→Dry=0.7, Dry→Rain=0.2, Dry→Dry=0.8—find P(Dry,Dry). Also find the conditional probability of next Dry given current Dry.

Given

Current → nextRainDry
Rain0.30.7
Dry0.20.8

Initial probabilities: P(Rain)=0.4, P(Dry)=0.6. Rows describe the current state; columns describe the next state.

To find

The two-state joint probability and the one-step conditional probability.

Formula and why it applies

\"P(D,D)=P(D)P(D\\mid

The joint path includes an uncertain initial state. The conditional transition assumes the current state is already Dry.

Solution

Step 1. Read the conditional transition
\"P(D\\mid
Step 2. Include the initial state for the joint path
\"P(D,D)=0.6\\times0.8=0.48\"
Final answer. P(Dry,Dry)=0.48; P(next Dry|current Dry)=0.8. These answer different questions.

Yaad rakho: Path: pehle start probability, phir har arrow ki probability.

"},{"id":"hmm-components","title":"Hidden states, observations, transitions and emissions","html":"

A Hidden Markov Model (HMM) is a probabilistic sequence model in which the underlying state sequence is unobserved, while each state emits an observed item. For POS tagging, words are observed and their grammatical tags are hidden. The task is to infer a suitable tag sequence from the words.

Hidden: Noun
→
Hidden: Verb
→
Hidden: Noun
Observed: birds
→
Observed: eat
→
Observed: seeds

Horizontal state-to-state links are transitions. The relation from each hidden state to its observed word is an emission. A transition P(V|N) asks how often a verb tag follows a noun tag. An emission P(eat|V) asks how often the observed word eat appears among verb-tagged tokens.

ComponentNotationMeaning
State setS={N,V,…}Possible hidden tags.
Observation vocabularyO={birds,eat,seeds,…}Visible word tokens.
Initial distributionπⱼ=P(t₁=j)Probability of starting in tag j.
Transition matrixaᵢⱼ=P(tₖ=j|tₖ₋₁=i)Probability of next tag j after i.
Emission matrixbⱼ(w)=P(w|tag j)Probability that state j emits word w.

Two basic assumptions are used: the current hidden state depends only on the previous hidden state, and the current observation depends only on the current hidden state. A standard first-order HMM therefore does not explicitly condition an emitted word on the whole sentence.

\"P(t_{1:T},w_{1:T})=\\pi_{t_1}b_{t_1}(w_1)\\prod_{k=2}^{T}a_{t_{k-1},t_k}b_{t_k}(w_k)\"

Read the formula as start × first emission × transition × emission × transition × emission…. There are T emission factors and T−1 transitions, plus one initial factor. If the question supplies an explicit end-state probability, multiply that too. Do not invent an end factor when the model has none.

Three different questions. (1) Scoring a specified tag path uses the joint product above. (2) Finding the probability of the observed words sums scores over all possible tag paths. (3) Finding the best tag path maximizes the path score, typically using Viterbi. A joint path score is not a normalized posterior P(tags|words).

Yaad rakho: Transition = tag se agla tag. Emission = tag se visible word.

"},{"id":"transition-emission-estimation","title":"Estimate transition and emission probabilities from tagged data","html":"
\"P(t'\\mid
\"P(w\\mid

In a transition denominator, count the previous tag's outgoing events. In an emission denominator, count all tokens bearing that tag. With explicit end transitions the totals align conveniently. Keep the two numerator types separate: tag→tag counts are not word→tag counts.

Question. A tagged corpus contains 116454 determiner tags DT, each counted with an outgoing event. DT is followed by NN 56509 times. Estimate P(NN|DT).

Given

C(DT)=116454; C(DT,NN)=56509.

To find

The DT→NN transition probability.

Formula and why it applies

\"P(NN\\mid

The denominator is the number of previous DT contexts because DT is the condition.

Solution

Step 1. Identify the transition count

DT followed by NN occurs 56509 times.

Step 2. Identify the conditional total

There are 116454 DT contexts.

Step 3. Divide
\"P(NN\\mid
Final answer. P(NN|DT)≈0.485247.

Question. The tag VBZ occurs 21627 times, and 10073 of those tokens are the word “is”. Estimate P(is|VBZ).

Given

C(VBZ)=21627; C(is,VBZ)=10073.

To find

The emission probability of is from VBZ.

Formula and why it applies

\"P(is\\mid

The condition is the tag VBZ; we ask which word it emits.

Solution

Step 1. Choose the word-tag count

The required word is is with tag VBZ: count 10073.

Step 2. Choose the tag total

All VBZ-tagged word tokens: count 21627.

Step 3. Divide
\"P(is\\mid
Final answer. P(is|VBZ)≈0.465760. This is different from P(VBZ|is).

Question. Use the tagged sentences “dogs/N chase/V cats/N” and “cats/N sleep/V”. Estimate the initial tag probabilities, all nonzero transitions including END, and emissions.

Given

Corpus: START dogs/N chase/V cats/N END; START cats/N sleep/V END.

To find

Initial distribution, transition rows and emission rows.

Formula and why it applies

\"\\pi_t=C(START,t)/2,\\quad

The corpus explicitly tells us the tags, so we can count training frequencies directly.

Solution

Step 1. Count tag tokens

N occurs 3 times: dogs, cats, cats. V occurs 2 times: chase, sleep.

Step 2. Count sentence starts

Both sentences start with N.

\"\\pi_N=2/2=1,\\quad
Step 3. Count transitions
PairCount
N→V2
N→END1
V→N1
V→END1
Step 4. Normalize transition rows
CurrentTo NTo VTo END
N02/31/3
V1/201/2
Step 5. Count and normalize emissions
Tagdogscatschasesleep
N1/32/300
V001/21/2
Final answer. π=(1,0); transitions and emissions are given in the completed tables. Each complete probability row sums to 1.

Yaad rakho: Tag→tag table aur tag→word table ko mix mat karna.

"},{"id":"hot-cold-paths","title":"HMM path probabilities: three complete numerical examples","html":"

The following model uses hidden weather states Hot (H) and Cold (C). Observations are numbers 1, 2 and 3. These numbers are observation labels, not quantities to multiply directly. Their emission probabilities come from the table.

Initial stateProbability
H0.8
C0.2
From → toHC
H0.70.3
C0.40.6
Emission stateObservation 1Observation 2Observation 3
H0.20.40.4
C0.50.40.1

Question. An HMM has π(H)=0.8, π(C)=0.2; H→H=0.7, H→C=0.3, C→H=0.4, C→C=0.6. Emissions for observations (1,2,3) are H:(0.2,0.4,0.4) and C:(0.5,0.4,0.1). Find the joint probability of hidden path H,C,H and observations 1,3,1.

Given

Hidden path H→C→H. Observations 1→3→1. Model probabilities are specified in the question. There is no END factor.

To find

P(H,C,H; 1,3,1).

Formula and why it applies

\"P=\\pi_H

There are three hidden states and observations: three emissions, two transitions, and one initial probability.

Solution

Step 1. Start Hot and emit 1
\"0.8\\times0.2=0.16\"
Step 2. Transition Hot to Cold
\"0.16\\times0.3=0.048\"
Step 3. Cold emits 3
\"0.048\\times0.1=0.0048\"
Step 4. Transition Cold to Hot
\"0.0048\\times0.4=0.00192\"
Step 5. Hot emits the last 1
\"0.00192\\times0.2=0.000384\"
Final answer. Joint probability = 0.000384 = 3.84×10⁻⁴.

Question. For π(H)=0.8, π(C)=0.2; H→H=0.7, H→C=0.3, C→H=0.4, C→C=0.6; emissions H:(0.2,0.4,0.4), C:(0.5,0.4,0.1) for labels (1,2,3), find the joint probability of path H,H,H and observations 1,3,1.

Given

State path H→H→H, observations 1→3→1; no END factor.

To find

P(H,H,H;1,3,1).

Formula and why it applies

\"P=\\pi_H

The state remains Hot; each transition still contributes its probability 0.7.

Solution

Step 1. Initial state and first emission
\"0.8(0.2)=0.16\"
Step 2. First self-transition
\"0.16(0.7)=0.112\"
Step 3. Emit observation 3
\"0.112(0.4)=0.0448\"
Step 4. Second self-transition
\"0.0448(0.7)=0.03136\"
Step 5. Emit final observation 1
\"0.03136(0.2)=0.006272\"
Final answer. Joint probability = 0.006272. A self-transition is not automatically probability 1.

Question. For π(H)=0.8, π(C)=0.2; transitions H:(0.7,0.3), C:(0.4,0.6) in next-state order H,C; emissions H:(0.2,0.4,0.4), C:(0.5,0.4,0.1) for observations 1,2,3, find the score of path C,C,H with observations 1,3,1.

Given

Hidden path C→C→H. Observation sequence 1→3→1. No end-state factor.

To find

The joint path-and-observation probability.

Formula and why it applies

\"P=\\pi_C

The initial state is Cold, so use π(C), not π(H). Each emission is selected from the state at the same position.

Solution

Step 1. Start Cold and emit 1
\"0.2(0.5)=0.1\"
Step 2. Stay Cold
\"0.1(0.6)=0.06\"
Step 3. Emit 3 from Cold
\"0.06(0.1)=0.006\"
Step 4. Move Cold to Hot
\"0.006(0.4)=0.0024\"
Step 5. Emit 1 from Hot
\"0.0024(0.2)=0.00048\"
Final answer. Joint probability = 0.00048.

Yaad rakho: Har column par: state ka emission. Columns ke beech: transition.

"},{"id":"tagging-corpus-case","title":"POS HMM worked corpus: Mary, Jane, can, will and spot","html":"

The class example contains words that can be either names or ordinary verbs/modal auxiliaries. The term is modal (M), not “model”. To avoid guessing tags from spelling, the training corpus below supplies them explicitly and lowercases words for the emission vocabulary.

Training sentenceGold tags
mary jane can see willN N M V N
spot will see maryN M V N
will jane spot maryM N V N
mary will pat spotN M V N

There are 9 noun tokens, 4 modal tokens and 4 verb tokens. Each sentence includes a start and end transition. For nouns, the next event can be another noun, modal, verb or END.

Current → nextNMVEND
START3/41/400
N1/93/91/94/9
M1/403/40
V4/4000
Tag → wordmaryjanecanseewillspotpat
N4/92/9001/92/90
M001/403/400
V0002/401/41/4

Question. Using the fully tagged four-sentence corpus above, calculate the joint score of “will jane spot mary” assigned tags M,N,V,N, including END.

Given

Required factors from the tables: START→M=1/4; emit will|M=3/4; M→N=1/4; jane|N=2/9; N→V=1/9; spot|V=1/4; V→N=1; mary|N=4/9; N→END=4/9.

To find

The specified tag-path score, including words and sentence end.

Formula and why it applies

\"P=\\pi_M

Both transitions and emissions are needed. A tag transition alone cannot tell us whether a particular word is plausible.

Solution

Step 1. Start and emit will
\"\\frac14\\times\\frac34=\\frac3{16}\"
Step 2. Move to N and emit jane
\"\\frac3{16}\\times\\frac14\\times\\frac29=\\frac1{96}\"
Step 3. Move to V and emit spot
\"\\frac1{96}\\times\\frac19\\times\\frac14=\\frac1{3456}\"
Step 4. Move to N and emit mary
\"\\frac1{3456}\\times1\\times\\frac49=\\frac1{7776}\"
Step 5. End the sentence
\"\\frac1{7776}\\times\\frac49=\\frac1{17496}\\approx0.0000571559\"
Final answer. The joint score is 1/17496 ≈ 0.0000571559.

Question. For the same tagged corpus and tables, explain the unsmoothed score of “will can spot mary” assigned tags M,V,N,N.

Given

The proposed second word is can with hidden tag V. In the training corpus, can occurs only with M, so b_V(can)=0.

To find

Whether the complete path score is zero and what this means.

Formula and why it applies

\"P(\\text{path,words})=(\\text{other

A product containing one zero factor is zero even if every other factor is positive.

Solution

Step 1. Find the critical emission
\"b_V(can)=C(can,V)/C(V)=0/4=0\"
Step 2. Use the full-product consequence
\"P(\\text{path,words})=0\"
Step 3. Interpret correctly

This unsmoothed corpus model has no evidence for can as a verb. It does not prove that a verb usage of can is impossible in English.

Final answer. The unsmoothed path score is zero.
"}]},{"id":"m2-hmm-decoding","title":"Choosing the most likely tag sequence","unit":4,"scope":"Unit 4 page 73: Viterbi motivation, explained with small worked extensions","sections":[{"id":"sequence-count","title":"Why exhaustive search grows exponentially","html":"

If each of T observed words can receive any of N tags, there are N choices at each position. Thus the total number of complete tag sequences is N multiplied by itself T times. This counts possible assignments before grammar rules or zero probabilities eliminate any.

\"\\text{Number

Question. A four-word sentence is tagged using three possible POS tags per word. How many complete tag sequences are possible?

Given

N=3 possible tags per position; T=4 word positions.

To find

The number of tag sequences.

Formula and why it applies

\"M=N^T\"

Every independent position contributes a factor of three choices.

Solution

Step 1. Write one factor per position
\"M=3\\times3\\times3\\times3\"
Step 2. Multiply
\"3\\times3=9,\\quad9\\times3=27,\\quad27\\times3=81\"
Final answer. 81 possible tag sequences.

Question. A five-word sentence has two candidate tags per word. Count its tag sequences.

Given

N=2; T=5.

To find

M.

Formula and why it applies

\"M=N^T\"

The length is the exponent, not a multiplier.

Solution

Step 1. Substitute
\"M=2^5\"
Step 2. Evaluate
\"2\\times2\\times2\\times2\\times2=32\"
Final answer. 32 possible sequences.

Question. Using 45 possible tags, compare the number of tag sequences for sentences of length 2 and 3.

Given

N=45. First T=2, then T=3.

To find

Both counts and the growth factor.

Formula and why it applies

\"M_2=45^2,\\quad

Adding one position multiplies the possibilities by the number of tag choices.

Solution

Step 1. Two words
\"45^2=2025\"
Step 2. Three words
\"45^3=2025\\times45=91125\"
Step 3. Compare
\"91125/2025=45\"
Final answer. 2 words: 2025 paths. 3 words: 91125 paths. One additional word multiplies the count by 45.

The slides introduce Viterbi to avoid enumerating every path. It reuses the best prefix score for each ending state. For a dense first-order HMM, it needs O(TN²) recurrence work instead of enumerating NT complete paths.

Yaad rakho: Choices per word ki power = sentence length.

"},{"id":"viterbi-explained","title":"Viterbi decoding: compare incoming paths, keep the best, trace back","html":"

The Viterbi algorithm finds the most likely hidden-state sequence for a fixed observation sequence. At each position, it keeps the best path ending in each state. This is safe because, under the HMM assumptions, future extensions depend on that ending state, not the discarded details of the prefix.

\"\\delta_1(j)=\\pi_jb_j(o_1)\"
\"\\delta_t(j)=b_j(o_t)\\max_i[\\delta_{t-1}(i)a_{ij}]\"
\"\\psi_t(j)=\\arg\\max_i[\\delta_{t-1}(i)a_{ij}]\"
SymbolMeaning in ordinary words
δₜ(j)Score of the best prefix that ends in state j at position t.
ψₜ(j)Previous state that supplied that best prefix; keep it as a backpointer.
maxChoose the larger candidate score; do not add.
argmaxRemember which previous state won, rather than only its value.
Initialize: start × first emission
→
For each next state: compare incoming scores
→
Multiply winner by this state's emission
→
Keep score + winning predecessor
→
Choose best final state and trace arrows backward

Do not choose the best state independently at each word. The best last state can come from a different best predecessor than a local guess. Backpointers recover one consistent complete path. If scores tie, either tied path is optimal under the model; state the tie.

Question. Use an HMM with π(H)=0.8, π(C)=0.2, transitions H:(0.7,0.3), C:(0.4,0.6) in H,C order, and emissions H:(0.2,0.4,0.4), C:(0.5,0.4,0.1) for observations 1,2,3. Find the most likely hidden path for observations 1,3,1 using Viterbi.

Given

Initial probabilities: π(H)=0.8, π(C)=0.2. Transition matrix: H→H=0.7, H→C=0.3, C→H=0.4, C→C=0.6. Emissions for labels (1,2,3): H=(0.2,0.4,0.4), C=(0.5,0.4,0.1). There is no END-state factor.

To find

The best hidden-state sequence and its joint score.

Formula and why it applies

\"\\delta_1(j)=\\pi_jb_j(o_1),\\quad\\delta_t(j)=b_j(o_t)\\max_i[\\delta_{t-1}(i)a_{ij}]\"

Viterbi keeps two best prefix scores per column, one ending H and one ending C.

Solution

Step 1. Initialize at observation 1
\"\\delta_1(H)=0.8(0.2)=0.16\"
\"\\delta_1(C)=0.2(0.5)=0.10\"
Step 2. At observation 3, calculate the H cell
\"H\\to
\"C\\to
\"\\delta_2(H)=\\max(0.112,0.040)(0.4)=0.0448\"

Backpointer for this H cell: H.

Step 3. At observation 3, calculate the C cell
\"H\\to
\"C\\to
\"\\delta_2(C)=\\max(0.048,0.060)(0.1)=0.006\"

Backpointer for this C cell: C.

Step 4. At final observation 1, calculate H
\"H\\to
\"C\\to
\"\\delta_3(H)=0.03136(0.2)=0.006272\"

Backpointer: H.

Step 5. At final observation 1, calculate C
\"H\\to
\"C\\to
\"\\delta_3(C)=0.01344(0.5)=0.00672\"

Backpointer: H.

Step 6. Select the final state

0.00672 > 0.006272, so the best final state is C.

Step 7. Trace back

Final C came from H at position2. That H came from H at position1. Reading forward gives H,H,C.

Step 8. Verify the recovered complete path
\"0.8(0.2)(0.7)(0.4)(0.3)(0.5)=0.00672\"
Final answer. Best path H→H→C with joint score 0.00672.

Question. For π(H)=0.8, π(C)=0.2; transitions H:(0.7,0.3), C:(0.4,0.6); emissions H:(0.2,0.4,0.4), C:(0.5,0.4,0.1) for labels1,2,3, decode observations 3,1.

Given

Initial probabilities: π(H)=0.8, π(C)=0.2. Transition matrix: H→H=0.7, H→C=0.3, C→H=0.4, C→C=0.6. Emissions for labels (1,2,3): H=(0.2,0.4,0.4), C=(0.5,0.4,0.1). There is no END-state factor.

To find

Best two-state hidden path and score.

Formula and why it applies

\"\\delta_1(j)=\\pi_jb_j(3),\\quad\\delta_2(j)=b_j(1)\\max_i[\\delta_1(i)a_{ij}]\"

The same recurrence applies even though the observation order changed.

Solution

Step 1. Initialize observation 3
\"\\delta_1(H)=0.8(0.4)=0.32,\\quad\\delta_1(C)=0.2(0.1)=0.02\"
Step 2. Calculate the final H
\"\\max(0.32(0.7),0.02(0.4))(0.2)\"
\"=\\max(0.224,0.008)(0.2)=0.0448\"

Winning predecessor H.

Step 3. Calculate the final C
\"\\max(0.32(0.3),0.02(0.6))(0.5)\"
\"=\\max(0.096,0.012)(0.5)=0.048\"

Winning predecessor H.

Step 4. Select and trace

Final C wins with 0.048 and came from H. The path is H,C.

Final answer. Best path H→C, score 0.048.

Question. For π(H)=0.8, π(C)=0.2; transitions H:(0.7,0.3), C:(0.4,0.6); emissions H:(0.2,0.4,0.4), C:(0.5,0.4,0.1) for labels1,2,3, decode observations 2,2.

Given

Initial probabilities: π(H)=0.8, π(C)=0.2. Transition matrix: H→H=0.7, H→C=0.3, C→H=0.4, C→C=0.6. Emissions for labels (1,2,3): H=(0.2,0.4,0.4), C=(0.5,0.4,0.1). There is no END-state factor.

To find

Best path and score.

Formula and why it applies

\"\\delta_1(j)=\\pi_jb_j(2),\\quad\\delta_2(j)=b_j(2)\\max_i[\\delta_1(i)a_{ij}]\"

Both states emit2 with probability 0.4, but their initial and transition probabilities still differ.

Solution

Step 1. Initialize
\"\\delta_1(H)=0.8(0.4)=0.32,\\quad\\delta_1(C)=0.2(0.4)=0.08\"
Step 2. Final H
\"\\max(0.32(0.7),0.08(0.4))(0.4)\"
\"=\\max(0.224,0.032)(0.4)=0.0896\"

Predecessor H.

Step 3. Final C
\"\\max(0.32(0.3),0.08(0.6))(0.4)\"
\"=\\max(0.096,0.048)(0.4)=0.0384\"

Predecessor H.

Step 4. Select and trace

The final H score 0.0896 is larger. Its predecessor is H.

Final answer. Best path H→H with score 0.0896.

Yaad rakho: Har destination ke liye incoming paths compare; winner ka arrow yaad.

"},{"id":"joint-vs-posterior","title":"A path score is not the probability of the observed sentence","html":"

Question. An HMM has π(H)=0.8, π(C)=0.2; transitions H:(0.7,0.3), C:(0.4,0.6); emissions H:(0.2,0.4,0.4), C:(0.5,0.4,0.1) for labels1,2,3. For observations 1,3, calculate the probability of the observations by summing all four two-state paths.

Given

Initial probabilities: π(H)=0.8, π(C)=0.2. Transition matrix: H→H=0.7, H→C=0.3, C→H=0.4, C→C=0.6. Emissions for labels (1,2,3): H=(0.2,0.4,0.4), C=(0.5,0.4,0.1). There is no END-state factor.

To find

P(observations 1,3), not just the best-path score.

Formula and why it applies

\"P(o_1,o_2)=\\sum_{s_1,s_2}\\pi_{s_1}b_{s_1}(o_1)a_{s_1s_2}b_{s_2}(o_2)\"

The hidden path is unspecified, so its mutually exclusive alternatives must be summed.

Solution

Step 1. Score H,H
\"0.8(0.2)(0.7)(0.4)=0.0448\"
Step 2. Score H,C
\"0.8(0.2)(0.3)(0.1)=0.0048\"
Step 3. Score C,H
\"0.2(0.5)(0.4)(0.4)=0.016\"
Step 4. Score C,C
\"0.2(0.5)(0.6)(0.1)=0.006\"
Step 5. Add all paths
\"P(1,3)=0.0448+0.0048+0.016+0.006=0.0716\"
Final answer. Observation probability =0.0716. The best single-path score 0.0448 is a different quantity.

If asked for the posterior of path H,H given observations 1,3, normalize: 0.0448/0.0716 ≈0.625698. Viterbi chooses the best path using joint scores because the observation denominator is common to every candidate. It does not compute that denominator by taking a maximum.

"}]},{"id":"m2-chunks-grammar","title":"Chunking, chinking and context-free grammar","unit":4,"scope":"Unit 4, pages 14–19, 74–97","sections":[{"id":"chunking-chinking","title":"Chunking groups words; chinking removes words from a candidate chunk","html":"

Chunking, also called shallow parsing, groups adjacent tagged words into meaningful non-overlapping units, commonly noun phrases. It usually identifies local phrase boundaries without producing the complete hierarchical structure of a sentence. Chinking removes specified tokens from an existing candidate chunk, splitting or trimming it.

The/DT small/JJ cat/NN sat/VBD on/IN the/DT mat/NN
→
NP: [The small cat]
→
Outside: sat on
→
NP: [the mat]

A POS tagger labels individual words. A chunker groups words into phrases. A full parser shows how phrases relate recursively to form the entire sentence. These are related tasks with different outputs.

OperationInputOutput
POS taggingThe small catThe/DT small/JJ cat/NN
NP chunkingTagged tokens[NP The small cat]
ChinkingA broad candidate chunkRemove verbs/prepositions, keep noun-phrase segments
Full parsingSentence + grammarA nested tree with sentence, noun phrase and verb phrase nodes

Rule-based chunking uses tag patterns. Statistical or learned chunkers infer boundaries from labeled examples. A useful NP rule is an optional determiner, followed by zero or more adjectives, followed by a noun. In NLTK-style notation: NP: {<DT>?<JJ>*<NN>}. Here ? means zero or one occurrence; * means zero or more; + means one or more when used. The braces mean “form a chunk around the matched sequence”.

Question. Apply the NP rule optional DT, zero or more JJ, then one NN to “The/DT quick/JJ brown/JJ fox/NN jumps/VBZ over/IN the/DT lazy/JJ dog/NN”.

Given

Tagged input is supplied. Rule: DT? JJ* NN.

To find

All noun-phrase chunks.

Formula and why it applies

NP → DT? JJ* NN

The rule groups a noun with its preceding determiner and adjectives; a verb or preposition does not match this NP pattern.

Solution

Step 1. Start at The

The matches optional DT; quick and brown both match JJ*; fox matches NN.

Step 2. Form the first NP

[NP The quick brown fox]

Step 3. Skip nonmatching tokens

jumps/VBZ and over/IN do not satisfy the NP pattern.

Step 4. Match the second NP

the is DT, lazy is JJ, dog is NN: [NP the lazy dog].

Final answer. [NP The quick brown fox] jumps over [NP the lazy dog].

Question. Using NP → DT? JJ* NN, identify the chunks in “bright/JJ stars/NN shine/VBP and/CC a/DT moon/NN glows/VBZ”. Use the supplied tags exactly.

Given

Input tags label stars as NN for this exercise. Rule requires optional DT, zero or more JJ, then NN.

To find

The matched NP spans.

Formula and why it applies

NP → DT? JJ* NN

The optional determiner may be absent and the adjective repetition may be empty.

Solution

Step 1. Match bright stars

No determiner is present, which DT? permits. bright matches JJ; stars matches NN.

Step 2. Inspect shine and and

VBP and CC fall outside the NP rule.

Step 3. Match a moon

a matches DT; no adjective is present; moon matches NN.

Final answer. Chunks are [NP bright stars] and [NP a moon]. With ordinary Penn tagging, stars is usually NNS; broaden the noun pattern to include NNS when that tag is used.

Question. Begin with one candidate chunk containing “The/DT cat/NN sat/VBD on/IN the/DT mat/NN”. Chink out verbs and prepositions. Show the remaining chunks.

Given

Initial candidate: [The cat sat on the mat]. Remove tags VBD and IN.

To find

The chunk segments after exclusion.

Formula and why it applies

Chinking removes matched tokens from the existing chunk.

Removing the middle verb/preposition region divides the original candidate into two remaining contiguous spans.

Solution

Step 1. Locate the tokens to remove

sat is VBD and on is IN.

Step 2. Remove those tokens from chunk membership

The remaining token groups are “The cat” and “the mat”. sat and on still exist in the sentence; they are simply outside these chunks.

Step 3. Write the final chunked sentence

[The cat] sat on [the mat].

Final answer. Two chunks remain: The cat; the mat.

Yaad rakho: Chunking = andar group karo; chinking = unwanted tokens ko group se bahar karo.

"},{"id":"cfg-foundation","title":"CFG: terminals, nonterminals, production rules and parse trees","html":"

A context-free grammar (CFG) specifies a language using production rules whose left side is a single nonterminal. Formally, G=(N,Σ,R,S): N is the set of nonterminals, Σ the terminal vocabulary, R the production rules, and S the start symbol. A rule can expand its nonterminal regardless of the neighboring symbols; that is the meaning of “context-free”.

ConceptPlain explanationExample
TerminalA word at a leaf; it is not expanded furtherthe, girl, read
NonterminalA grammatical category that can be expandedS, NP, VP, Det, N, V
ProductionA permitted replacementS → NP VP
Start symbolCategory for a complete sentenceS
LexiconWord-to-category rulesN → girl | paper
DerivationSequence of rule applicationsS ⇒ NP VP ⇒ Det N VP ⇒ …
Parse treeHierarchical picture of those expansionsRoot S, internal phrases, word leaves
\"S\\to

Read S→NP VP as “a sentence can contain a noun phrase followed by a verb phrase”. It does not mean subtraction or multiplication. A vertical bar in a grammar rule means alternatives: Det→the | a allows either the or a. This is different from the probability bar meaning “given”.

S: the whole sentence
→
NP: the girl + VP: read the paper
→
NP→Det N; VP→V NP
→
Leaves in order: the / girl / read / the / paper

A constituent is a group of words that acts as a grammatical unit, such as “the girl”. Syntactic parsing assigns a grammatical structure to an input sentence using a grammar. It can support grammar checking, information extraction, translation and later semantic analysis. Failure under one restricted grammar means that grammar could not derive the sentence; it does not necessarily prove the sentence is invalid English.

A treebank is a corpus annotated with parse trees. It supplies training/evaluation structures for parsers and lets linguists study recurring syntactic patterns. A tagged corpus gives word tags; a treebank additionally records relationships among phrases.

RepresentationExample information retained
Bag of wordsWords occur, but word order/grammar is lost
POS tagsEach word's local grammatical category
ChunksShort phrase boundaries
Parse treeNested constituent structure of the whole sentence

Yaad rakho: Grammar rule ek expansion instruction hai: S ko NP aur VP mein kholo.

"},{"id":"structural-ambiguity","title":"Ambiguity and grammar coverage","html":"

Structural ambiguity occurs when a sentence has more than one valid parse tree. In “I saw the man with a telescope”, the prepositional phrase “with a telescope” can attach to saw (the instrument I used) or to the man (the man possessing the telescope). The words and their order stay the same, but the grouping changes.

ReadingAttachmentMeaning
I [saw the man] [with a telescope]PP modifies VPI used a telescope to see the man.
I saw [the man with a telescope]PP modifies NPThe man had a telescope.

A parser may need context, probabilities or semantic constraints to choose between trees. A plain CFG describes permitted structures but does not itself assign a probability to each tree. Do not invent rule probabilities for an ordinary CFG question.

Lexical ambiguity also matters: book can be a noun or verb. In “Book the flight”, the imperative analysis begins S→VP, followed by VP→V NP. A parser that initially chooses book as a noun may reach a dead end and must try another permitted analysis.

Yaad rakho: Same words, alag grouping → alag meaning.

"}]},{"id":"m2-parsing","title":"Top-down and bottom-up parsing: complete traces","unit":4,"scope":"Unit 4, pages 98–116","sections":[{"id":"top-down-theory","title":"Recursive-descent top-down parsing","html":"

Top-down parsing starts at the grammar's start symbol S and expands nonterminals until the leaves match the input words in order. Recursive descent commonly implements each nonterminal as a procedure that tries its productions. It builds from the root toward the leaves.

Start with S
→
Choose an allowed S production
→
Expand its next nonterminal
→
Match the resulting terminal to the input
→
Continue, or backtrack if the choice fails

Algorithm. Start with S as the current goal. Choose a production for the leftmost unexpanded nonterminal. Replace that nonterminal by the production's right side. If a terminal appears, compare it with the next input token. If it matches, consume the token; if it does not, reject that branch and try another production. Accept only when the complete input is consumed and all grammar goals are satisfied.

A leftmost derivation always expands the leftmost remaining nonterminal first. Top-down parsing can make guesses that the actual words cannot support, so it may need backtracking. A naive recursive-descent parser can loop on left-recursive rules such as NP→NP PP unless the grammar or algorithm is changed. Predictive LL(1) parsing chooses productions using one lookahead token when the grammar permits; not every English CFG is LL(1).

ApproachBegins withDirectionCommon difficulty
Top-downStart symbol SRoot to word leavesPredictions that do not match input; left recursion
Bottom-upActual input tokensLeaves to root SReductions that cannot contribute to a complete sentence

Yaad rakho: Top-down: sentence ka structure pehle, words tak neeche aao.

"},{"id":"top-down-examples","title":"Three full top-down derivations with parse trees","html":"

Question. Using S→NP VP, NP→Det N, VP→V NP, Det→the, N→girl|paper and V→read, give a leftmost top-down derivation and parse tree for “the girl read the paper”.

Given

Rule typeRules
StructureS → NP VP; NP → Det N; VP → V NP
LexiconDet → the; N → girl | paper; V → read

To find

A successful derivation and tree whose leaves equal the input.

Formula and why it applies

At every stage expand the leftmost nonterminal using a supplied production.

A top-down derivation begins at S; the input determines which terminal alternatives can succeed.

Solution

Step 1. Begin at the sentence symbol

S

Step 2. Expand the sentence

S ⇒ NP VP

Step 3. Expand its first noun phrase

⇒ Det N VP

Step 4. Match the first determiner

⇒ the N VP

Step 5. Match the subject noun

⇒ the girl VP

Step 6. Expand the verb phrase

⇒ the girl V NP

Step 7. Match the verb

⇒ the girl read NP

Step 8. Expand the object noun phrase

⇒ the girl read Det N

Step 9. Match its determiner

⇒ the girl read the N

Step 10. Match the last noun

⇒ the girl read the paper

Step 11. Draw the tree
Parse tree: the girl read the paperSNPDettheNgirlVPVreadNPDettheNpaper
Parse tree: the girl read the paper
Final answer. The grammar derives the complete sentence. The subject NP is “the girl”; the object NP is “the paper”.

Question. Use S→NP VP, NP→Det N|N, VP→V NP, Det→A, N→restaurant|Dosas and V→serves to parse “A restaurant serves Dosas” by a leftmost top-down derivation.

Given

Rule typeRules
StructureS → NP VP; NP → Det N | N; VP → V NP
LexiconDet → A; N → restaurant | Dosas; V → serves

To find

A derivation and parse tree.

Formula and why it applies

Expand the leftmost nonterminal; choose NP→Det N for the subject and NP→N for the object.

The object has no determiner, so forcing NP→Det N there would fail.

Solution

Step 1. Start and expand S

S ⇒ NP VP

Step 2. Choose the subject NP

⇒ Det N VP

Step 3. Match A

⇒ A N VP

Step 4. Match restaurant

⇒ A restaurant VP

Step 5. Expand the VP

⇒ A restaurant V NP

Step 6. Match serves

⇒ A restaurant serves NP

Step 7. Choose the object NP rule

⇒ A restaurant serves N

Step 8. Match Dosas

⇒ A restaurant serves Dosas

Step 9. Draw the successful tree
Parse tree: A restaurant serves DosasSNPDetANrestaurantVPVservesNPNDosas
Parse tree: A restaurant serves Dosas
Final answer. The parse succeeds with subject NP→Det N and object NP→N.

Question. Using S→NP VP|VP, NP→Det N, VP→V NP, Det→that, N→book|flight and V→book, derive the imperative “book that flight” and explain the role of book.

Given

Rule typeRules
StructureS → NP VP | VP; NP → Det N; VP → V NP
LexiconDet → that; N → book | flight; V → book

To find

The successful derivation, tree and POS of book.

Formula and why it applies

Choose a production for S that can derive the actual input; an imperative may consist of VP.

The grammar allows book as both noun and verb, but this imperative needs the verb reading.

Solution

Step 1. Choose the imperative sentence rule

S ⇒ VP

Step 2. Expand the verb phrase

⇒ V NP

Step 3. Choose the verb book

⇒ book NP

Step 4. Expand the object

⇒ book Det N

Step 5. Match that

⇒ book that N

Step 6. Match flight

⇒ book that flight

Step 7. Draw the tree
Imperative parse: book that flightSVPVbookNPDetthatNflight
Imperative parse: book that flight
Final answer. book is V; that flight is NP; S→VP derives the imperative.
"},{"id":"shift-reduce-theory","title":"Shift-reduce bottom-up parsing: stack, input and actions","html":"

Bottom-up parsing starts from the input tokens and combines them into larger constituents until the start symbol is reached. A shift-reduce parser maintains a stack of recognized material and a list of remaining input tokens.

ActionExact operationExample
ShiftMove the next input token onto the stackStack: the; remaining input: dog runs
ReduceReplace a suffix of the stack matching a rule's RHS with its LHSDet N becomes NP using NP→Det N
AcceptStack is S and input is exhaustedStack: S; input: empty
Fail/backtrackChosen reductions cannot complete a parseA noun reading of book blocks an imperative parse

Reduction uses a production in reverse: the grammar NP→Det N expands an NP top-down, but reduces Det N into NP bottom-up. The stack's rightmost item is its top in the traces below. Each row shows the state after the listed action, avoiding an off-by-one mismatch between actions and stack contents.

A lexical reduction such as N→dog still counts as a reduction. Do not skip the sequence from word to category to phrase when the question asks for a complete trace. Backtracking means returning to an earlier choice and trying another applicable rule, not changing the original input.

Yaad rakho: Shift = input se stack. Reduce = chhote pieces se bada grammatical unit.

"},{"id":"shift-reduce-examples","title":"Three full shift-reduce traces","html":"

Question. Show a complete shift-reduce parse of “the big dog sees the cat” using S→NP VP, NP→Det N|Det Adj N, VP→V NP, Det→the, Adj→big, N→dog|cat and V→sees.

Given

Grammar is as specified. Stack starts empty; input is the big dog sees the cat. Top of stack is on the right.

To find

The stack, remaining input and action at every step.

Formula and why it applies

Shift tokens; reduce matched right-hand sides to their left-hand nonterminals.

The first noun phrase includes an adjective and must use NP→Det Adj N.

Solution

Step 1. Trace each action
Step / actionStack after actionRemaining input
0. Start∅the big dog sees the cat
1. Shift thethebig dog sees the cat
2. Reduce Det→theDetbig dog sees the cat
3. Shift bigDet bigdog sees the cat
4. Reduce Adj→bigDet Adjdog sees the cat
5. Shift dogDet Adj dogsees the cat
6. Reduce N→dogDet Adj Nsees the cat
7. Reduce NP→Det Adj NNPsees the cat
8. Shift seesNP seesthe cat
9. Reduce V→seesNP Vthe cat
10. Shift theNP V thecat
11. Reduce Det→theNP V Detcat
12. Shift catNP V Det cat∅
13. Reduce N→catNP V Det N∅
14. Reduce NP→Det NNP V NP∅
15. Reduce VP→V NPNP VP∅
16. Reduce S→NP VPS∅
Step 2. Check acceptance

Input is empty and stack is exactly S; therefore accept.

Step 3. Draw the resulting structure
Parse tree: the big dog sees the catSNPDettheAdjbigNdogVPVseesNPDettheNcat
Parse tree: the big dog sees the cat
Final answer. Accepted. The parser builds both noun phrases, then VP, then S.

Question. Use S→NP VP, NP→Det N, VP→V NP, Det→the|this, N→man|book and V→read to give a shift-reduce trace for “the man read this book”.

Given

The input has five tokens. Stack is empty initially. The lexical verb read is supplied by V→read.

To find

A complete bottom-up trace.

Formula and why it applies

Reduce only when the top stack sequence matches a supplied production's right side.

Both noun phrases follow Det N, and the verb phrase follows V NP.

Solution

Step 1. Trace all operations
Step / actionStack after actionRemaining input
0. Start∅the man read this book
1. Shift thetheman read this book
2. Reduce Det→theDetman read this book
3. Shift manDet manread this book
4. Reduce N→manDet Nread this book
5. Reduce NP→Det NNPread this book
6. Shift readNP readthis book
7. Reduce V→readNP Vthis book
8. Shift thisNP V thisbook
9. Reduce Det→thisNP V Detbook
10. Shift bookNP V Det book∅
11. Reduce N→bookNP V Det N∅
12. Reduce NP→Det NNP V NP∅
13. Reduce VP→V NPNP VP∅
14. Reduce S→NP VPS∅
Step 2. Verify the final state

The complete sentence has reduced to S with no input left.

Final answer. Accepted. Subject: the man. Predicate: read this book. Object: this book.

Question. Parse “book the flight” bottom-up using S→NP VP|VP, NP→Det N, VP→V NP, Det→the, N→book|flight, V→book. Explain why choosing the noun reading first can fail and show the successful alternative.

Given

book has two lexical categories N and V. The input is an imperative. Stack starts empty.

To find

The failed choice and a complete successful trace with backtracking.

Formula and why it applies

Try lexical reductions allowed by the grammar; on a dead end, return to the ambiguous choice and try another.

Grammar ambiguity requires a search choice. The noun reading cannot supply the required imperative verb.

Solution

Step 1. Show the noun-first dead end

Shift book and reduce it to N. Continue shifting/reducing the flight to Det N. Reducing Det N→NP leaves stack N NP and empty input. No supplied rule reduces N NP into S, so this branch fails.

Step 2. Return to the lexical choice

Restore the state just after shifting book, then choose V→book instead of N→book.

Step 3. Trace the successful branch
Step / actionStack after actionRemaining input
0. Restored after shifting bookbookthe flight
1. Reduce V→bookVthe flight
2. Shift theV theflight
3. Reduce Det→theV Detflight
4. Shift flightV Det flight∅
5. Reduce N→flightV Det N∅
6. Reduce NP→Det NV NP∅
7. Reduce VP→V NPVP∅
8. Reduce S→VPS∅
Step 4. Accept the alternative

The verb reading produces S and consumes all tokens.

Final answer. The noun-first branch fails; backtracking to book/V yields the successful imperative parse.

Yaad rakho: Wrong branch fail hua toh sentence wrong nahi; next legal rule try karo.

"}]},{"id":"m2-ner","title":"Named Entity Recognition","unit":4,"scope":"Unit 4, pages 117–128","sections":[{"id":"ner-definition","title":"Entity spans, categories and context","html":"

Named Entity Recognition (NER) identifies text spans referring to entities and assigns their types, such as PERSON, ORGANIZATION and LOCATION. Many practical schemas also include DATE, TIME, MONEY or domain-specific types. The exact allowed labels depend on the application's annotation scheme.

NER has two parts: boundary detection finds where the entity starts and ends; classification assigns the appropriate type. A multiword name should normally be one entity span: “New York” is a location span, not two unrelated location decisions.

Text: Mary joined Google in New York on Monday.
→
Mary → PERSON
→
Google → ORGANIZATION
→
New York → LOCATION
→
Monday → DATE

Not every noun is a named entity. In “a developer works at Google”, developer is an ordinary job description while Google is a named organization. A specialized HR system may define a JOB_TITLE category, but it must state that schema explicitly. NER identifies mentions; entity linking is the separate task of connecting a mention to a unique database identity.

TaskQuestion it answersExample
POSWhat grammatical role does this token have?Google/NNP
ChunkingWhich adjacent words form a phrase?[NP a skilled developer]
NERWhich span names an entity, and of what type?[Google] ORGANIZATION
Entity linkingWhich real entity does this mention denote?Connect Paris to the intended city or person record

Applications. NER structures search results and documents, extracts organizations/people from news, routes support tickets by product or organization, supports recommendation, and extracts specialized entities from healthcare or legal text. Context is essential: “Apple released a laptop” names an organization, while “she ate an apple” describes fruit.

Yaad rakho: NER mein do kaam: naam kahan se kahan tak, aur kis category ka.

"},{"id":"ner-approaches","title":"Dictionary, rules, machine learning and deep learning","html":"
ApproachHow it worksExampleMain limitation
Dictionary / gazetteerMatch text against lists of known namesA city list includes Mumbai and PuneNew names, spelling variants and ambiguous names are missed
Rule-basedUse word shapes, regular expressions and contextA title such as Dr. followed by a name-like span suggests PERSONManual maintenance; exceptions and domain/language dependence
Machine learningLearn a labeling rule from annotated examples using featuresCapitalization, suffixes, POS and surrounding words help a classifier/CRFRequires labeled data and useful features
Deep learningLearn contextual word/character representations and predict labelsA contextual sequence encoder labels multiword entity spansTraining/domain dependence and resource demands

A pattern-based rule examines the form of text, such as an email pattern or a date format. A context-based rule examines neighboring words: “Dr.” can support a following PERSON span, while “headquartered in” can support a following LOCATION span. Capitalization alone is insufficient because sentence-initial common nouns are also capitalized.

A dictionary may find “May” as a person's name, but “in May” may refer to a month. A learned model uses surrounding evidence to disambiguate. Hybrid systems can combine dictionaries, rules and learned models; these categories are practical tools rather than mutually exclusive restrictions.

Preprocessing for NER. Tokenize while preserving token offsets and useful capitalization. Blindly removing capital letters or stopwords before recognition can damage names such as “Bank of India”. NER output should preserve the original span so the reader can locate it in the input.

Exam-ready 3-mark answer. NER locates named-entity spans and classifies them into categories such as person, organization and location. A dictionary-based approach matches known name lists; a rule-based approach uses patterns or context; machine-learning and deep-learning approaches learn entity decisions from labeled data and representations. For example, in “Mary works at Google in Mumbai”, Mary is PERSON, Google is ORGANIZATION and Mumbai is LOCATION.

"},{"id":"ner-worked-examples","title":"Three complete entity-recognition examples","html":"

Question. Identify PERSON, ORGANIZATION and LOCATION entities in “Asha Rao joined Infosys in Bengaluru.”

Given

Sentence: Asha Rao joined Infosys in Bengaluru. Allowed entity types: PERSON, ORGANIZATION, LOCATION.

To find

Complete entity spans and labels.

Formula and why it applies

Find the span first, then assign its type from context.

A multiword personal name is a single entity; the verb and preposition remain outside entities.

Solution

Step 1. Identify the person span

Asha Rao names the person who joined, so take both words together.

Step 2. Identify the organization

Infosys is the organization joined.

Step 3. Identify the location

Bengaluru names the city.

Step 4. Write the output
SpanLabel
Asha RaoPERSON
InfosysORGANIZATION
BengaluruLOCATION
Final answer. [Asha Rao]PERSON joined [Infosys]ORGANIZATION in [Bengaluru]LOCATION.

Question. Using PERSON, ORGANIZATION, LOCATION and DATE, identify entities in “Dr. Meera Shah visited New Delhi on 5 October 2026.” Treat professional titles as outside the PERSON span for this exercise.

Given

Allowed types and title-boundary convention are specified. Tokens include a multiword name, city and date.

To find

Entity boundaries and types.

Formula and why it applies

Boundary rules come from the annotation schema; use the stated title convention.

Dr. supplies context for a person but is excluded from that span here.

Solution

Step 1. Locate the person

Meera Shah is the PERSON span; Dr. is outside by instruction.

Step 2. Locate the location

New Delhi is one LOCATION span.

Step 3. Locate the date

5 October 2026 is one DATE span.

Step 4. Check organization mentions

No organization is named in this sentence.

Final answer. Meera Shah→PERSON; New Delhi→LOCATION; 5 October 2026→DATE.

Question. Explain the entity label of Apple in “Apple opened an office in Mumbai” and apple in “Ravi ate an apple”. Use PERSON, ORGANIZATION and LOCATION only.

Given

Two contexts contain the same surface word apart from capitalization. The schema contains no FOOD category.

To find

Contextual labels and the ordinary noun case.

Formula and why it applies

NER labels a mention in context, not every occurrence of a dictionary string identically.

A company can open an office; the object eaten in the second sentence is a fruit.

Solution

Step 1. Analyze the first sentence

Apple refers to the company, so label it ORGANIZATION. Mumbai is LOCATION.

Step 2. Analyze the second sentence

Ravi is PERSON. apple is an ordinary fruit noun and receives no entity label under this schema.

Step 3. State the dictionary limitation

Blindly labeling every apple as an organization would produce a false positive.

Final answer. First: Apple→ORGANIZATION, Mumbai→LOCATION. Second: Ravi→PERSON; apple is outside the entity types.
"}]},{"id":"u4-forward-cky","title":"Faculty topic: CNF and CYK / CKY parsing","unit":4,"mode":"tee","order":100,"scope":"Unit 4 PDF p105 names CYK. The supplied papers ask top-down / bottom-up CFG trees, not CKY charts. These traces explain the taught topic.","sections":[{"id":"cnf","title":"Prepare a grammar for CYK / CKY","html":"

Faculty topic; not an observed numerical PYQ. Unit 4 PDF p105 names CYK. The supplied papers ask top-down / bottom-up CFG trees, not CKY charts. These traces explain the taught topic.

CYK and CKY are two names for the same dynamic-programming parsing family. A chart stores which grammatical categories can generate each contiguous span of the sentence. Small spans are solved first and reused to build larger spans. Basic CKY uses a context-free grammar in Chomsky normal form (CNF): a production has either two nonterminals on its right, A→B C, or one terminal, A→word. Empty-string handling is a separate special case; the examples here have no empty production.

RuleBasic CKY statusReason
S → NP VPAllowedTwo nonterminals
N → catAllowedOne terminal
NP → Det Adj NConvert firstThree nonterminals
NP → NEliminate or handle unary closureOne nonterminal, not a lexical terminal
VP → eats NPConvert firstMixed terminal and nonterminal

Question. Convert NP→Det Adj N, NP→N, N→cat and VP→eats NP into basic CNF-compatible productions. Preserve their possible word yields. [Authored worked example]

Answer

Given

The symbols NP,Det,Adj,N,VP are nonterminals; cat and eats are terminals. Other needed lexical rules remain as supplied.

To find

Equivalent binary or terminal productions for these cases.

Formula and why it applies

Introduce helper categories for a long right-hand side or a terminal mixed with nonterminals. Eliminate a unary nonterminal rule by inheriting the reachable non-unit productions.

Solution

Step 1. Binarize the three-part NP

Introduce X: NP→Det X and X→Adj N. X is a helper, not a word; the terminal order remains Det, Adj, N.

Step 2. Remove the unit production

NP→N and N→cat allow NP to derive cat. Add NP→cat, retain N→cat where needed, and remove NP→N. With more rules reachable through unit chains, inherit every corresponding non-unit rule, not only the first one.

Step 3. Replace the mixed terminal

Introduce Vlex→eats and replace VP→eats NP by VP→Vlex NP.

Step 4. Read the converted forms

NP→Det X | cat; X→Adj N; N→cat; VP→Vlex NP; Vlex→eats. All displayed productions are binary nonterminal or single-terminal rules.

Final answer. The converted rules preserve the original word sequences while satisfying the displayed basic CKY forms. Helper nodes change tree shape; they can be removed when presenting the original grammar’s tree.

General CNF conversion may also require eliminating nullable productions and useless symbols while preserving the intended language. Do not claim that simply splitting long rules handles every arbitrary CFG. An Earley parser instead supports general CFG rules using prediction, scanning and completion; it combines top-down expectations with bottom-up evidence. LR and operator-precedence parsers are other methods named in the faculty list, with grammar restrictions of their own.

"},{"id":"cky","title":"CYK / CKY: complete charts, backpointers and ambiguity","html":"

Faculty topic; not an observed numerical PYQ. Unit 4 PDF p105 names CYK. The supplied papers ask top-down / bottom-up CFG trees, not CKY charts. These traces explain the taught topic.

Number the words 1 to n. Let T[i,j] contain all categories that derive the span from word i through word j, inclusive. Start with one-word cells from lexical rules. For every longer span try every split k between i and j: if B belongs to T[i,k], C belongs to T[k+1,j], and A→B C is a rule, put A in T[i,j]. Store the rule and split as backpointers when a tree is required.

\"A\\in

Accept precisely when S is in T[1,n]. Empty cells are meaningful: no category in this grammar derives that span. Basic recognition takes O(n³|G|) time using the straightforward rule-loop analysis and O(n²|N|) category storage; packed backpointers can preserve alternative parses without listing all trees immediately.

Question. Use CKY to parse “the cat eats a fish” with S→NP VP, NP→Det N, VP→V NP; Det→the|a; N→cat|fish; V→eats. Show the complete span chart and reconstruct a tree. [Authored worked example]

Answer

Given

Words 1=the, 2=cat, 3=eats, 4=a, 5=fish. The grammar is already in the basic CNF form. ∅ denotes an empty cell.

To find

All chart cells, acceptance, and a parse tree.

Formula and why it applies

Fill lexical cells, then spans of length 2,3,4,5. For each span try all split points and combine only categories permitted by a binary grammar rule.

Solution

Step 1. Fill the five one-word cells

T[1,1]={Det}; T[2,2]={N}; T[3,3]={V}; T[4,4]={Det}; T[5,5]={N}.

Step 2. Fill length-two spans

T[1,2]={NP} because Det+N matches NP→Det N. T[2,3]=∅ (N V has no rule). T[3,4]=∅ (V Det has no rule). T[4,5]={NP} using Det+N.

Step 3. Fill length-three spans

T[1,3]=∅: neither Det+(N eats) nor NP+V matches a rule. T[2,4]=∅. For T[3,5], split after word 3: V from [3,3] plus NP from [4,5] gives VP. Thus T[3,5]={VP}; the other split has an empty left child.

Step 4. Fill length-four spans

T[1,4]=∅ because every split has an empty child or an unmatched category pair. T[2,5]=∅: the only wholly nonempty possibility is N+VP, and no rule produces a parent from that pair.

Step 5. Fill the complete span

For T[1,5], k=1 has an empty right cell; k=2 gives NP from [1,2] and VP from [3,5], hence S. k=3 and k=4 have empty left cells. Therefore T[1,5]={S}.

Step 6. Copy the completed upper-triangular chart
Start i / end j1: the2: cat3: eats4: a5: fish
1DetNP∅∅S
2—N∅∅∅
3——V∅VP
4———DetNP
5————N

A dash is outside the chart (i>j); ∅ is an evaluated but empty valid span. Read the chart by increasing span length, not left-to-right across a row.

Step 7. Follow the stored backpointers

S[1,5]→NP[1,2] VP[3,5]; NP[1,2]→Det[1,1] N[2,2]; VP[3,5]→V[3,3] NP[4,5]; NP[4,5]→Det[4,4] N[5,5].

S\n├── NP\n│   ├── Det: the\n│   └── N: cat\n└── VP\n    ├── V: eats\n    └── NP\n        ├── Det: a\n        └── N: fish
Final answer. Accepted because S∈T[1,5]. The reconstructed tree’s leaves are exactly the five input words in order.

Question. A grammar has S→A A, A→A A and A→a. Use CKY for “a a a” and explain how two parse trees are preserved. [Authored worked example]

Answer

Given

Three identical terminal tokens. S is the start symbol. No unit or empty rules are present.

To find

Chart categories and both successful split choices for the root.

Formula and why it applies

A chart cell is a set of possible categories. For each category keep all successful backpointers when the task requires every parse.

Solution

Step 1. Initialize the diagonal

Each of T[1,1], T[2,2], T[3,3] is {A}, from A→a.

Step 2. Fill two-word spans

T[1,2] and T[2,3] each contain {A,S}: A+A can use A→A A or S→A A.

Step 3. Try both root splits

k=1: A[1,1] + A[2,3] permits S. k=2: A[1,2] + A[3,3] also permits S. The complete cell is {A,S}, but S has two distinct backpointer alternatives.

Step 4. Draw both bracketed trees

[S [A a] [A [A a] [A a]]] and [S [A [A a] [A a]] [A a]]. Both yield a a a; the constituent grouping differs.

Final answer. Accepted, with two S-rooted parses. A category appearing once in a set does not mean there is only one derivation.

Question. With S→NP VP, NP→Det N, VP→V NP; Det→the|a; N→cat|fish; V→eats, test “the eats cat” using CKY. [Authored worked example]

Answer

Given

Tokens are the/eats/cat, with no punctuation. Use exactly the supplied grammar.

To find

Whether the grammar accepts the whole input.

Formula and why it applies

A parse exists only if the complete chart cell contains the start symbol.

Solution

Step 1. Initialize the diagonal

T[1,1]={Det}; T[2,2]={V}; T[3,3]={N}.

Step 2. Try length-two combinations

Det+V matches no binary rule; V+N also matches no rule (VP requires V+NP). Both two-word cells are empty.

Step 3. Try the two full-span splits

Each split has an empty child, so T[1,3]=∅ and S is absent.

Final answer. Rejected by this grammar. Do not invent NP→N to force acceptance. Grammar rejection and a universal judgment about every possible English grammar are different claims.

Yaad rakho: Diagonal par words ke tags; phir chhote spans jodo. Har split dekho. Full sentence cell mein S aaya toh parse possible.

"}]},{"id":"u5-meaning","title":"Word meaning, lexical relations and WordNet","unit":5,"mode":"both","scope":"Unit 5 · PDF pages 1–22","sections":[{"id":"u5-words","title":"Lexeme, lemma, sense and context","html":"

Semantics is the study of meaning in language. Lexical semantics studies meanings of words and relationships between their meanings. Sentence semantics studies how word meanings and sentence structure combine.

A word form is what appears in the sentence. A lexeme is the underlying vocabulary item whose grammatical forms belong together. A lemma is the dictionary form used to name it. A sense is one particular meaning.

TermExampleWhat to understand
Word formsrun, runs, ran, runningDifferent grammatical forms
LexemeRUNThe shared underlying vocabulary item
LemmarunDictionary heading
Senserun a race / run a businessDifferent meanings need context
LexiconA language's vocabulary resourceStores lexical items and information about them

The four forms run, ran, runs, running belong to the same lexeme. They are not four unrelated lexemes. “I deposited cash in the bank” and “We sat by the bank of the river” contain the same written form but different senses.

word: bank
→
context: cash, deposit
→
sense: financial institution

Syntax asks how words are arranged. Semantics asks what the expression means. Pragmatics uses situation and speaker intention: “Can you open the window?” is grammatically a question about ability but is normally a request.

Form = jo likha; lemma = dictionary naam; sense = iss sentence mein matlab.

"},{"id":"u5-relations","title":"Every lexical relation with examples","html":"

A lexical relation connects words or senses by a meaning relationship. State the relation and its direction before giving the example.

RelationExam definitionExamples and direction
HomonymyThe same form has unrelated meanings.bank: financial institution / river edge; bat: animal / sports implement
PolysemyOne lexical item has multiple related senses.head: body part / head of a department; paper: material / article
SynonymyDifferent words have identical or closely similar meanings in a specified sense.sofa / couch; automobile / car
AntonymyWords have opposite meanings.hot / cold; alive / dead; buy / sell
HyponymyA more specific concept is a type of a more general one.rose is a flower; car is a vehicle
HypernymyA broader concept includes a more specific one.flower is a hypernym of rose
MeronymyA concept is a part of another concept.wheel is part of car; finger is part of hand
HolonymyA concept is the whole containing a part.car is a holonym of wheel

How to distinguish the commonly confused pairs

For homonymy versus polysemy, ask whether the senses share an identifiable meaning. A river bank and a financial bank are unrelated. A financial bank and a blood bank share a repository/storage idea, illustrating polysemy in the supplied notes.

For hyponymy versus meronymy, use the sentence test: “X is a kind of Y” indicates hyponymy; “X is part of Y” indicates meronymy. A wheel is not a type of car.

ExpressionCorrect relationReason
sparrow → birdHyponymyA sparrow is a kind of bird
bird → sparrowHypernymyBird is the broader category
keyboard → computerMeronymyKeyboard is a component
computer → keyboardHolonymyComputer is the containing whole
mouth of a person / mouth of a riverPolysemyBoth senses share the idea of an opening; the river use is a related extension

Synonyms are not always interchangeable in every context or register. “Big mistake” and “large mistake” can sound different even though big and large overlap in meaning. Antonyms can be gradable (hot/cold), complementary (alive/dead) or relational converses (buy/sell).

HYPO = specific type; HYPER = bigger category. MERO = part; HOLO = whole.

"},{"id":"u5-wordnet","title":"WordNet: synsets, glosses and hierarchy","html":"

WordNet is a lexical database that groups word senses into synonym sets called synsets and links them through semantic and lexical relations. It is a network of meanings, not merely a list of similar spellings.

A synset represents one concept. Its lemmas are the words that can express that concept. A gloss is a short definition, often accompanied by an example. An ambiguous word can occur in several synsets.

vehicle (general)
→
car / automobile (same concept)
→
sports car (specific)

This is a simplified teaching hierarchy, not a claim about every intermediate WordNet node. Going upward reaches hypernyms; going downward reaches hyponyms. Multiple parents can exist, so the structure need not be a single simple tree.

OperationMeaningIllustration
Get senses of a wordFind candidate synsetsbank → financial sense, river-edge sense, …
Get lemmasFind synonyms within one synsetcar, automobile, motorcar
Get definition/examplesRead the gloss and usesChoose the intended concept
Get hypernymsMove to broader categoriescar → vehicle
Get hyponymsMove to narrower categoriesvehicle → particular vehicle categories
Get meronyms/holonymsFind part/whole linkscar ↔ wheel

WordNet versus a thesaurus

A thesaurus mainly groups related expressions for vocabulary lookup. WordNet explicitly distinguishes senses and labels relations such as hypernymy, hyponymy and meronymy. This supports WSD, query expansion, lexical similarity and semantic search.

Three uses explained

  1. Search expansion: a query for “automobile” can retrieve relevant documents using “car”. Expansion should use the right sense.
  2. WSD: definitions of the senses of “bank” provide words that can be compared with the sentence context.
  3. Similarity: concepts near one another in an “is-a” hierarchy tend to be semantically related. Path-based measures depend on which senses were selected.

WordNet mein spelling ka group nahi—meaning ka synset, aur meanings ke beech labelled links.

"}]},{"id":"u5-wsd","title":"Word sense disambiguation: Lesk, Walker and graph methods","unit":5,"mode":"both","scope":"Unit 5 · PDF pages 23–48","sections":[{"id":"u5-wsd-basics","title":"What WSD solves and how approaches differ","html":"

Word sense disambiguation (WSD) is the task of selecting the intended sense of an ambiguous word in its context from a set of possible senses. The output is a sense label, not a different replacement word.

Input: “He grilled the bass.”
→
Candidate senses: fish / low sound
→
Context: grilled
→
Output: fish sense

The target word is the ambiguous word. The context is the surrounding evidence. The sense inventory lists candidate meanings, often using WordNet. A feature is a usable clue: nearby word, position, part-of-speech tag or a phrase.

ApproachWhat it needsHow it decidesLimitation
Knowledge basedDictionary, thesaurus or semantic graphDefinition overlap, category evidence or graph scoresShort definitions and missing knowledge
SupervisedExamples already labelled with sensesLearns context → sense mappingLabelling cost, limited rare-sense examples
Semi-supervisedSmall labelled set plus unlabelled dataBootstraps from reliable seed examplesEarly errors can spread
Unsupervised sense inductionUnlabelled contextual occurrencesClusters similar contextsClusters are not automatically named dictionary senses
Neural/contextualLearned word/context representationsPredicts or compares a context representationNeeds suitable data/model and evaluation

Bag-of-words features record nearby words while ignoring exact positions. Collocational features preserve a positional phrase such as “river bank”, “bank account”, “chocolate bar” or “bar exam”.

Why WSD matters

Machine translation needs the correct target-language meaning; information retrieval should distinguish river-bank travel from banking services; text-to-speech may need different pronunciations for “bass”; information extraction should avoid connecting entities through the wrong relation.

WSD: same spelling ke options mein context dekh ke sahi meaning select.

"},{"id":"u5-lesk","title":"Lesk algorithm, from words to overlap scores","html":"

Lesk is a knowledge-based WSD method that chooses senses whose dictionary descriptions have the greatest overlap with contextual evidence. Original Lesk compares definitions of neighbouring words' senses. A common simplified version compares each target-sense gloss directly with the sentence context. State which version you use.

Simplified Lesk procedure

  1. Remove the target word from the context; tokenize and normalize the remaining content words.
  2. Obtain the gloss for each candidate sense and apply the same normalization.
  3. Count shared words between the context and each gloss.
  4. Select the sense with maximum overlap. If scores tie, report the tie or use a specified tie-break rule.
\"\\operatorname{score}(s)=|C\\cap

\"C\" is the set of context words, \"G_s\" the set of words in sense \"s\"'s gloss, \"\\cap\" means common elements, \"|\\cdot|\" means count, and \"\\arg\\max\" means “the sense giving the largest score”. This version counts unique matching words once.

Question. Use simplified Lesk to disambiguate “bank” in “The river water rose near the bank.” Use C = {river, water, rose, near}; financial gloss = {institution, money, deposit}; river gloss = {land, river, water, edge}.

Given

The normalized context and two gloss sets are given. Count exact shared words.

To find

The score of each sense and the selected meaning.

Formula and why it applies

Use \"\\operatorname{score}(s)=|C\\cap because the task specifies context–gloss overlap.

Step 1. Compare the financial sense

No word occurs in both {river, water, rose, near} and {institution, money, deposit}. Overlap = {}.

Step 2. Count the financial score

\"\\operatorname{score}(F)=0\".

Step 3. Compare the river sense

Shared words are {river, water}. “Rose” and “near” are not in the gloss.

Step 4. Count and select

\"\\operatorname{score}(R)=2\". Since \"2>0\", choose the river-edge sense.

Answer. Bank means land beside a river.

Question. Use simplified Lesk for “The dog barked loudly.” Treat barked as bark. Context C = {dog, loudly}; sound gloss = {sharp, cry, dog}; tree gloss = {outer, covering, tree}; ship gloss = {sailing, ship}.

Given

Three glosses and normalized context {dog, loudly}.

To find

Select the sense of bark.

Formula and why it applies

\"\\operatorname{score}(s)=|C\\cap. Lemmatization lets barked match the target lemma bark; it does not create additional overlap words.

Step 1. Sound sense

Intersection = {dog}; score = 1.

Step 2. Tree sense

Intersection = {}; score = 0.

Step 3. Ship sense

Intersection = {}; score = 0.

Step 4. Compare

The sound sense has the unique maximum: 1 > 0.

Answer. Bark is the dog's cry.

Question. Apply definition–definition Lesk to “pine cone”. Pine senses: P1 = {evergreen, tree, needle, leaf}; P2 = {waste, sorrow, illness}. Cone senses: C1 = {solid, shape, circular}; C2 = {fruit, evergreen, tree}.

Given

Two candidate senses for each word. Tokens are already normalized; tree and trees would be reduced to the same form if necessary.

To find

The most compatible pair of senses.

Formula and why it applies

Use \"|G_{P_i}\\cap for every pair because this is original-style gloss–gloss comparison.

Step 1. P1 with C1

No shared word → 0.

Step 2. P1 with C2

Shared {evergreen, tree} → 2.

Step 3. P2 with each cone sense

P2–C1 has no overlap → 0; P2–C2 has no overlap → 0.

Step 4. Choose the best pair

The four scores are 0, 2, 0, 0. Maximum 2 belongs to P1–C2.

Answer. Pine is the evergreen tree; cone is its fruit-like reproductive structure.

Limitations: exact overlap misses synonyms, short glosses may produce zeros, and different preprocessing choices change the score. Extended Lesk may include related synset definitions or longer phrase overlaps; do not silently substitute it for a simple question.

Lesk: context aur dictionary mein common words gino; sabse bada overlap jeeta.

"},{"id":"u5-walker","title":"Thesaurus / Walker approach, with three complete examples","html":"

The Walker approach uses thesaurus categories as semantic evidence. Associate each candidate sense with a category. Count or weight context words that support that category. Choose the largest total.

\"\\operatorname{score}(s)=\\sum_{w\\in

\"I=1\" if the supplied category mapping says the context word supports the sense, otherwise \"I=0\". The summation symbol means add all those contributions. Weighted questions may replace 1 with a supplied weight. Category membership must be supplied or justified; it is not guessed from word frequency.

Question. Disambiguate bank using Walker scoring. Context words are money, interest, annum, fetch. The finance category supports money, interest and annum; the location category supports none of these words.

Given

Two categories: Finance and Location. Each supported word contributes 1.

To find

The two totals and selected bank sense.

Formula and why it applies

Sum the category indicators for each sense.

Step 1. Make a contribution table

WordFinanceLocation
money10
interest10
annum10
fetch00

Step 2. Add Finance

\"1+1+1+0=3\".

Step 3. Add Location

\"0+0+0+0=0\".

Step 4. Select

3 > 0, so select the financial institution.

Answer. Finance, with score 3 versus 0.

Question. Use Walker scoring for “The fisherman caught bass beside the lake.” Context {fisherman,caught,lake}. The Fish category contains all three clues; the Music category contains none.

Given

Fish and Music senses, binary category membership given.

To find

The correct bass sense.

Formula and why it applies

Score = sum of context-category matches.

Step 1. Record Fish matches

fisherman → 1; caught → 1; lake → 1.

Step 2. Add Fish score

\"1+1+1=3\".

Step 3. Add Music score

\"0+0+0=0\".

Step 4. Choose

Fish has the larger score.

Answer. Bass refers to a fish.

Question. Score “The player used a bat in the match.” Context {player,match}. Sport supports both words with weights 2 and 1; Animal supports neither. Use the supplied weighted Walker scheme.

Given

Sport weights {player:2, match:1}; Animal weights {player:0, match:0}.

To find

Weighted sense scores.

Formula and why it applies

Use the sum of supplied weights, not an unweighted count, because the question explicitly gives weights.

Step 1. Sport contribution of player

Add 2.

Step 2. Sport contribution of match

Add 1. Sport total = 2 + 1 = 3.

Step 3. Animal total

0 + 0 = 0.

Step 4. Compare

3 > 0 selects the sporting implement.

Answer. Bat is the sports implement.

Walker: dictionary ke exact words nahi, context kis topic/category ko support karta hai woh count.

"},{"id":"u5-graph","title":"Random-walk WSD and supervised alternatives","html":"

Graph-based WSD represents candidate senses as vertices and semantic relationships as weighted edges. A random walk repeatedly transfers score along these edges. Senses supported by other strongly connected context senses tend to obtain higher ranks.

  1. Create a vertex for every candidate sense of every relevant word.
  2. Connect semantically related senses; similarity or gloss overlap can set edge weights.
  3. Normalize outgoing edge weights into transition probabilities.
  4. Run a graph-ranking method such as PageRank.
  5. For each target word, choose its highest-ranked candidate sense.
church context
→
bell: instrument sense ↔ ring: sound sense
→
mutually supported senses receive rank

One common rank update is

\"r_{t+1}(j)=\\frac{1-d}{N}+d\\sum_i

\"r_t(i)\" is current score of vertex \"i\", \"P(i\\to is its outgoing probability to \"j\", \"N\" is the number of vertices, and \"d\" is the probability of following a graph edge. The remaining probability \"1-d\" gives a uniform restart here. A personalized variant uses a nonuniform restart. The course requires the graph method concept; numerical details depend on supplied ranking rules.

Question. A two-vertex toy sense graph has transition matrix rows A:[0.2,0.8], B:[0.4,0.6]. Starting scores are A=0.5, B=0.5. Find one random-walk update with no restart.

Given

Rows denote source vertices, columns destination vertices. d = 1.

To find

New scores at A and B.

Formula and why it applies

Use \"r_{t+1}(j)=\\sum_i: incoming contributions are added.

Step 1. Contribution to A from A

\"0.5\\times0.2=0.1\".

Step 2. Contribution to A from B

\"0.5\\times0.4=0.2\", hence \"r_1(A)=0.1+0.2=0.3\".

Step 3. Contribution to B from A and B

\"0.5\\times0.8=0.4\"; \"0.5\\times0.6=0.3\".

Step 4. Add B and verify total

\"r_1(B)=0.4+0.3=0.7\"; \"0.3+0.7=1\".

Answer. A=0.3, B=0.7 after one update. This is one iteration, not a converged PageRank result.

Other approaches explained without assuming ML knowledge

A supervised classifier learns from examples such as “cash … bank” labelled FINANCE and “river … bank” labelled RIVER. A decision list applies ranked context rules; a decision tree branches on clues; an SVM separates feature representations; a neural model learns contextual patterns. Naive Bayes, developed next, multiplies class-specific context probabilities.

Semi-supervised bootstrapping begins with a few labelled examples or high-confidence rules, labels suitable unlabelled cases, and retrains. Unsupervised clustering groups usages by context similarity without gold sense labels; an additional mapping is needed to name the discovered groups.

Graph WSD: meanings ko jodo; jo meaning context ke strong meanings se supported hai uska rank badhega.

"}]},{"id":"u5-naive-bayes","title":"Naive Bayes WSD: probabilities to a complete numerical","unit":5,"mode":"both","scope":"Unit 5 · PDF pages 49–57","sections":[{"id":"u5-bayes-meaning","title":"Prior, likelihood and posterior—before the formula","html":"

A probability is a number from 0 to 1 measuring how likely an event is under a stated model. A class probability of \"3/6=0.5\" means 3 of 6 training examples belong to that class.

Conditional probability means probability inside a restricted group. \"P(\\text{loan}\\mid reads “probability of the word loan given the finance sense”. The vertical bar means “given”; it is not division.

SymbolNamePlain meaning
\"P(F)\"PriorHow common finance is before reading the new context
\"P(C\\midLikelihoodHow compatible the observed context is with finance
\"P(F\\midPosteriorHow likely finance is after seeing the context
\"P(C)\"EvidenceTotal probability of this context under all senses
\"P(F\\mid

Bayes' theorem reverses a conditional direction using the prior. Do not swap \"P(F\\mid and \"P(C\\mid: one asks about the sense, the other about the evidence.

The naive conditional independence assumption says that, once the sense is fixed, the context-word features can be multiplied as if they were independent. It is an approximation: natural-language words are not actually independent in general.

Training examples → count classes
→
Prior per sense
→
Count words within each sense
→
Likelihood of query clues
→
Multiply → compare → normalize if asked

Prior = pehle kitna common; likelihood = clue kitna fit; posterior = clue dekhne ke baad meaning ki chance.

"},{"id":"u5-bayes-steps","title":"Multinomial Naive Bayes: the exact solving method","html":"

Naive Bayes selects the sense with the largest prior × likelihood product. The context evidence denominator is identical across candidate senses, so it can be omitted for comparison.

\"\\operatorname{score}(s)=P(s)\\prod_j

For add-one multinomial smoothing:

\"P(w\\mid

\"C(w,s)\" counts occurrences of word \"w\" in training contexts of sense \"s\"; \"N_s\" is the total number of word occurrences in those contexts; \"V\" is the shared vocabulary size. It is not the number of training sentences. Repeated query words contribute repeated factors.

Always keep these counts separate

Class priors use number of examples. Word likelihoods in this multinomial model use number of word occurrences. The vocabulary counts unique words across all classes.

  1. List the class examples and calculate priors.
  2. Write one shared vocabulary and count its size.
  3. Count total word occurrences in each class.
  4. Count each query word in each class.
  5. Apply the given smoothing rule.
  6. Multiply each prior by all its query likelihoods.
  7. Compare scores; divide each score by their sum only if a normalized posterior is requested.
\"P(s\\mid

Why smoothing is needed: an unseen clue gives \"0/N_s=0\"; one zero makes the whole product zero. Add-one smoothing redistributes probability across every vocabulary word and prevents these zero factors. Use it only when requested, taught for the task, or state the assumption.

Different model warning: a Bernoulli Naive Bayes question counts presence/absence per document and may include absence factors. Do not use its denominator in a multinomial word-count question.

Pehle sentences se prior; phir words se likelihood; dono counts mix mat kar.

"},{"id":"u5-bayes-mam","title":"Mam's bank numerical—every count and every multiplication","html":"

Question. Classify the sense of “bank” using the training table below. The sentence is: “As the stream flowed gently past the shore, we set up camp by the bank of the river, discussing the possibility of borrowing a loan to buy a boat for future trips.” Use only the highlighted feature words stream, shore and loan, and add-one multinomial smoothing.

Given

Training rowContext wordsSense
1money, deposit, loanFinance
2river, water, shoreRiverbank
3account, savings, investmentFinance
4boat, river, streamRiverbank
5branch, deposit, checkFinance
6fishing, shore, riverRiverbank

Query feature set: {stream, shore, loan}. Other sentence words are not included as features in this supplied example.

To find

The most probable bank sense and normalized posteriors.

Formula and why it applies

Naive Bayes selects the sense with the largest prior × likelihood product. The context evidence denominator is identical across candidate senses, so it can be omitted for comparison.

\"\\operatorname{score}(s)=P(s)\\prod_j

For add-one multinomial smoothing:

\"P(w\\mid

\"C(w,s)\" counts occurrences of word \"w\" in training contexts of sense \"s\"; \"N_s\" is the total number of word occurrences in those contexts; \"V\" is the shared vocabulary size. It is not the number of training sentences. Repeated query words contribute repeated factors.

Step 1. Calculate priors

Finance appears in 3 of the 6 rows; Riverbank appears in 3 of 6.

\"P(F)=\\frac36=\\frac12,\\qquad

Step 2. Construct the vocabulary

{money, deposit, loan, river, water, shore, account, savings, investment, boat, stream, fishing, branch, check}. Thus V=14. Repeated deposit or river is counted once in vocabulary.

Step 3. Count word occurrences per class

There are 3 finance rows × 3 words = 9 word occurrences. There are 3 river rows × 3 words = 9. Thus \"N_F=N_R=9\", and each smoothed denominator is \"9+14=23\".

Step 4. Count the three query words

WordFinance countRiverbank count
stream01
shore02
loan10

Step 5. Calculate finance likelihoods

\"P(\\text{stream}\\mid
\"P(\\text{shore}\\mid
\"P(\\text{loan}\\mid

Step 6. Multiply the finance score

\"\\operatorname{score}(F)=\\frac12\\times\\frac1{23}\\times\\frac1{23}\\times\\frac2{23}=\\frac1{23^3}=\\frac1{12167}\\approx0.00008219.\"

Step 7. Calculate river likelihoods

\"P(\\text{stream}\\mid
\"P(\\text{shore}\\mid
\"P(\\text{loan}\\mid

Step 8. Multiply the river score

\"\\operatorname{score}(R)=\\frac12\\times\\frac2{23}\\times\\frac3{23}\\times\\frac1{23}=\\frac3{12167}\\approx0.00024657.\"

Step 9. Compare scores

3/12167 is larger than 1/12167. Riverbank wins even though “loan” is financial evidence: the combined clues matter.

Step 10. Normalize if a posterior is requested

\"P(F\\mid
\"P(R\\mid

Answer. Bank is the riverbank sense. Scores are 1/12167 and 3/12167; normalized model posteriors are 0.25 and 0.75.

Without smoothing: stream and shore are absent in Finance; loan is absent in Riverbank. Both unsmoothed scores become zero, so there is no valid winning class from those products. Also, unnormalized scores should not be labelled as final posterior probabilities.

Zero word? Add-one: count+1 upar, total words+vocabulary neeche. Last mein score bada wala sense.

"},{"id":"u5-bayes-practice","title":"More WSD numericals: given likelihoods, unequal priors and repeated clues","html":"

Question. For bass, let P(Fish)=0.6 and P(Music)=0.4. Given Fish likelihoods P(caught|Fish)=0.5, P(lake|Fish)=0.4 and Music likelihoods P(caught|Music)=0.05, P(lake|Music)=0.1, classify context {caught,lake} and normalize.

Given

Priors and likelihoods are provided directly; no smoothing or training-count estimation is requested.

To find

The winning sense and its posterior.

Formula and why it applies

\"\\operatorname{score}(s)=P(s)P(\\text{caught}\\mid, because the two features are treated as conditionally independent.

Step 1. Fish multiplication

\"0.6\\times0.5=0.3\", then \"0.3\\times0.4=0.12\".

Step 2. Music multiplication

\"0.4\\times0.05=0.02\", then \"0.02\\times0.1=0.002\".

Step 3. Total evidence

\"0.12+0.002=0.122\".

Step 4. Normalize

\"P(Fish\\mid
\"P(Music\\mid

Answer. Fish, with model posterior approximately 0.983607.

Question. Disambiguate bank from the repeated context tokens {loan, loan, river}. Priors are Finance=2/3 and River=1/3. Vocabulary V=4. Finance has N=8 total tokens, loan count 3 and river count 0; River has N=4, loan count 0 and river count 2. Apply add-one smoothing.

Given

Multinomial counts are supplied. The two appearances of loan must both contribute.

To find

Scores and normalized Finance posterior.

Formula and why it applies

Naive Bayes selects the sense with the largest prior × likelihood product. The context evidence denominator is identical across candidate senses, so it can be omitted for comparison.

\"\\operatorname{score}(s)=P(s)\\prod_j

For add-one multinomial smoothing:

\"P(w\\mid

\"C(w,s)\" counts occurrences of word \"w\" in training contexts of sense \"s\"; \"N_s\" is the total number of word occurrences in those contexts; \"V\" is the shared vocabulary size. It is not the number of training sentences. Repeated query words contribute repeated factors.

Step 1. Finance denominator

\"8+4=12\". \"P(loan\\mid; \"P(river\\mid.

Step 2. River denominator

\"4+4=8\". \"P(loan\\mid; \"P(river\\mid.

Step 3. Finance product

\"\\operatorname{score}(F)=\\frac23\\times\\frac13\\times\\frac13\\times\\frac1{12}=\\frac1{162}\\approx0.00617284.\"

Step 4. River product

\"\\operatorname{score}(R)=\\frac13\\times\\frac18\\times\\frac18\\times\\frac38=\\frac1{512}\\approx0.001953125.\"

Step 5. Compare and normalize

\"P(F\\mid

Answer. Finance wins. Repeated loan contributes a squared likelihood; do not reduce the query to a unique-word set.

Answer-writing order in an exam

Write the class priors, vocabulary size, class token totals, a likelihood table, both score products and the final class. Keep fractions until the comparison where possible. If the question only asks classification, normalization is optional; if it asks posterior probabilities, normalization is required.

Repeated word aaye toh factor bhi repeat. Sirf word-set bana diya toh multinomial answer badal jayega.

"}]},{"id":"u5-vectors","title":"Vector semantics and word embeddings","unit":5,"mode":"tee","scope":"TEE · Unit 5 PDF pages 58–118","sections":[{"id":"u5-distribution","title":"Distributional hypothesis and a co-occurrence matrix","html":"

The distributional hypothesis states that words occurring in similar contexts tend to have similar meanings or semantic behaviour. “Cat” and “dog” often occur near pet, feed and animal. Their context patterns can therefore be represented and compared numerically.

A vector is an ordered list of numbers. Each position must have a fixed meaning. For context order [pet, food, road], cat=[4,3,0] means four co-occurrences with pet, three with food, none with road. A matrix stacks vectors as rows in a table.

A word–context co-occurrence matrix records how often a target word occurs with each context word under a specified context definition. A window may be a sentence, document, or a fixed number of neighbouring tokens. State its width, direction and whether targets count themselves.

Choose target words
→
Choose context/window
→
Count co-occurrences
→
Store each row as a vector
→
Compare vectors

Question. Build the symmetric co-occurrence matrix for “cats eat fish” using a window of one token on either side, no self-counts and vocabulary [cats,eat,fish].

Given

One sentence, one occurrence of each token. Left and right adjacent tokens are contexts.

To find

A 3×3 target-by-context count matrix.

Formula and why it applies

\"X_{ij}\" counts the number of times context word \"j\" occurs within the chosen window of target \"i\".

Step 1. Target cats

Only eat is adjacent. Row cats = [0,1,0].

Step 2. Target eat

cats is left and fish is right. Row eat = [1,0,1].

Step 3. Target fish

Only eat is adjacent. Row fish = [0,1,0].

Step 4. Assemble the matrix

Target / contextcatseatfish
cats010
eat101
fish010

Answer. Rows are [0,1,0], [1,0,1], [0,1,0]. The symmetry follows from the symmetric window.

Question. Build a target–context matrix for sentences “cats eat fish” and “dogs eat fish”. Target words are cats and dogs; context columns [eat,fish]; use the entire sentence excluding the target.

Given

Each sentence contains one target. Count context token occurrences.

To find

Two context vectors.

Formula and why it applies

Count each context word in every sentence containing the target.

Step 1. Count cats contexts

In its sentence: eat occurs once, fish once → [1,1].

Step 2. Count dogs contexts

In its sentence: eat occurs once, fish once → [1,1].

Step 3. Compare

Both targets have identical observed context patterns in this very small corpus.

Answer. cats=[1,1], dogs=[1,1]. This corpus supplies no context evidence to distinguish them.

Question. For “a b a”, use a right-only window of one token. Build the matrix with rows/columns [a,b].

Given

Count directed target→next-word events, not both directions.

To find

The co-occurrence matrix.

Formula and why it applies

\"X_{ij}=C(\\text{target.

Step 1. First target a

Next token is b → add 1 to X(a,b).

Step 2. Target b

Next token is a → add 1 to X(b,a).

Step 3. Final a

No following token → no addition.

Step 4. Write matrix

Target / nextab
a01
b10

Answer. a=[0,1], b=[1,0]. A different context window would produce a different matrix.

Word ka meaning: uske aas-paas kaunse words kitni baar aaye, uski number-list.

"},{"id":"u5-representation","title":"Sparse, dense, distributional and distributed","html":"

A sparse representation has many zero entries and often one coordinate per vocabulary item. BoW and TF–IDF are common sparse document representations. A dense embedding has comparatively few learned real-valued coordinates; one word's properties are spread across many coordinates.

RepresentationWhat coordinates meanStrengthLimitation
One-hotOne identity position per vocabulary wordUnambiguous IDDifferent words are orthogonal; no learned similarity
Count-based context vectorContext-word co-occurrence countsTransparent corpus evidenceLarge, sparse; frequent words may dominate
TF–IDF document vectorWeighted vocabulary termsUseful retrieval weightingWord order and contextual meaning are limited
Dense word embeddingLearned latent featuresCompact semantic/syntactic relationshipsCoordinates are not generally human-labelled features

Distributional describes learning from patterns of linguistic context. Distributed describes representing an item across multiple coordinates rather than a single active identity bit. These are related but different ideas. A one-hot ID alone does not learn distributional similarity.

Model design choices

Choose contexts (document, sentence, fixed window), weighting (raw counts, normalization, TF–IDF-like weighting or association scores), dimensionality reduction or learned embeddings, and a similarity measure. Cosine compares direction and is less affected by length than a raw dot product; Euclidean distance also reflects magnitude.

Sparse = bahut zero. Dense = chhoti learned number-list. Similar context se useful similarity seekhte hain.

"},{"id":"u5-word2vec","title":"Word2Vec: CBOW and skip-gram from training pairs","html":"

Word2Vec learns dense word embeddings by predicting words from nearby context. Its two common architectures are CBOW and skip-gram. A vocabulary ID or one-hot vector is the input representation; the learned embedding table converts that identity into a short trainable vector.

ArchitectureInputPredictionMemory cue
CBOWSurrounding context wordsCentre/target wordContext → centre
Skip-gramCentre/target wordSurrounding wordsCentre → context
CBOW: the + sat
→
Average context embeddings
→
Predict: cat
Skip-gram: cat
→
Embedding of cat
→
Predict: the and sat

For vocabulary size \"V\" and embedding dimension \"d\", an input one-hot row is \"1\\times; the input embedding matrix is \"V\\times; the hidden row is \"1\\times; the output matrix is \"d\\times; output scores are \"1\\times. Multiplying a one-hot row by the embedding matrix simply selects one row.

\"P(w_o\\mid

\"v_{w_i}\" is the input word embedding, \"u_w\" an output-context embedding, and the dot product is a compatibility score. Softmax turns all scores into probabilities that sum to one. Training increases the probability of observed context words. Negative sampling or hierarchical softmax can reduce computation in larger vocabularies; use the algorithm specified by the question.

Question. For the sentence “the cat sat on mat”, generate CBOW and skip-gram training data for centre word sat with a window of one token on each side.

Given

Tokens indexed 1 to 5: the, cat, sat, on, mat. Centre = sat.

To find

The CBOW example and the skip-gram pairs.

Formula and why it applies

CBOW predicts the centre from its neighbours; skip-gram predicts each neighbour from the centre.

Step 1. Identify neighbours

Left neighbour = cat; right neighbour = on. “the” and “mat” are outside this window.

Step 2. CBOW example

Input [cat,on] → target sat.

Step 3. Skip-gram examples

(sat,cat) and (sat,on), with first item as input and second as predicted context.

Answer. CBOW: [cat,on]→sat. Skip-gram: sat→cat and sat→on.

Question. The embeddings of context words cat and on are [1,2] and [3,0]. Compute the CBOW hidden vector using mean aggregation. Output vectors for [sat,ran] are [1,0] and [0,1]. Compute softmax probabilities.

Given

Context embeddings [1,2], [3,0]; mean aggregation; two candidate output vectors.

To find

The hidden vector, logits and probabilities.

Formula and why it applies

Mean context vector \"h=(v_1+v_2)/2\". Logit \"z_w=u_w^Th\". Softmax \"P(w)=e^{z_w}/\\sum_j.

Step 1. Add coordinates

\"[1+3,\\,2+0]=[4,2]\".

Step 2. Divide by number of words

\"h=[4/2,2/2]=[2,1]\".

Step 3. Compute logits

\"z_{sat}=1(2)+0(1)=2\". \"z_{ran}=0(2)+1(1)=1\".

Step 4. Exponentials

\"e^2\\approx7.389056\", \"e^1\\approx2.718282\"; sum \"\\approx10.107338\".

Step 5. Divide

\"P(sat)\\approx0.731059\", \"P(ran)\\approx0.268941\".

Answer. h=[2,1]; sat is more probable (0.731059).

Question. A skip-gram centre embedding is v=[1,−1]. Output embeddings for [eat,sleep,drive] are [1,1], [0,1], [1,0]. Calculate the predicted context probabilities.

Given

One centre vector and three output vectors; full softmax over these three words.

To find

Three logits and three probabilities.

Formula and why it applies

\"z_j=u_j^Tv\", then \"P(j)=e^{z_j}/\\sum_k.

Step 1. Eat dot product

\"1(1)+1(-1)=0\".

Step 2. Sleep and drive dot products

\"z_{sleep}=0(1)+1(-1)=-1\". \"z_{drive}=1(1)+0(-1)=1\".

Step 3. Exponentiate and sum

\"e^0=1\", \"e^{-1}\\approx0.367879\", \"e^1\\approx2.718282\". Sum \"\\approx4.086161\".

Step 4. Normalize

\"P(eat)\\approx0.244728\"; \"P(sleep)\\approx0.090031\"; \"P(drive)\\approx0.665241\".

Answer. drive has the highest probability in this toy learned model.

Limitations: a standard static Word2Vec vector merges a word's senses; it has limited representation of subword morphology and out-of-vocabulary words; its local prediction objective does not directly fit a full global co-occurrence table. Similarity is learned from data and is not a guarantee of factual or causal understanding.

CBOW = neighbours se missing centre. Skip-gram = centre se neighbours.

"},{"id":"u5-sentence","title":"Word analogies and sentence embeddings","html":"

A sentence embedding represents an entire sentence by one fixed-length vector. A basic method averages word embeddings. More advanced encoders learn a sentence representation from context and task objectives.

\"v_{\\text{sentence}}=\\frac1n\\sum_{i=1}^{n}v_{w_i}.\"

This means add corresponding coordinates and divide each coordinate by the number of words. Averaging ignores order: “dog bites man” and “man bites dog” have the same average if their words are identical.

Question. Average the word vectors [1,0], [0,2] and [2,1] to represent a three-word sentence.

Given

Three two-dimensional vectors with equal weights.

To find

The two-dimensional sentence vector.

Formula and why it applies

Use coordinate-wise mean \"v=(v_1+v_2+v_3)/3\".

Step 1. First coordinate

\"1+0+2=3\"; \"3/3=1\".

Step 2. Second coordinate

\"0+2+1=3\"; \"3/3=1\".

Answer. Sentence vector = [1,1].

Question. Toy word vectors are king=[3,2], man=[1,0], woman=[1,1]. Find king−man+woman and compare to queen=[3,3].

Given

All vectors use the same two-coordinate space.

To find

The analogy vector and its distance from queen.

Formula and why it applies

Vector arithmetic is coordinate by coordinate. Euclidean distance is the square root of the sum of squared coordinate differences.

Step 1. Subtract man

\"[3-1,2-0]=[2,2]\".

Step 2. Add woman

\"[2+1,2+1]=[3,3]\".

Step 3. Compare with queen

\"\\sqrt{(3-3)^2+(3-3)^2}=0\".

Answer. The toy analogy exactly equals queen. Real learned embeddings only approximate some analogies.

Contextual encoders can assign “bank” different representations in different sentences. BERT and RoBERTa produce contextual token representations; obtaining strong sentence similarity vectors requires an appropriate pooling/training method, not simply assuming every raw token vector is a sentence embedding. Universal Sentence Encoder is designed to encode sentence-level meaning. Skip-thought predicts surrounding sentences; InferSent learns from natural-language-inference supervision.

Sentence average: har column add karo, words ki count se divide. Word order ka loss yaad rakh.

"}]},{"id":"u5-glove","title":"GloVe: global co-occurrence and weighted error","unit":5,"mode":"tee","scope":"TEE · Unit 5 PDF pages 119–130","sections":[{"id":"u5-glove-theory","title":"Why GloVe uses global statistics","html":"

GloVe (Global Vectors) learns word embeddings by fitting weighted relationships in a global word–context co-occurrence matrix. Word2Vec uses local prediction examples; GloVe uses aggregate counts directly in its objective.

\"X_i=\\sum_k

\"X_{ik}\" is how often context word \"k\" appears around target \"i\". \"X_i\" is the total of row \"i\". \"P_{ik}\" is the resulting context probability.

The ratio \"P(k\\mid helps identify how a probe context differs between two words. For targets ice and steam, a probe such as solid should favour ice; gas should favour steam; a probe shared similarly by both gives a ratio near one. These are data tendencies, not universal numerical identities.

Question. For context solid, ice has count 12 out of row total 20; steam has count 2 out of row total 40. Compute P(solid|ice), P(solid|steam) and their ratio.

Given

X(ice,solid)=12, X(ice)=20; X(steam,solid)=2, X(steam)=40.

To find

Two conditional probabilities and the ice/steam ratio.

Formula and why it applies

\"P(k\\mid, and \"R=P(k\\mid.

Step 1. Ice probability

\"12/20=0.6\".

Step 2. Steam probability

\"2/40=0.05\".

Step 3. Ratio

\"0.6/0.05=12\".

Answer. The probe solid is 12 times as probable in ice's context under these counts.

Question. For probe water, ice and steam both have row total 50. Counts are 10 and 10. Calculate the ratio.

Given

Both probabilities use a denominator of 50.

To find

The probability ratio.

Formula and why it applies

Use row-normalized counts before taking the ratio.

Step 1. Ice probability

\"10/50=0.2\".

Step 2. Steam probability

\"10/50=0.2\".

Step 3. Divide

\"0.2/0.2=1\".

Answer. Ratio 1: this probe does not distinguish the targets in this toy data.

Question. For probe gas, counts are 1/100 around ice and 20/100 around steam. Compute the ice-to-steam ratio.

Given

Conditional counts have the same row denominator.

To find

The ratio and interpretation.

Formula and why it applies

\"R=P(gas\\mid.

Step 1. Ice probability

\"1/100=0.01\".

Step 2. Steam probability

\"20/100=0.2\".

Step 3. Ratio

\"0.01/0.2=0.05\".

Answer. Ratio 0.05, so gas is much more strongly associated with steam.

Ratio >1: pehla word favoured. <1: doosra. ≈1: probe dono mein similar.

"},{"id":"u5-glove-loss","title":"The GloVe objective and numerical interpretation","html":"
\"J=\\sum_{i,j:X_{ij}>0}f(X_{ij})\\left(w_i^T\\widetilde

What each piece means: \"w_i\" is a target vector; \"\\widetilde is a separate context vector; the dot product estimates their compatibility; biases adjust word-level baselines. The desired score is the natural log of the observed count. Squaring penalizes both positive and negative errors. Weighting prevents unreliable rare pairs or extremely frequent pairs from dominating inappropriately.

\"f(x)=\\begin{cases}(x/x_{\\max})^\\alpha,&0<x<x_{\\max}\\\\1,&x\\ge

The original commonly used settings are \"x_{\\max}=100\", \"\\alpha=0.75\". They are hyperparameters, not learned word embeddings. Zero-count pairs do not require evaluating \"\\ln in the displayed nonzero-count objective.

Question. For one nonzero GloVe pair, X=4, weighting f(X)=0.5, target vector [1,2], context vector [0.5,0], and both biases 0. Find its contribution to J.

Given

Use natural logarithm. Only one pair is being evaluated.

To find

Weighted squared residual.

Formula and why it applies

\"J_{ij}=f(X)(w^T\\widetilde.

Step 1. Dot product

\"1(0.5)+2(0)=0.5\".

Step 2. Log-count target

\"\\ln4\\approx1.386294\".

Step 3. Residual

\"0.5+0+0-1.386294=-0.886294\".

Step 4. Square and weight

\"(-0.886294)^2\\approx0.785518\". Multiply by 0.5 → approximately 0.392759.

Answer. The pair contributes approximately 0.392759 to the objective.

Question. A GloVe pair has X=e², weight 1, dot product 1.5 and biases 0.2 and 0.3. Calculate its objective contribution.

Given

Natural logarithm, so ln(e²)=2.

To find

The squared weighted error.

Formula and why it applies

\"J_{ij}=f(X)(\\text{dot}+b+\\widetilde.

Step 1. Predicted score

\"1.5+0.2+0.3=2\".

Step 2. Target score

\"\\ln(e^2)=2\".

Step 3. Residual and square

\"2-2=0\"; \"1\\times0^2=0\".

Answer. Contribution 0: this pair is fitted exactly.

Question. Use xmax=100 and α=0.75. A GloVe pair has X=16 and residual 2. Find its weighting and loss contribution.

Given

16 is below xmax, so use the power branch.

To find

f(16) and weighted squared residual.

Formula and why it applies

\"f(16)=(16/100)^{0.75}\", and \"J_{ij}=f(16)\\times2^2\".

Step 1. Ratio

\"16/100=0.16\".

Step 2. Power

\"0.16^{0.75}=0.16^{3/4}\\approx0.252982\".

Step 3. Square residual

\"2^2=4\".

Step 4. Multiply

\"0.252982\\times4\\approx1.011929\".

Answer. f≈0.252982 and contribution≈1.011929.

Six steps to explain the algorithm

  1. Collect a corpus and choose a context window.
  2. Build the global co-occurrence matrix.
  3. Study row-normalized context probabilities and their ratios.
  4. Define the weighted log-count fitting objective.
  5. Optimize word/context vectors and biases.
  6. Use the learned embeddings for similarity, analogy or downstream tasks.

GloVe: global counts → log target → dot product prediction → weighted squared error kam karo.

"}]},{"id":"u6","title":"Unit 6 · reference & discourse","unit":6,"sections":[{"id":"reference","title":"Reference, pragmatics and discourse","html":"

Pragmatic analysis interprets an utterance using its situation and the speaker’s intended meaning. Discourse analysis studies how utterances connect across a text or conversation. Syntax and literal word meanings alone cannot settle every interpretation.

TextLiteral contentContextual interpretation
“Can you close the window?”A question about ability.Usually a polite request.
“Meera bought a laptop. It was expensive.”Two sentences.“It” connects the second sentence to the laptop.
“Meet me here tomorrow.”An instruction.Me, here and tomorrow depend on speaker, place and date.

Reference, mention and antecedent

A referring expression points to an entity; a mention is its occurrence in the text. Its referent is the entity being discussed. An antecedent is a preceding expression used to interpret a later referring expression. Reference resolution identifies the intended referent. Coreference resolution groups mentions that refer to the same entity.

“Meera”name mention
→
Person entitysame referent
→
“she” / “the engineer”other mentions
ConceptDefinitionExample
AnaphoraReference back to something introduced earlier.“Riya arrived. She sat down.” She → Riya.
CataphoraReference forward to a later expression.“Before she spoke, Riya checked the microphone.” she → Riya.
ExophoraReference to the situation outside the text.“Put it there”, said while pointing.
CoreferenceTwo or more mentions refer to the same entity.“Dr Rao”, “the lecturer”, “she”.
Pronoun resolutionFind the referent of a pronoun.Resolve “it” in “The laptop stopped because it overheated.”

Coreference is broader than pronoun resolution: repeated names and descriptions also form a chain. Anaphora describes direction/dependence; coreference describes identity. In “I bought a car. The engine is noisy”, engine is related to the car but is not the same entity.

Yaad rakho: Reference = kiski baat? Coreference = alag mentions, same entity. Pronoun resolution = he/she/it kis se linked?

Cohesion, coherence and discourse markers

\n

Cohesion is the explicit linguistic linking of a text through reference, repeated or related words, conjunctions and other devices. In “Rita bought a book. She read it”, she and it connect the second sentence to Rita and the book.

\n

Coherence is the conceptual unity that makes a discourse meaningful. “The road flooded, so the match was cancelled” presents a plausible cause and consequence. A reader relates the propositions using context and knowledge.

\n

Discourse markers signal relationships or organize speech: however signals contrast; therefore signals a result; first and next signal sequence; for example introduces an illustration.

\n

These concepts differ: “I like apples. Therefore the moon is square” has a linking word but no sensible inference. A connector alone does not guarantee coherence.

\n

Yaad rakho: Cohesion = words se link; coherence = meaning se link; marker = relation ka signal.

"},{"id":"resolution","title":"Pronoun resolution: methods and examples","html":"

A practical resolution procedure

First detect the pronoun and candidate noun phrases. Eliminate candidates inconsistent with person, number, grammatical binding or relevant semantic constraints. Rank the surviving candidates using syntactic prominence, recency and discourse context. Preserve ambiguity if the available evidence does not decide.

ApproachHow it worksStrengthLimitation
Rule based / syntacticUse parse trees, agreement and explicit search order; Hobbs is a classic example.Can explain each decision without labelled training data.Parsing errors and world knowledge gaps cause errors.
Salience basedScore candidates using recency, subjecthood and other discourse features; select the strongest compatible candidate.Captures the tendency to refer again to prominent entities.Feature weights and competing candidates can mislead.
Supervised statisticalLearn candidate/pronoun compatibility from labelled examples.Combines many features from data.Needs representative annotated examples.
Neural / contextualEncode the surrounding text and score mention pairs or antecedents.Can learn contextual patterns and long-range dependencies.Needs data/compute and can still make confident mistakes.

Worked analysis 1 · number and semantics

Resolve “they” in: “Riya put the books on the shelf because they were heavy.”

Given

Candidates: Riya (singular person), books (plural objects), shelf (singular object); pronoun they is plural here.

To find

The antecedent and a justified resolution procedure.

Formula and why it applies

Use number agreement first, then semantic compatibility. There is no multiplication formula in this linguistic task.

Solution

Step 1. List candidates

Riya, the books, the shelf.

Step 2. Apply number agreement

Riya and shelf are singular. Books is plural, matching they.

Step 3. Check meaning

Books can be heavy; the causal explanation fits carrying/placing books.

“They” refers to “the books”.

Worked analysis 2 · a complete coreference chain

Identify the coreference chain in: “Dr Sen entered. The surgeon greeted Ravi. She opened her notebook.”

Given

Mentions: Dr Sen, the surgeon, Ravi, she, her, her notebook. Assume the surgeon is Dr Sen and she/her refers to that surgeon.

To find

The person chain, and a mention that must not be placed in it.

Formula and why it applies

Group expressions by identity of referent; a possessive expression and the owned object are different entities.

Solution

Step 1. Link the name and description

Dr Sen and the surgeon denote the same person under the given context.

Step 2. Link the pronouns

She and her refer to the surgeon.

Step 3. Keep the notebook separate

“Her notebook” is an object owned by the person; it is not the person herself.

Person chain: {Dr Sen, the surgeon, She, her}. “Her notebook” is a separate object mention.

Worked analysis 3 · honest ambiguity

Discuss the pronoun in: “Asha called Nisha after she finished work.”

Given

Both named candidates are singular; the sentence gives no further context.

To find

Whether one referent is logically guaranteed.

Formula and why it applies

Agreement filters incompatible candidates but cannot always select between equally compatible candidates.

Solution

Step 1. Check agreement

Both Asha and Nisha are compatible with she.

Step 2. Check the event

Either person could have finished work.

Step 3. State the limitation

A heuristic may prefer a candidate, but grammar and this sentence alone do not prove the intended referent.

Ambiguous without further context. State any heuristic and parse assumption before giving its preferred candidate.
"},{"id":"hobbs","title":"Hobbs’ algorithm, step by step","html":"

Hobbs’ algorithm is a syntax-based procedure that searches constituent parse trees in a fixed order for a compatible noun-phrase antecedent of a pronoun. The tree determines which candidates are proposed first. Agreement and other relevant constraints determine whether a proposed candidate can be accepted.

It is not simply the rule “choose the closest noun”. A nearby noun may have the wrong number, may be syntactically ineligible, or may be visited after a structurally more prominent noun. The algorithm produces a preference under the supplied parse; ambiguous language can still require context.

The tree vocabulary you need

TermMeaningExample
NPNoun phrase: a phrase referring to an entitythe student; Anita; they
SA sentence/clause nodethe node containing a subject and predicate
VPVerb phrasetold Ravi that she should revise
XThe NP or S node currently being searchedAn algorithm label, not a word
Path pThe branch travelled upward from the pronoun/previous XUsed to distinguish left and right subtrees
N-bar (N′)The noun-centred part of an NP in a grammar using that labelThe nominal-child test depends on the given tree
Breadth-firstVisit shallower nodes before deeper onesLevel 1 before level 2
Left-to-rightVisit equally deep nodes in their written/tree orderEarlier left branch before right branch
Locate pronoun NP
→
Climb to NP/S
→
Propose candidates in tree order
→
Check compatibility
→
Accept or continue searching

The complete search procedure

Step 1. Start at the pronoun

Locate the NP immediately dominating the pronoun. “Dominating” means containing it as a descendant in the tree.

Step 2. Climb to the first NP or S

Move upward to the first NP or S above that starting NP. Call this node X. Keep the upward path p.

Step 3. Search the left branches initially

Below X, traverse branches to the left of p breadth-first and left-to-right. At this initial search, propose an NP only if another NP or S lies between that candidate and X. Test proposed candidates for the required agreement and constraints.

Step 4. If X is the highest sentence node, inspect earlier sentences

Search previous sentence trees from most recent to oldest. Within each, use breadth-first, left-to-right order to propose NPs. Accept the first eligible compatible candidate, or report that no candidate was found in the available context.

Step 5. Otherwise climb again

Move from X to the next NP or S above it. Record this new X and the path followed during the climb.

Step 6. Apply the higher-NP test

If the new X is an NP, propose X when the path did not pass through its nominal N-bar child. Use the given parse labels; do not invent an N-bar branch missing from the question's structure.

Step 7. Search its left branches

Traverse branches below the new X to the left of the path, breadth-first and left-to-right, proposing NPs. The special intervening-NP/S restriction belongs to the initial search in Step 3.

Step 8. If X is S, search the permitted right branches

Search branches to the right of the path breadth-first and left-to-right. Do not descend below an encountered NP or S. Propose eligible NP candidates there.

Step 9. Repeat or stop

Return to the highest-S check and continue the climb/search procedure until a compatible candidate is accepted or the available trees are exhausted. Record the candidate order rather than only writing an unexplained name.

Why the first left search has an extra condition

In a simple clause such as “John saw him”, the subject NP John is directly under the clause S. There is no extra NP or S between John and that S. The initial-search restriction therefore prevents casually selecting John as the ordinary local antecedent of him. Compare the reflexive “John saw himself”, which is a different grammatical case; do not treat every pronoun type as interchangeable.

Worked tree 1 · move to the previous sentence

Question. Apply Hobbs’ search to “The student arrived. She smiled.” Assume the student is compatible with she.

Given

Previous tree: [S [NP The student] [VP arrived]]. Current tree: [S [NP She] [VP smiled]].

To find

The selected antecedent and the search path.

Method and why it applies

Use the supplied constituent trees. The current sentence has no earlier candidate, so the highest-S rule activates previous-sentence search.

Solution

Step 1. Draw both sentence structures

Previous sentence             Current sentence\nS                             S\n├── NP: The student            ├── NP: She ← start\n└── VP: arrived                └── VP: smiled

Step 2. Locate and climb

Start at NP She. Its first higher NP/S is the current S. Set X to that S.

Step 3. Search left of the path

She is the leftmost subject. No NP lies to its left in the current sentence.

Step 4. Use the highest-S rule

This X is the highest S. Inspect the immediately preceding sentence tree.

Step 5. Propose its first NP

Breadth-first, left-to-right finds NP The student.

Step 6. Check compatibility and accept

The question stipulates that the student agrees with she. Accept this candidate.

Answer. She refers to the student under the supplied assumptions.

Worked tree 2 · reject an incompatible subject

Question. Resolve they in “Riya spoke to the students. They laughed.” Use a previous-sentence search and interpret they as plural.

Given

Previous tree has subject NP Riya, then VP spoke [PP to [NP the students]]. Current subject is NP They.

To find

The candidate order and accepted antecedent.

Method and why it applies

Breadth-first search favours a shallow subject, but agreement must still be applied; a first candidate is not automatically a valid answer.

Solution

Step 1. Draw the useful part of the previous tree

S\n├── NP: Riya                 ← candidate 1\n└── VP\n    ├── V: spoke\n    └── PP\n        ├── P: to\n        └── NP: the students ← candidate 2

Step 2. Search the current sentence

From NP They climb to its S; there is no earlier NP in “They laughed”. This is the highest S, so move to the previous tree.

Step 3. Visit Riya first

Riya is the shallower leftmost NP. Under the stated plural interpretation, this singular candidate fails number agreement.

Step 4. Visit the students

Continue breadth-first. The next relevant NP is the students.

Step 5. Apply number agreement

Students is plural and is an eligible candidate. It matches plural they.

Step 6. State the resolution

Accept the students. The method did not simply accept the first noun it saw.

Answer. They → the students. Candidate order: Riya rejected; the students accepted.

Worked tree 3 · climb out of an embedded clause

Question. Resolve she in “Anita told Ravi that she should revise.” Assume Anita is compatible with she and Ravi is not. Use the displayed parse.

Given

The pronoun is the subject of an embedded S inside the main VP.

To find

The preferred antecedent found by the climb and left-branch search.

Method and why it applies

The embedded clause has no left candidate. Climbing to the main S exposes its subject Anita as the first suitable higher-level candidate.

Solution

Step 1. Show the clause structure

S (main)\n├── NP: Anita                  ← accepted candidate\n└── VP\n    ├── V: told\n    ├── NP: Ravi\n    └── SBAR: that\n        └── S (embedded)       ← first X\n            ├── NP: she        ← start\n            └── VP: should revise

Step 2. Start inside the embedded clause

From NP she climb to the embedded S. There is no candidate to the left of she within this embedded S.

Step 3. Climb to the next NP/S

The embedded S is not the highest S. Move upward through SBAR and VP to the main S. These intermediate nodes are not NP/S search nodes.

Step 4. Search left of the upward path

At the main S, the subject NP Anita is a shallow left candidate. The object Ravi is deeper within the VP. Breadth-first order proposes Anita first.

Step 5. Check the stated compatibility

Anita is compatible with she by the question's assumption. Accept it; no need to choose a later incompatible candidate.

Step 6. Give the result and its limit

This is the algorithm's result under the displayed parse and agreement assumptions. It does not prove the intention of every speaker using an otherwise ambiguous sentence.

Answer. She → Anita under the stated tree and compatibility information.

Advantages, limitations and a complete exam answer

AdvantagesLimitations
Explicit candidate order makes the decision traceable.Wrong parse structure can produce the wrong search order.
Requires no labelled coreference-training corpus for its basic rules.World knowledge and discourse intention may override syntactic preference.
Uses structure rather than only surface distance.Some ambiguous, long-distance or unusual constructions need additional treatment.

For a longer answer, write the definition, inputs, tree notation, ordered algorithm, one labelled tree with a candidate table, final antecedent, and limitations. In a worked question show separately: candidate → why proposed → agreement result → accept/reject. Do not equate the selected candidate with a logically guaranteed interpretation when the sentence is ambiguous.

Yaad rakho: Tree order candidate deta hai; agreement usko allow/reject karta hai. Nearest noun automatically answer nahi.

"}],"scope":"TEE only · Unit 6 · syllabus-based supplement; faculty slide pack not supplied"},{"id":"u7","title":"Unit 7 · applications","unit":7,"sections":[{"id":"classification","title":"Text classification and its full pipeline","html":"

Text classification assigns a document to one or more predefined categories. Spam detection usually uses two classes; news classification may use several topics. In single-label multiclass classification, choose one of several categories. In multilabel classification, several labels may apply to the same document.

Define labels
→
Prepare labelled examples
→
Train representation + model
→
Evaluate on unseen text
→
Predict new labels

An exam-ready pipeline

StageWhat you doWhy it matters
Task definitionDecide what counts as spam or which news topics exist.Unclear labels create inconsistent training data.
Data collection and annotationCollect representative examples and assign correct labels.The model learns from these examples.
Split dataSeparate training, validation and test data.Test examples must not influence training choices.
PreprocessTokenize and normalise where suitable; retain meaningful negation.Reduce irrelevant variation without deleting signal.
RepresentUse BoW, TF-IDF or learned embeddings.Models need numerical features.
TrainFit Naive Bayes, logistic regression or another appropriate model.Learn how features predict categories.
EvaluateMeasure relevant precision, recall and F1 on held-out examples.Accuracy alone may hide poor minority-class performance.
Deploy and monitorApply the same preprocessing and monitor changing language.Incoming data can differ from old examples.

Naive Bayes is especially useful for hand calculations. With a bag-of-words representation, its class score is the prior multiplied by the conditional probability of each token; repeated tokens repeat the factor. The “naive” assumption is conditional independence of features given the class, not that words are independent in real language.

\"s(c)=P(c)\\prod_{w\\in
\"P(w\\mid

C(w,d) counts word w in document d. C(w,c) counts it in training documents belonging to class c. Nc is the total token count for class c. Add-one smoothing adds one to every vocabulary count and therefore adds |V| to the denominator. This is the same arithmetic taught for WSD, with document classes instead of senses.

Worked classification · two-token message

A spam classifier has vocabulary {win, prize, meeting}. Priors are P(spam)=P(ham)=0.5. Spam token counts are [3,2,0]; ham counts are [0,0,5]. With add-one smoothing, classify “win prize”.

Given

|V|=3; N(spam)=5; N(ham)=5; message has win once and prize once.

To find

Class scores and predicted label.

Formula and why it applies

\"s(c)=P(c)P(\\text{win}\\mid
\"P(w\\mid

The model is multinomial Naive Bayes; multiply the two token likelihoods and the class prior.

Solution

Step 1. Compute the smoothed denominators

\"5+3=8\"

Step 2. Compute spam likelihoods

\"P(\\text{win}\\mid
\"P(\\text{prize}\\mid

Step 3. Compute ham likelihoods

\"P(\\text{win}\\mid

Step 4. Multiply the spam score

\"s(S)=\\frac12\\times\\frac12\\times\\frac38=\\frac{3}{32}=0.09375\"

Step 5. Multiply the ham score

\"s(H)=\\frac12\\times\\frac18\\times\\frac18=\\frac{1}{128}=0.0078125\"

Step 6. Compare

0.09375 > 0.0078125. These are unnormalised scores; the label is chosen by the larger score.

Predicted label: spam.

Yaad rakho: Class prior × har word ki class-likelihood; sabse bada score wins.

"},{"id":"sentiment","title":"Sentiment analysis: meaning, methods, example","html":"

Sentiment analysis identifies expressed opinion or attitude, often as positive, negative or neutral. Document-level analysis labels a complete text; sentence-level analysis labels a sentence; aspect-level analysis attaches sentiment to a specific feature or entity.

ReviewUseful interpretation
“The display is excellent, but the battery is poor.”Display: positive; battery: negative. One overall label loses this distinction.
“The service is not good.”Negative: negation changes good. Removing not can reverse the intended meaning.
“Great, another two-hour delay.”Likely sarcasm: literal positive word alone is insufficient.

Approaches

Lexicon based: look up sentiment scores, combine them, and apply stated handling for negation/intensifiers. It is interpretable but fragile with context and sarcasm. Supervised: learn a mapping from labelled text features to sentiment labels. Neural/contextual: learn or use contextual representations to capture word interactions, then fine-tune a sentiment classifier. None guarantees correct interpretation of every review.

A specified lexicon calculation

Use good = +2, excellent = +3 and slow = −2. A “not” immediately before a sentiment word reverses its score. Score “The support was not good but the delivery was excellent and slow.”

Given

Three sentiment words; not modifies good only; sum word scores under the stated rule.

To find

Total sentiment score and label, taking positive totals as positive.

Formula and why it applies

\"S=\\sum_i

Apply the given negation rule before summing. This is a simplified specified model, not a universal linguistic rule.

Solution

Step 1. Identify sentiment words

good, excellent and slow.

Step 2. Apply negation

\"s(\\text{not

Step 3. Add the contributions

\"S=-2+3-2=-1\"

Step 4. Map score to label

The sum is negative.

Total score = −1; negative under this rule.
"},{"id":"evaluation","title":"Precision, recall, F1 and accuracy","html":"

Evaluation compares predicted results with known answers. A true positive is a correctly predicted positive; a false positive is a negative item predicted positive; a false negative is a missed positive; a true negative is a correctly rejected negative.

\"\\text{Precision}=\\frac{TP}{TP+FP}\"
\"\\text{Recall}=\\frac{TP}{TP+FN}\"
\"F_1=\\frac{2PR}{P+R}=\\frac{2TP}{2TP+FP+FN}\"
\"\\text{Accuracy}=\\frac{TP+TN}{TP+TN+FP+FN}\"

Precision asks how many predicted positives were correct. Recall asks how many actual positives were found. F1 balances them by a harmonic mean. If a denominator is zero, state the convention rather than dividing by zero.

Example 1 · evaluate a classifier

A binary text classifier gives TP=6, FP=2, FN=4, TN=8. Compute precision, recall, F1 and accuracy.

Given

TP=6; FP=2; FN=4; TN=8.

To find

Four evaluation measures.

Formula and why it applies

\"P=\\frac{TP}{TP+FP},\\quad
\"F_1=\\frac{2TP}{2TP+FP+FN},\\quad

Use predicted positives in precision, actual positives in recall, and all examples in accuracy.

Solution

Step 1. Precision

\"P=\\frac{6}{6+2}=0.750000\"

Step 2. Recall

\"R=\\frac{6}{6+4}=0.600000\"

Step 3. F1

\"F_1=\\frac{12}{12+2+4}=0.666667\"

Step 4. Accuracy

\"A=\\frac{6+8}{6+2+4+8}=0.700000\"
Precision 0.7500; recall 0.6000; F1 0.6667; accuracy 0.7000.

Example 2 · evaluate a classifier

A binary text classifier gives TP=9, FP=3, FN=1, TN=7. Compute precision, recall, F1 and accuracy.

Given

TP=9; FP=3; FN=1; TN=7.

To find

Four evaluation measures.

Formula and why it applies

\"P=\\frac{TP}{TP+FP},\\quad
\"F_1=\\frac{2TP}{2TP+FP+FN},\\quad

Use predicted positives in precision, actual positives in recall, and all examples in accuracy.

Solution

Step 1. Precision

\"P=\\frac{9}{9+3}=0.750000\"

Step 2. Recall

\"R=\\frac{9}{9+1}=0.900000\"

Step 3. F1

\"F_1=\\frac{18}{18+3+1}=0.818182\"

Step 4. Accuracy

\"A=\\frac{9+7}{9+3+1+7}=0.800000\"
Precision 0.7500; recall 0.9000; F1 0.8182; accuracy 0.8000.

Example 3 · evaluate a classifier

A binary text classifier gives TP=4, FP=0, FN=4, TN=12. Compute precision, recall, F1 and accuracy.

Given

TP=4; FP=0; FN=4; TN=12.

To find

Four evaluation measures.

Formula and why it applies

\"P=\\frac{TP}{TP+FP},\\quad
\"F_1=\\frac{2TP}{2TP+FP+FN},\\quad

Use predicted positives in precision, actual positives in recall, and all examples in accuracy.

Solution

Step 1. Precision

\"P=\\frac{4}{4+0}=1.000000\"

Step 2. Recall

\"R=\\frac{4}{4+4}=0.500000\"

Step 3. F1

\"F_1=\\frac{8}{8+0+4}=0.666667\"

Step 4. Accuracy

\"A=\\frac{4+12}{4+0+4+12}=0.800000\"
Precision 1.0000; recall 0.5000; F1 0.6667; accuracy 0.8000.
"},{"id":"spelling","title":"Spelling correction and edit distance","html":"

Spelling correction proposes the intended word or sentence when the observed text contains an error. A non-word error produces an invalid vocabulary item (“recieve”); a real-word error produces an existing word in the wrong context (“I will meat you”). Dictionary lookup can detect many non-words but cannot reliably detect real-word errors.

Detect suspicious text
→
Generate candidates
→
Score edit + context fit
→
Choose correction

Minimum edit distance

Levenshtein distance is the minimum total cost of insertions, deletions and substitutions needed to transform one string into another. In the examples below, each edit costs 1; matching characters cost 0. Transposition is not a single Levenshtein operation. State different costs explicitly if the question gives them.

\"D(i,j)=\\min\\begin{cases}D(i-1,j)+1\\\\D(i,j-1)+1\\\\D(i-1,j-1)+[a_i\\ne

Rows represent prefixes of the source; columns represent prefixes of the target. D(i,j) is the best cost for transforming the first i source characters to the first j target characters. Initialise D(i,0)=i and D(0,j)=j. The three choices are deletion, insertion and diagonal match/substitution.

Example 1 · cat → cut

Calculate the unit-cost Levenshtein distance from “cat” to “cut”. Show the dynamic-programming table.

Given

Source=cat; target=cut; insertion=deletion=substitution=1; match=0.

To find

Minimum edit cost.

Formula and why it applies

\"D(i,j)=\\min(D(i-1,j)+1,D(i,j-1)+1,D(i-1,j-1)+c)\"

c = 0 for matching characters, 1 otherwise. Compare all three possible ways to reach each cell.

Solution

Step 1. Initialise empty prefixes

First row counts insertions; first column counts deletions.

Step 2. Fill row 1: source character “c”

\"D(1,1)=\\min(1+1,\\
\"D(1,2)=\\min(2+1,\\
\"D(1,3)=\\min(3+1,\\

Step 3. Fill row 2: source character “a”

\"D(2,1)=\\min(0+1,\\
\"D(2,2)=\\min(1+1,\\
\"D(2,3)=\\min(2+1,\\

Step 4. Fill row 3: source character “t”

\"D(3,1)=\\min(1+1,\\
\"D(3,2)=\\min(1+1,\\
\"D(3,3)=\\min(2+1,\\

Step 5. Read the completed matrix

Source / target∅cut
∅0123
c1012
a2112
t3221
Minimum edit distance = 1.

Example 2 · at → cat

Calculate the unit-cost Levenshtein distance from “at” to “cat”. Show the dynamic-programming table.

Given

Source=at; target=cat; insertion=deletion=substitution=1; match=0.

To find

Minimum edit cost.

Formula and why it applies

\"D(i,j)=\\min(D(i-1,j)+1,D(i,j-1)+1,D(i-1,j-1)+c)\"

c = 0 for matching characters, 1 otherwise. Compare all three possible ways to reach each cell.

Solution

Step 1. Initialise empty prefixes

First row counts insertions; first column counts deletions.

Step 2. Fill row 1: source character “a”

\"D(1,1)=\\min(1+1,\\
\"D(1,2)=\\min(2+1,\\
\"D(1,3)=\\min(3+1,\\

Step 3. Fill row 2: source character “t”

\"D(2,1)=\\min(1+1,\\
\"D(2,2)=\\min(1+1,\\
\"D(2,3)=\\min(2+1,\\

Step 4. Read the completed matrix

Source / target∅cat
∅0123
a1112
t2221
Minimum edit distance = 1.

Example 3 · cats → cat

Calculate the unit-cost Levenshtein distance from “cats” to “cat”. Show the dynamic-programming table.

Given

Source=cats; target=cat; insertion=deletion=substitution=1; match=0.

To find

Minimum edit cost.

Formula and why it applies

\"D(i,j)=\\min(D(i-1,j)+1,D(i,j-1)+1,D(i-1,j-1)+c)\"

c = 0 for matching characters, 1 otherwise. Compare all three possible ways to reach each cell.

Solution

Step 1. Initialise empty prefixes

First row counts insertions; first column counts deletions.

Step 2. Fill row 1: source character “c”

\"D(1,1)=\\min(1+1,\\
\"D(1,2)=\\min(2+1,\\
\"D(1,3)=\\min(3+1,\\

Step 3. Fill row 2: source character “a”

\"D(2,1)=\\min(0+1,\\
\"D(2,2)=\\min(1+1,\\
\"D(2,3)=\\min(2+1,\\

Step 4. Fill row 3: source character “t”

\"D(3,1)=\\min(1+1,\\
\"D(3,2)=\\min(0+1,\\
\"D(3,3)=\\min(1+1,\\

Step 5. Fill row 4: source character “s”

\"D(4,1)=\\min(2+1,\\
\"D(4,2)=\\min(1+1,\\
\"D(4,3)=\\min(0+1,\\

Step 6. Read the completed matrix

Source / target∅cat
∅0123
c1012
a2101
t3210
s4321
Minimum edit distance = 1.

Noisy-channel correction

\"\\hat

x is the observed misspelling and w a candidate intended word. P(x | w) is the error-model likelihood; P(w) is the language-model prior. In context, replace the isolated-word prior with an appropriate contextual probability. A frequent word with a plausible typo may beat a rarer candidate, but context can reverse the ranking.

Candidate ranking

For observed “teh”, candidate “the” has prior 0.05 and error likelihood 0.02. Candidate “ten” has prior 0.01 and likelihood 0.01. Select the intended word.

Given

Two candidates and their prior/error-model likelihoods are supplied.

To find

Largest noisy-channel score.

Formula and why it applies

\"s(w)=P(x\\mid

Multiply likelihood by prior; the same evidence normaliser is irrelevant to ranking.

Solution

Step 1. Score the

\"s(\\text{the})=0.02\\times0.05=0.001\"

Step 2. Score ten

\"s(\\text{ten})=0.01\\times0.01=0.0001\"

Step 3. Compare

0.001 is ten times 0.0001.

Choose “the”.
"},{"id":"translation","title":"Machine translation","html":"

Machine translation (MT) automatically converts a source-language text into a target-language text while aiming to preserve its meaning and produce natural, grammatical output. The source is the language received; the target is the language generated. Translating words independently is insufficient because languages differ in order, agreement, idioms and word senses.

What a translation must preserve

RequirementWhat to checkExample
ParticipantsWho performed the action and who received itAsha helped Riya must not become Riya helped Asha
Event and polarityWhat happened, and whether it did not happenDo not remove not
Time/aspectWhen and how the event unfoldsis working differs from worked
Entities and quantitiesNames, dates and numbers₹500 must not become ₹5,000
Intended senseMeaning selected by contextFinancial bank versus riverbank
Target grammarNatural order and agreementHindi object-before-verb often becomes English verb-before-object
Source text
→
Tokenize and analyse
→
Resolve structure + sense
→
Choose target words/structure
→
Generate target sentence
→
Check adequacy and fluency

The three major approaches

ApproachProcedureAdvantagesLimitations
Rule-based MTAnalyse source words/grammar → transfer structure or meaning → generate target language using dictionaries and rules.Explicit rules and terminology can be inspected.Rule writing is expensive; exceptions and language pairs require maintenance.
Statistical MTLearn translation correspondences from parallel text; combine translation adequacy with a target-language model; search candidate translations.Learns from examples and can use phrase correspondences.Sparse data, word-order differences and rare phrases cause errors.
Neural MTEncode source context; a decoder generates target tokens conditioned on that context, often using attention.Learns contextual representations and long-range interactions.Needs suitable data/compute; can omit, distort or generate unsupported content.

Parallel corpus: aligned source and target examples expressing corresponding content. A monolingual corpus contains text in only one language. Parallel alignment does not mean every source word maps to exactly one target word: translation may insert, omit or reorder function words to preserve meaning.

In a simple noisy-channel view of statistical MT, let f be the observed source sentence and e a candidate target sentence. The decoder compares translation compatibility and target-language fluency:

\"\\hat

P(f|e) is the translation-model likelihood; P(e) is the target-language-model prior. The common P(f) denominator is omitted for ranking. A modern MT system may use a different objective; this formula explains the stated statistical model rather than every translation system.

Worked application 1 · translation requires reordering

Question. Translate the simple Hindi sentence “मैं पानी पीता हूँ” into natural English and explain the processing stages.

Given

Intended subject is I; object is water; predicate expresses drinking in a habitual/simple-present reading.

To find

A natural translation with preserved roles and the stage-by-stage explanation.

Method and why it applies

Identify roles before generating the target order. Use natural English subject–verb–object order for this sentence rather than copying the Hindi order.

Solution

Step 1. Identify the source components

Source componentFunctionMeaning
मैंSubjectI
पानीObjectwater
पीता हूँPredicatedrink, habitual/simple-present reading

Step 2. Build the meaning relation

The speaker performs the action drink; water is what is drunk. The subject and object must stay attached to these roles.

Step 3. Choose target lexical forms

मैं → I; पानी → water; the predicate is naturally realized as drink with subject I in this reading.

Step 4. Arrange target syntax

English order here is subject I + verb drink + object water.

Step 5. Check adequacy

The participant, action and object are retained. No new object, negation or time expression has been invented.

Step 6. Check fluency

“I drink water” is natural English. A literal “I water drink” retains words but fails the required target order.

Answer. A suitable translation is “I drink water.” The processing preserves meaning while changing structure.

Worked application 2 · one word, two meanings

Question. Explain how an MT system should distinguish bank in “She deposited money in the bank” and “They sat on the bank of the river” before translation.

Given

Two English source sentences contain the same written word bank.

To find

The intended sense in each and suitable Hindi phrase-level renderings.

Method and why it applies

Apply word-sense disambiguation using nearby context before choosing the target term. Identical spelling does not force identical translation.

Solution

Step 1. Collect the first sentence's clues

Deposited and money strongly support the financial-institution sense.

Step 2. Select its target expression

Use बैंक for the financial institution; retain the money-deposit event in the complete translated sentence.

Step 3. Collect the second sentence's clues

River, sat and the location phrase support land beside a river.

Step 4. Select its target expression

Use नदी का किनारा / नदी के किनारे as appropriate to the target sentence. A financial-bank translation would be wrong here.

Step 5. Generate grammatical context

A suitable rendering of the second sentence is “वे नदी के किनारे बैठे।” Grammar adjusts the phrase to its role; do not paste an isolated dictionary form without inflection.

Step 6. State why a dictionary alone fails

A bilingual entry offers candidate words. Context and syntax select and inflect the candidate expressing the intended sense.

Answer. Financial bank → बैंक; riverbank → नदी का किनारा, with grammatical realization such as नदी के किनारे in the sentence.

Worked numerical 3 · compare two statistical translations

Question. A statistical MT system compares candidate A with P(f|A)=0.12 and P(A)=0.03, and candidate B with P(f|B)=0.08 and P(B)=0.06. Select the candidate under the noisy-channel product model.

Given

Two candidate translations; likelihoods and target-language priors are provided. Ignore all other candidates for this ranking exercise.

To find

Each score and the selected translation.

Method and why it applies

Multiply translation likelihood by target-language prior because this specified model values both compatibility and fluency. These products are comparison scores, not normalized posterior probabilities.

Solution

Step 1. Write the scoring rule

\"s(e)=P(f\\mid

Step 2. Compute candidate A

\"s(A)=0.12\\times0.03=0.0036.\"

Step 3. Compute candidate B

\"s(B)=0.08\\times0.06=0.0048.\"

Step 4. Compare the products

0.0048 > 0.0036. B wins even though A had a larger translation likelihood; B's target-language prior compensates.

Step 5. Interpret the result

The model prefers B under the supplied numbers. The score alone does not establish that B is semantically correct; a poor model can rank a bad translation highly.

Answer. Select B, with score 0.0048 versus A's 0.0036.

Evaluation and common failure cases

Adequacy asks whether the original information and intended meaning were preserved. Fluency asks whether the target text is natural and grammatical. A fluent sentence can still mistranslate a negation or number, so assess both. Automatic reference-overlap metrics provide a useful signal, but several valid translations can use different words.

FailureWhy it occursWhat the system needs
Word-for-word orderSource and target syntax differStructural modelling/reordering
Wrong bank/bat senseMultiple senses share a written formContext-sensitive lexical choice
Idiomatic phrase translated literallyMeaning is not the sum of isolated wordsPhrase/context knowledge
Rare name or specialist term corruptedWeak vocabulary/domain coverageAppropriate data and terminology handling
Missing negation or numberOmission or generation errorContent-preservation checks
Low-resource language pairFew representative parallel examplesSuitable data collection/adaptation

For a 10-mark design explanation, connect the definition, pipeline, three approaches, worked source-to-target example, data requirements, evaluation and limitations. Merely listing “rule based, statistical, neural” does not explain how a translation is produced.

Yaad rakho: MT: pehle kaun-kya-kisko samjho; phir target language ke words aur order mein wahi meaning banao.

"},{"id":"qa","title":"Question answering","html":"

Question answering (QA) returns an answer to a natural-language question using available evidence or knowledge. Information retrieval usually returns ranked documents; QA must identify the requested information and deliver the answer. A relevant document is useful evidence but is not automatically the answer.

Question types and answer types

QuestionExpected answer typeRequired operation
Who founded the company?Person or organisationFind the founder relation
When did the museum open?Date/timeLocate the opening date
Where is the campus?LocationLocate the campus-location relation
How many books were issued?QuantityRead or compute the stated count
Why was the train cancelled?Reason/explanationFind the causal evidence
Is the office open on Sunday?Yes/no with evidenceCheck the relevant schedule statement

Closed-domain QA operates within a restricted area such as a university handbook. Open-domain QA addresses a wider range of subjects and usually needs broad retrieval. Extractive QA selects a span from a passage. Generative QA constructs answer text; it still must preserve the evidence and avoid unsupported details.

Question analysis
→
Evidence retrieval
→
Passage/record understanding
→
Answer selection or composition
→
Support check
→
Present answer

Explain each module

ModuleWhat happensWhy it matters
Question analysisIdentify topic, intent, constraints and answer type.When asks for a time, not a place.
RetrievalFind a relevant passage or structured fact.Reduces the search space.
Evidence understandingParse relations, resolve references and ambiguous words.The correct date must belong to the correct event.
Answer formationExtract a span or compose a short supported response.Turns a document into an actual answer.
VerificationCheck whether every claimed detail follows from evidence.Prevents filling gaps by guessing.
PresentationReturn a concise answer with suitable context.Avoids dumping unrelated text.

Worked application 1 · extract the correct fact

Question. Given “The City Museum opened in 1998 in Pune. It was renovated in 2016”, answer “When did the museum open?” Show the QA stages.

Given

A two-sentence passage. Dates 1998 and 2016 refer to different events.

To find

The opening date, not the renovation date.

Method and why it applies

Use extractive QA: identify the requested event and expected date type, then choose the span linked to that event.

Solution

Step 1. Analyse the question

When requires a date/time answer. The event is opening; the entity is the City Museum.

Step 2. Retrieve the relevant sentence

The first sentence mentions the museum and its opening. The second sentence discusses renovation.

Step 3. Identify event–date relations

EventDate
opened1998
renovated2016

Step 4. Select the answer span

1998 is attached to opened, so select the four-character year span 1998.

Step 5. Reject the distractor

2016 is also a date, but it answers a different question: when was the museum renovated?

Step 6. Present the answer

“The museum opened in 1998.”

Answer. 1998. Matching only the answer type date would be insufficient without matching the event.

Worked application 2 · resolve a pronoun before answering

Question. Given “Mira joined Orion Labs in 2022. She became its research lead in 2024”, answer “Who became Orion Labs' research lead, and when?”

Given

The passage contains the name Mira, pronoun She, organisation Orion Labs and possessive its. Assume the normal reading that She refers to Mira and its to Orion Labs.

To find

The person and the date of becoming research lead.

Method and why it applies

Combine reference resolution with relation extraction. The question asks for two fields, both tied to the promotion/role event.

Solution

Step 1. Identify requested fields

Who → person; when → date. The relation is became research lead of Orion Labs.

Step 2. Resolve mentions

MentionReferent
SheMira
itsOrion Labs

Step 3. Read the target event

After resolution, the second sentence states that Mira became Orion Labs' research lead in 2024.

Step 4. Distinguish the dates

2022 belongs to joining the organisation. 2024 belongs to becoming research lead.

Step 5. Compose a supported answer

“Mira became Orion Labs' research lead in 2024.”

Step 6. Verify completeness

The answer supplies both requested fields and does not add an unsupported department, qualification or reason.

Answer. Mira, in 2024. Reference resolution connects the second sentence to the named person and organisation.

Worked application 3 · detect insufficient evidence

Question. A handbook excerpt states “The library opens at 8 a.m. on weekdays.” A user asks “At what time does it close on Sunday?” Explain the correct QA response.

Given

Only the opening time for weekdays is supplied. No closing time or Sunday schedule is present.

To find

Whether an answer is supported and how to handle the missing information.

Method and why it applies

Compare every question constraint against the evidence. Topic overlap with library is not sufficient: time type and day must also match.

Solution

Step 1. Extract the question constraints

ConstraintRequested
EntityLibrary
EventClosing
DaySunday
Answer typeTime

Step 2. Extract the supplied fact

Entity=library; event=opening; day=weekdays; time=8 a.m.

Step 3. Identify mismatches

Opening is not closing. Weekdays do not supply a Sunday schedule.

Step 4. Avoid the unsupported substitution

Do not answer 8 a.m. just because it is the only time in the passage.

Step 5. Return an evidence-limited response

“The excerpt does not state the library's Sunday closing time.” A system with further retrieval can search a relevant schedule.

Step 6. State the benefit

The system avoids converting a relevant-looking number into an incorrect answer.

Answer. No Sunday closing time is supported by the excerpt. Report insufficient evidence or retrieve the missing schedule.

Advantages, limits and evaluation

Design choiceBenefitLimitation
Extractive answerThe returned wording comes directly from evidence.The answer may require combining sentences or may not be present as one span.
Generative answerCan combine supported facts into a readable response.Can introduce unsupported information unless carefully checked.
Closed-domain collectionControlled terminology and relevant resources.Cannot answer facts outside its knowledge source.
Broad retrievalWider question coverage.Irrelevant, outdated or conflicting sources can be retrieved.

Evaluate exact match when a canonical answer string is expected, token overlap/F1 when wording may vary, and factual correctness and completeness when answers are composed. An answer can be grammatically fluent but wrong. Questions with no answer in the evidence should be included in evaluation so the system's restraint is also tested.

For a longer exam answer, define QA, distinguish it from retrieval, explain answer types and system variants, draw the pipeline, work through a passage including distractors, and discuss ambiguity, missing evidence and evaluation. A list of chatbot names is not a QA design.

Yaad rakho: Question ka entity + event + constraint match karo. Passage mein koi bhi number mil gaya matlab answer nahi.

"},{"id":"extraction","title":"Information extraction","html":"

Information extraction (IE) converts unstructured language into structured information such as named entities, relations and events. A structured output has explicit fields or records that can be stored, searched or analysed. IE is broader than NER: finding two names does not yet determine how those entities are related.

The three levels of extracted information

LevelWhat it identifiesExample
Entity extraction / NERA span and its entity typeAsha → person; Orbit Labs → organisation
Relation extractionA labelled connection between entitiesworks-for(Asha, Orbit Labs)
Event extractionAn occurrence with roles and attributesJoining: employee=Asha, employer=Orbit Labs, date=2025

An entity is a participant such as a person or organisation. A relation links participants. An event describes something that happened, may happen or did not happen. An event role names a participant's function: buyer, seller, employee, employer, location or time. These labels must follow the chosen extraction schema.

Raw text
→
Sentence/token processing
→
Entity spans + types
→
Reference resolution
→
Relations and event roles
→
Normalize and store records

An extraction pipeline explained

StepOperationReason
Specify schemaChoose fields and allowed relations/events.A system needs to know what records it is building.
PreprocessSegment sentences and tokenize without losing useful structure.Locate expressions and boundaries.
Recognise entitiesFind spans and assign types.Names can contain several words.
Resolve referencesLink pronouns/descriptions to the right entities.Two mentions may denote one participant.
Extract links/eventsUse patterns, syntax or learned models.Co-occurrence alone does not identify a relationship.
Handle qualifiersPreserve negation, time and uncertainty.Not hired differs from hired.
NormalizeStandardize dates, names or quantities where justified.Equivalent forms should be comparable.
Store/evaluateSave records and compare them with annotated answers.Wrong links matter even if entity names are correct.

Worked application 1 · turn a sentence into a record

Question. Extract entities and a joining event from “Asha joined Orbit Labs in Mumbai on 3 May 2025.” Use fields person, organisation, location, event and date.

Given

One affirmative sentence; Orbit Labs is an organisation and Mumbai is a location.

To find

Entity spans/types and one structured joining record.

Method and why it applies

Find the complete entity spans, then assign their roles using the verb joined and its grammatical context.

Solution

Step 1. Mark entity spans

SpanType
AshaPerson
Orbit LabsOrganisation
MumbaiLocation
3 May 2025Date

Step 2. Identify the event trigger

Joined signals a joining event. The subject Asha is the joining person; Orbit Labs is the joined organisation.

Step 3. Attach contextual fields

In Mumbai supplies a location associated with the stated joining event. On 3 May 2025 supplies its date. Do not infer the company's permanent headquarters from this sentence alone.

Step 4. Normalize the date

The unambiguous date can be stored as 2025-05-03 while preserving its original text.

Step 5. Build the record

FieldValue
eventjoining
personAsha
organisationOrbit Labs
locationMumbai
date2025-05-03

Step 6. State the information limit

The sentence supports a joining event on that date. By itself it does not prove that Asha still works there indefinitely.

Answer. A joining record for Asha and Orbit Labs at the stated location/date, with complete entity spans preserved.

Worked application 2 · direction of a relation matters

Question. From “Nova acquired Luma for ₹50 crore in 2025”, extract an acquisition event with buyer, acquired organisation, amount and year.

Given

Nova and Luma are organisation names. The active verb is acquired; no seller is named.

To find

A correctly directed acquisition record.

Method and why it applies

Use grammatical roles to identify buyer and acquired organisation. Merely returning the unordered pair {Nova,Luma} loses the event direction.

Solution

Step 1. Recognise participant entities

Nova is the subject organisation; Luma is the object organisation.

Step 2. Locate the trigger

Acquired identifies an acquisition event.

Step 3. Assign roles

RoleEntity
Buyer/acquirerNova
Acquired organisationLuma

Step 4. Extract amount and time

Amount text=₹50 crore; year=2025. Currency and magnitude belong together; do not drop crore or replace it by lakh.

Step 5. Create the record

FieldValue
eventacquisition
buyerNova
acquired_organisationLuma
amountINR 50 crore
year2025
sellerNot stated

Step 6. Reject an unsupported inference

The sentence does not name the former owner/seller. Luma is the acquired organisation, not automatically the seller.

Answer. Acquisition(buyer=Nova, acquired organisation=Luma, amount=INR 50 crore, year=2025). Seller remains unspecified.

Worked application 3 · reference, negation and time

Question. Extract employment-related events from “Ravi joined Vega in 2021. He left the company in 2024. He did not join Orion.” Keep event type, person, organisation, date and polarity.

Given

He refers to Ravi; the company refers to Vega. The last sentence explicitly negates joining Orion.

To find

A structured event list without claiming an unsupported current employer.

Method and why it applies

Resolve references, identify each event separately and preserve whether it is affirmed or negated.

Solution

Step 1. Resolve the repeated mentions

MentionReferent
He in sentence 2Ravi
the companyVega
He in sentence 3Ravi

Step 2. Extract the first event

Joining; person=Ravi; organisation=Vega; date=2021; polarity=affirmed.

Step 3. Extract the second event

Leaving; person=Ravi; organisation=Vega; date=2024; polarity=affirmed.

Step 4. Extract the third statement

Joining; person=Ravi; organisation=Orion; date=not stated; polarity=negated. This must not create an affirmative employment relation with Orion.

Step 5. Write the event table

EventPersonOrganisationDatePolarity
joiningRaviVega2021Affirmed
leavingRaviVega2024Affirmed
joiningRaviOrionNot statedNegated

Step 6. Interpret current status cautiously

The text says that Ravi left Vega and did not join Orion. It supplies no other current employer. Do not fill that field by guessing.

Answer. Three qualified event statements, including a negated Orion joining. No current employer can be established from this text.

Approaches and their trade-offs

ApproachMechanismAdvantageLimitation
Dictionary/rule basedMatch known names, trigger words and patterns such as person joined organisation.Transparent and useful in a narrow domain.New names, paraphrases and complex syntax can escape patterns.
Supervised statisticalLearn entity/relation/event labels from annotated examples and features.Combines several cues consistently.Requires representative labels and careful feature design.
Neural/contextualUse contextual representations to classify spans and links or generate a structured record.Can capture paraphrases and broader context.May still invent or misassign fields; schema and factual checks remain necessary.

Applications include extracting company events from news, conditions/treatments from clinical text, skills from resumes and attributes from product descriptions. Extraction quality is measured against annotated records: an entity match can require the correct span and type; a relation match additionally requires the correct participants and direction. Precision measures how many extracted records are correct; recall measures how many gold records were found.

For a 10-mark answer, begin with the definition and structured output, distinguish NER/relation/event extraction, draw and explain the pipeline, work through a record with roles and qualifiers, compare approaches, then state evaluation and limitations. “Extract keywords” alone is too narrow: keywords do not necessarily represent a factual relation.

Yaad rakho: NER naam pakadta hai; relation kaun-kisse jodta hai; event kya hua batata hai. Negation aur date mat girao.

"},{"id":"retrieval","title":"Information retrieval","html":"

Information retrieval (IR) locates and ranks items relevant to an information need, usually expressed as a query. The collection may contain documents, product descriptions or scientific papers. Retrieval estimates relevance; it does not itself establish that every retrieved claim is true.

Documents → cleaning
→
Build index
→
Query → same representation
→
Match & rank
→
Return relevant results

The inverted index

An inverted index maps a term to the documents containing it, often with positions or counts. For D1 “cats drink milk” and D2 “dogs drink water”, the posting list for drink is {D1,D2}, while milk maps only to D1. This avoids rescanning every document for every query.

TermPosting list
catsD1
drinkD1, D2
milkD1
dogsD2
waterD2

Matching and ranking

Boolean retrieval combines term membership using AND, OR and NOT. Vector-space retrieval represents documents and queries as weighted vectors, for example with TF-IDF, and compares them with cosine similarity. Learned retrieval can use dense embeddings for meaning-based matching. The representation used at query time must be compatible with the document index.

Search engines and recommendation

A search engine receives an explicit query and ranks matching indexed pages. A recommender may derive an implicit query from a user’s liked items or profile and retrieve similar candidates. Both require candidate retrieval and ranking, but the source of the information need differs.

Evaluate the results

For retrieval, TP is the number of retrieved relevant documents, FP retrieved irrelevant documents, and FN relevant documents not retrieved. Precision and recall use the same formulas as classification. Ranking measures also consider the positions of relevant items; a relevant result at rank 1 is more useful than one far below.

Retrieval evaluation

A search system retrieves 8 documents; 6 are relevant. There are 10 relevant documents in the collection. Find precision, recall and F1.

Given

Retrieved=8; relevant retrieved=6; all relevant=10.

To find

P, R and F1.

Formula and why it applies

\"P=\\frac{\\text{relevant
\"F_1=\\frac{2PR}{P+R}\"

The denominator is 8 for precision and 10 for recall because they ask different questions.

Solution

Step 1. Identify the confusion counts

TP=6; FP=8−6=2; FN=10−6=4.

Step 2. Precision

\"P=\\frac68=0.75\"

Step 3. Recall

\"R=\\frac6{10}=0.6\"

Step 4. F1

\"F_1=\\frac{2(0.75)(0.6)}{0.75+0.6}=\\frac{0.9}{1.35}=\\frac23\\approx0.666667\"
Precision=0.75, recall=0.60, F1≈0.666667.

Yaad rakho: Precision: jo laaye, unmein kitne sahi? Recall: jo sahi the, unmein kitne laaye?

"}],"scope":"TEE only · Unit 7 · current syllabus, including spelling correction; faculty slide pack not supplied"}],"formulas":[{"id":"formula-index","title":"Core formulas","unit":0,"sections":[{"id":"all","title":"Core formula sheet","html":"

Read each formula with its meaning. The remaining lessons in this tab contain the complete worked examples for these methods. Use the same vocabulary, boundary, case and token-count conventions throughout one solution.

N-gram count

\"\\max(T-n+1,0)\"

Unpadded sequence of T tokens. Count only valid contiguous windows.

Unigram MLE

\"P(w)=C(w)/N\"

N is the training-token total under the stated boundary convention.

Bigram MLE

\"P(w_i\\mid

Denominator = history count.

Trigram MLE

\"P(w_i\\mid

Denominator = two-word history count.

Add-one / Laplace

\"P(w\\mid

Add one to every possible vocabulary continuation.

Sentence probability

\"P(W)\\approx\\prod_iP(w_i\\mid

Multiply in order; include an end factor if specified.

Perplexity

\"PP=P(W)^{-1/N}\"

N is the number of predicted tokens/events used by the question.

Emission

\"P(w\\mid

Word given tag; count tokens with that tag.

Transition

\"P(t_j\\mid

Next tag given previous tag; count outgoing transitions.

HMM joint path

\"P(W,T)=\\pi_{t_1}b_{t_1}(w_1)\\prod_{i=2}^{n}a_{t_{i-1},t_i}b_{t_i}(w_i)\"

Multiply initial, emission and transition factors; add end probability if used.

Tag sequences

\"K^T\"

K tags, T word positions; count before constraints.

Viterbi

\"\\delta_i(t)=b_t(w_i)\\max_s[\\delta_{i-1}(s)a_{s,t}]\"

Choose the largest predecessor score; store a backpointer.

Lesk

\"score(s)=|context\\cap

Use the specified normalisation and overlap convention.

Naive Bayes class prior

\"P(c)=D_c/D\"

D counts labelled examples; this is not the word-token total.

Smoothed NB likelihood

\"P(w\\mid

N_c counts training tokens in class c.

NB decision

\"\\hat

Select label with largest score; normalise only for posterior values.

Yaad rakho: Condition jis par hai, denominator usi ka; sequence mein multiply; options mein maximum.

"}],"scope":"M2 · formulas with meaning · full examples follow"},{"id":"f-counts","title":"N-grams & probability","unit":0,"sections":[{"id":"n-grams-construction","title":"Unigrams, bigrams and trigrams: construct them correctly","html":"

Definition. An n-gram is a contiguous sequence of n items. For word n-grams, the items are consecutive word tokens. A unigram contains one word; a bigram contains two; a trigram contains three. Character n-grams use characters instead, so always identify the unit being counted.

Tokens: we | study | natural | language
→
Move a window one token at a time
→
Bigrams: we study / study natural / natural language

For m tokens without padding, the number of n-gram positions is max(0, m−n+1). Repeated positions may contain the same n-gram type. A bag-of-n-grams representation counts each n-gram type and loses the overall sentence order beyond the local order inside each n-gram.

\"\\text{Number

Question. List all word unigrams, bigrams and trigrams for “the prime minister of our country”. Do not add boundary tokens.

Given

Six tokens: the, prime, minister, of, our, country. n = 1, 2 and 3.

To find

All three lists and their counts.

Formula and why it applies

\"\\#\\text{positions}=m-n+1\"

Each window must contain consecutive words and move one position. No wraparound or invented final word is allowed.

Solution

Step 1. Take one word at a time

Unigrams: (the), (prime), (minister), (of), (our), (country). Count = 6−1+1 = 6.

Step 2. Take two consecutive words

Bigrams: (the, prime), (prime, minister), (minister, of), (of, our), (our, country). Count = 6−2+1 = 5.

Step 3. Take three consecutive words

Trigrams: (the, prime, minister), (prime, minister, of), (minister, of, our), (of, our, country). Count = 6−3+1 = 4.

Final answer. 6 unigrams, 5 bigrams and 4 trigrams.

Question. Construct bigrams for “I like tea” using one start and one end marker.

Given

Original tokens: I, like, tea. Add <s> and </s> once each.

To find

The boundary-aware bigram sequence.

Formula and why it applies

\"\\#\\text{bigrams}=(m+2)-1=m+1\"

The start context predicts the first word, and the last word predicts that the sentence ends.

Solution

Step 1. Write the padded sequence

<s> I like tea </s>

Step 2. Move a two-token window

(<s>, I), (I, like), (like, tea), (tea, </s>).

Step 3. Count the windows
\"3+1=4\"
Final answer. There are four boundary-aware bigrams.

Question. Construct trigrams for “I like tea” using two start markers and one end marker.

Given

Original sentence has three words. For a trigram model each predicted token needs two prior context tokens.

To find

All trigram prediction events.

Formula and why it applies

\"P(w_i\\mid

Two start markers provide a complete context even for the first word.

Solution

Step 1. Pad with two starts

<s> <s> I like tea </s>

Step 2. Slide a three-token window

(<s>, <s>, I), (<s>, I, like), (I, like, tea), (like, tea, </s>).

Step 3. Count predicted events

The model predicts I, like, tea and the end marker: four predictions.

Final answer. Four trigrams. Start markers are context; they are not ordinary predicted words.

Question. Give the bag-of-bigrams counts for “a b a b” without padding.

Given

Four tokens in their given order.

To find

Bigram positions, distinct bigram types and their counts.

Formula and why it applies

\"\\#\\text{positions}=4-2+1=3\"

A bag stores frequencies; repeated bigrams must not be discarded.

Solution

Step 1. Enumerate all positions

(a,b), (b,a), (a,b).

Step 2. Group identical pairs
Bigram typeCount
a b2
b a1
Final answer. Three bigram positions but two distinct bigram types.

Yaad rakho: n-gram mein n words; context mein n−1 words.

"},{"id":"unigram-estimation","title":"Unigram estimation and complete worked examples","html":"

Maximum likelihood estimation (MLE) chooses probabilities equal to observed relative frequencies. In a unigram model, every word is predicted independently of its surrounding words. If a word occurs c times in N training tokens, its estimate is c/N. To score m specified words, multiply their m unigram probabilities.

\"P_{\\mathrm{MLE}}(w)=\\frac{C(w)}{N}\\qquad

Choose a tokenization convention before counting. Lowercasing, punctuation handling and boundary markers can change the vocabulary and counts. Never count words with one convention and form the denominator with another. A unigram model gives the same product to rearrangements of the same word multiset; that is one of its limitations.

Question. Train an unpadded, case-insensitive unigram model on “I like tea” and “I like coffee”. Find the probability of “I like tea”.

Given

Training sentences: I like tea; I like coffee. Use lowercase words; exclude punctuation and sentence boundaries.

To find

The vocabulary, word probabilities and test-sentence probability.

Formula and why it applies

\"P(w)=C(w)/N,\\qquad

The unigram approximation ignores order and uses total token count for every denominator.

Solution

Step 1. Count every training token

Each sentence contains three words, so N = 3+3 = 6.

Step 2. Count types and frequencies
WordCountMLE probability
i22/6 = 1/3
like22/6 = 1/3
tea11/6
coffee11/6

Vocabulary size V = 4.

Step 3. Write one factor per test word
\"P=\\frac{2}{6}\\times\\frac{2}{6}\\times\\frac{1}{6}\"
Step 4. Multiply the fractions
\"P=\\frac{4}{216}=\\frac{1}{54}\\approx0.018519\"
Final answer. The unigram sentence probability is approximately 0.018519.

Question. A corpus contains counts a: 4, b: 3 and c: 1. Find the unigram probability of “a b a”.

Given

C(a)=4, C(b)=3, C(c)=1. No boundary markers.

To find

P(a b a).

Formula and why it applies

\"P(a\\,b\\,a)=P(a)P(b)P(a)\"

A repeated test word contributes a probability factor at every occurrence; do not include it just once.

Solution

Step 1. Find the total
\"N=4+3+1=8\"
Step 2. Estimate the required probabilities
\"P(a)=4/8=1/2,\\qquad
Step 3. Use both occurrences of a
\"P=\\frac12\\times\\frac38\\times\\frac12\"
Step 4. Simplify
\"P=\\frac3{32}=0.09375\"
Final answer. P(a b a) = 0.09375.

Yaad rakho: Repeated word ka probability bhi repeat multiply hoga.

Question. An unsmoothed unigram model is trained on “red blue red”. Find the probability of “red green”.

Given

N=3. Counts: red=2, blue=1, green=0. The test contains red and green.

To find

The unsmoothed test probability and the reason for its value.

Formula and why it applies

\"P(red\\

MLE assigns zero to an unseen outcome. Smoothing is needed if an unseen in-vocabulary item must receive probability.

Solution

Step 1. Estimate red
\"P(red)=2/3\"
Step 2. Estimate the unseen word
\"P(green)=0/3=0\"
Step 3. Multiply
\"(2/3)\\times0=0\"
Final answer. Unsmoothed probability = 0. This reflects missing training evidence, not proof that the phrase is impossible.
"},{"id":"bigram-count-table","title":"Bigram counts: build the table before multiplying","html":"

A bigram probability table uses the previous word as its row and the next word as its column. Divide each cell count by its row's total outgoing count. With one end marker on every sentence, each ordinary training word has a continuation, so its outgoing count equals its token frequency. Do not create a bigram that crosses from one sentence into the next.

\"P(w\\mid
Add boundaries to each sentence separately
→
List adjacent pairs
→
Count repeated pairs
→
Divide each row by its total
→
Follow the test sentence through the table

Question. Train a bigram model on “I like tea”, “I like coffee” and “you like tea”. Use one start and one end marker. Find the probability of “I like tea”.

Given

Corpus: <s> I like tea </s>; <s> I like coffee </s>; <s> you like tea </s>. Preserve these tokens.

To find

All nonzero bigram estimates and the test probability.

Formula and why it applies

\"P(w\\mid

Every sentence starts and ends independently. Four probability factors are required for the three test words plus end.

Solution

Step 1. List the training pairs

Sentence 1: (<s>,I), (I,like), (like,tea), (tea,</s>).
Sentence 2: (<s>,I), (I,like), (like,coffee), (coffee,</s>).
Sentence 3: (<s>,you), (you,like), (like,tea), (tea,</s>).

Step 2. Create the nonzero count table
History → nextPair countHistory countProbability
<s> → I232/3
<s> → you131/3
I → like221
you → like111
like → tea232/3
like → coffee131/3
tea → </s>221
coffee → </s>111
Step 3. Write the test path
\"P=P(I\\mid\\langle
Step 4. Substitute each matching row
\"P=\\frac23\\times1\\times\\frac23\\times1\"
Step 5. Multiply
\"P=\\frac49\\approx0.444444\"
Final answer. P(I like tea) = 4/9 ≈ 0.444444.

Question. Using the same three-sentence corpus given here—“I like tea”, “I like coffee”, “you like tea”—find the bigram probabilities of “you like coffee” and “tea like I”. Include boundaries.

Given

Corpus with one start/end marker per sentence. Counts: start→you=1/3; you→like=1; like→coffee=1/3; coffee→end=1. No training sentence starts with tea.

To find

The two test probabilities.

Formula and why it applies

\"P(\\text{sentence})=\\prod

A model can assign a nonzero probability to a sentence it never saw if all of its local bigrams were seen.

Solution

Step 1. Score you like coffee
\"P=\\frac13\\times1\\times\\frac13\\times1=\\frac19\\approx0.111111\"
Step 2. Inspect tea like I
\"P(tea\\mid\\langle
Step 3. Apply the zero factor

The full product is zero, regardless of the remaining factors.

Final answer. P(you like coffee) = 1/9; P(tea like I) = 0. A new sentence is not necessarily a zero-probability sentence.

Question. Obtain the unsmoothed bigram probability of “students are from Vellore” from this corpus: “I am from Vellore”; “I am a teacher”; “students are good and are from various cities”; “students from Vellore do engineering”. Use sentence boundaries.

Given

Four training sentences, each padded with one <s> and </s>. Test sequence: <s> students are from Vellore </s>.

To find

The five required conditional probabilities and their product.

Formula and why it applies

\"P(w\\mid

Count all occurrences in the entire corpus. In particular, are occurs twice inside the third sentence, and Vellore is followed by end once and do once.

Solution

Step 1. Count the histories
HistoryOccurrences with continuation
<s>4
students2
are2
from3
Vellore2
Step 2. Count the required pairs
Required pairCountProbability
<s> students22/4
students are11/2
are from11/2
from Vellore22/3
Vellore </s>11/2
Step 3. Write the complete product
\"P=\\frac24\\times\\frac12\\times\\frac12\\times\\frac23\\times\\frac12\"
Step 4. Multiply progressively
\"\\frac24\\times\\frac12=\\frac14,\\quad
\"\\frac18\\times\\frac23=\\frac1{12},\\quad
Final answer. P = 1/24 ≈ 0.0416667.

Yaad rakho: Same sentence mein word do baar aaye toh count bhi do.

"},{"id":"trigram-estimation","title":"Trigram estimates: two-word history, not two denominators","html":"

A trigram model predicts the next word from the two preceding words. The numerator is the count of a triple. The denominator is the count of its two-word history with an available next token. Do not multiply two independent bigram probabilities and call that a trigram probability.

\"P(w_i\\mid

Question. A corpus contains 12 occurrences of “prime minister”. Eight are followed by “of”, three by “said”, and one by “visited”. Estimate the three trigram probabilities.

Given

History h = (prime, minister). Count of this history = 12. Triple counts for of, said, visited are 8, 3, 1.

To find

The conditional distribution of the next word.

Formula and why it applies

\"P(w\\mid

All three candidates share one two-word history, so they share denominator 12.

Solution

Step 1. Calculate of
\"P(of\\mid
Step 2. Calculate said
\"P(said\\mid
Step 3. Calculate visited
\"P(visited\\mid
Step 4. Confirm normalization
\"8/12+3/12+1/12=12/12=1\"
Final answer. The probabilities are 2/3, 1/4 and 1/12.

Question. Train a trigram model on “I like tea”, “I like coffee” and “you like tea”. Use two start markers and one end marker. Score “I like tea”.

Given

Three sentences. Padded first sentence is <s> <s> I like tea </s>; pad the other two in the same way.

To find

Four trigram prediction probabilities and the sentence probability.

Formula and why it applies

\"P=\\prod_i

Each prediction uses a two-token context; the two start markers supply the first history.

Solution

Step 1. Count the first-word event

The history (<s>,<s>) occurs three times, and two are followed by I.

\"P(I\\mid\\langle
Step 2. Count the second-word event

(<s>,I) occurs twice; both are followed by like.

\"P(like\\mid\\langle
Step 3. Count the third-word event

(I,like) occurs twice; one is followed by tea.

\"P(tea\\mid
Step 4. Count the end event

(like,tea) occurs twice; both are followed by end.

\"P(\\langle/s\\rangle\\mid
Step 5. Multiply all four
\"P=\\frac23\\times1\\times\\frac12\\times1=\\frac13\"
Final answer. Trigram P(I like tea) = 1/3. It differs from the bigram estimate because the context and denominator changed.

Question. A trigram model supplies P(we|<s>,<s>)=0.4, P(study|<s>,we)=0.5, P(NLP|we,study)=0.8 and P(</s>|study,NLP)=0.25. Score “we study NLP”.

Given

Four trigram conditional factors. The end marker is included.

To find

The complete sentence probability.

Formula and why it applies

\"P=P(we\\mid

Use each supplied two-word history exactly; do not swap its word order.

Solution

Step 1. Insert the values
\"P=0.4\\times0.5\\times0.8\\times0.25\"
Step 2. Multiply the first pair
\"0.4\\times0.5=0.2\"
Step 3. Include NLP
\"0.2\\times0.8=0.16\"
Step 4. Include the end
\"0.16\\times0.25=0.04\"
Final answer. P = 0.04.

Yaad rakho: Trigram: teen-word count upar, pehle do-word count neeche.

Why larger n is not automatically better. More context can capture local order more precisely, but each longer history occurs less often. Parameter count, storage and sparsity grow. A model trained on Shakespeare may assign poor probabilities to financial news because training and test distributions differ. Keep a held-out test set; do not evaluate only on memorized training sentences.

"}],"scope":"Formulas explained through full worked examples"},{"id":"f-smoothing","title":"Smoothing & perplexity","unit":0,"sections":[{"id":"laplace-basics","title":"Laplace smoothing: why both numerator and denominator change","html":"

Smoothing reallocates some probability mass to events that received zero or unreliable estimates from finite training data. Laplace, or add-one, smoothing adds one pseudo-count to every possible next outcome. If there are V possible outcomes, the total added count is V, so adding 1 only to the numerator would produce probabilities that do not sum to 1.

\"P_{\\mathrm{Laplace}}(w)=\\frac{C(w)+1}{N+V}\"
\"P_{\\mathrm{Laplace}}(w\\mid

For unigram smoothing, N is the total training token count. For bigram smoothing, C(h) is the count of the particular history. In both cases V is the number of outcomes being predicted, not the number of training sentences and not the number of distinct bigrams. When end is a possible prediction, include it in V; the start marker is usually context only.

Observed counts: 3, 1, 0
→
Add one to every outcome: 4, 2, 1
→
New total: 7
→
Smoothed probabilities: 4/7, 2/7, 1/7

Question. A fixed unigram vocabulary is {a,b,c}. The training counts are 3,1,0. Compute unsmoothed and Laplace-smoothed probabilities.

Given

C(a)=3, C(b)=1, C(c)=0; N=4; V=3.

To find

The complete distributions.

Formula and why it applies

\"P(w)=C(w)/N,\\qquad

The zero-count c is still in the fixed vocabulary, so it must receive one pseudo-count and contribute to V.

Solution

Step 1. Compute unsmoothed values
WordUnsmoothed
a3/4 = 0.75
b1/4 = 0.25
c0/4 = 0
Step 2. Calculate the new denominator
\"N+V=4+3=7\"
Step 3. Add one to each numerator
\"C(a)+1=4,\\quad
Step 4. Divide
WordSmoothed
a4/7 ≈ 0.571429
b2/7 ≈ 0.285714
c1/7 ≈ 0.142857
Step 5. Confirm the total
\"4/7+2/7+1/7=1\"
Final answer. Smoothed probabilities: a=4/7, b=2/7, c=1/7. Common a loses mass; unseen c gains mass.

Question. After history I, a corpus records like three times, study once and sleep zero times. The allowed next vocabulary is {like,study,sleep}. Compute all Laplace-smoothed bigram probabilities.

Given

C(I,like)=3; C(I,study)=1; C(I,sleep)=0. History count C(I)=4. V=3.

To find

P_L(like|I), P_L(study|I), P_L(sleep|I).

Formula and why it applies

\"P_L(w\\mid

This is smoothing within the I row; use its total 4, not the global corpus token count.

Solution

Step 1. Form the common denominator
\"C(I)+V=4+3=7\"
Step 2. Calculate like
\"P_L(like\\mid
Step 3. Calculate study
\"P_L(study\\mid
Step 4. Calculate sleep
\"P_L(sleep\\mid
Final answer. The row is (4/7, 2/7, 1/7). An unseen bigram is no longer forced to zero.

Question. A fixed vocabulary has five possible next tokens. A history h is unseen in training. What is its add-one probability for each next token?

Given

C(h)=0; C(h,w)=0 for each outcome; V=5.

To find

P_L(w|h) for each token.

Formula and why it applies

\"P_L(w\\mid

All outcomes receive equal pseudo-counts because the training data gives no preference for this history.

Solution

Step 1. Add the pseudo-count
\"0+1=1\"
Step 2. Form the denominator
\"0+5=5\"
Step 3. Divide and sum
\"P_L(w\\mid
Final answer. Every outcome gets 0.2 under this fixed-vocabulary model.

Vocabulary warning. Add-one does not automatically define probabilities for every word in the world. Unknown words require a declared vocabulary/UNK convention. If a question supplies V, use that stated V rather than inventing additional tokens.

Yaad rakho: Add-one: upar +1, neeche +V; bigram mein neeche history ka count.

"},{"id":"smoothed-sentence","title":"A complete unseen-bigram question from corpus to final answer","html":"

Question. Consider the corpus “I love to code in Python. I love to learn new things. I code every day.” Obtain the bigram probabilities needed for “I love Python”. Explain the zero-probability issue, then use Laplace smoothing to score it. Treat each sentence separately, preserve case, remove punctuation, and include an end token among predicted outcomes.

Given

Three sentences padded with one start/end marker. Distinct words: I, love, to, code, in, Python, learn, new, things, every, day. With </s>, V=12. Start is context only.

To find

Unsmoothed and smoothed P(I love Python </s> | <s>).

Formula and why it applies

\"P(w\\mid

The test contains an unseen transition love→Python. Smoothing assigns probability to that transition and renormalizes every required history row.

Solution

Step 1. Count the contexts
HistoryCount
<s>3
I3
love2
Python1
Step 2. Count each required pair
PairCount
<s> I3
I love2
love Python0
Python </s>1
Step 3. Compute the unsmoothed product
\"P=\\frac33\\times\\frac23\\times\\frac02\\times\\frac11=0\"

Both words love and Python were seen, but their adjacent pair was not. This is an unseen bigram, not an unknown-word case.

Step 4. Smooth the first event
\"P_L(I\\mid\\langle
Step 5. Smooth the second event
\"P_L(love\\mid
Step 6. Smooth the unseen event
\"P_L(Python\\mid
Step 7. Smooth the end event
\"P_L(\\langle/s\\rangle\\mid
Step 8. Multiply the smoothed factors
\"P_L=\\frac4{15}\\times\\frac3{15}\\times\\frac1{14}\\times\\frac2{13}\"
\"P_L=\\frac{24}{40950}=\\frac4{6825}\\approx0.000586081\"
Final answer. Unsmoothed P=0. With the stated V=12 and boundaries, Laplace-smoothed P≈0.000586081.

If an exam states a different convention—for example excluding boundary tokens from the entire calculation—recompute both V and the set of probability factors consistently. Do not combine a boundary-aware numerator with a boundary-free denominator.

"},{"id":"adjusted-count-discount","title":"Adjusted counts and relative discounting","html":"

The slides also express smoothing through an adjusted count c*. Instead of writing the smoothed probability with denominator N+V, they rescale its numerator so the denominator can remain N. Algebraically these are equivalent:

\"c^*=(c+1)\\frac{N}{N+V},\\qquad
\"d=\\frac{c^*}{c}\\quad(c>0)\"

The relative discount d compares adjusted and original counts. It is undefined when c=0 because division by zero is undefined. Add-one smoothing moves the distribution toward uniformity: frequent events are reduced, but some rare seen events may increase. It is not correct to claim that every nonzero count must decrease under Laplace smoothing.

Question. For N=20 and V=5, a word has original count c=10. Find its adjusted count, smoothed probability and relative discount.

Given

N=20; V=5; c=10.

To find

c*, P_L and d.

Formula and why it applies

\"c^*=(c+1)N/(N+V),\\quad

These expressions rewrite add-one smoothing without changing its probability.

Solution

Step 1. Form the new total
\"N+V=20+5=25\"
Step 2. Find the adjusted count
\"c^*=(10+1)\\frac{20}{25}=11(0.8)=8.8\"
Step 3. Find the probability
\"P_L=8.8/20=0.44\"
Step 4. Find the relative discount
\"d=8.8/10=0.88\"
Final answer. c*=8.8, P_L=0.44, d=0.88. The original probability 0.5 is reduced.

Question. Under N=20, V=5, a rare word occurs once. Compute its adjusted count and compare the original and smoothed probabilities.

Given

N=20; V=5; c=1.

To find

c*, original P and smoothed P_L.

Formula and why it applies

\"c^*=(c+1)N/(N+V)\"

The same smoothing rule may increase a rare event's probability.

Solution

Step 1. Original probability
\"P=1/20=0.05\"
Step 2. Adjusted count
\"c^*=(1+1)(20/25)=1.6\"
Step 3. Smoothed probability
\"P_L=1.6/20=0.08\"
Final answer. c*=1.6 and P_L=0.08, which exceeds the original 0.05.

Question. For N=20 and V=5, an in-vocabulary word has count zero. Find its adjusted count and smoothed probability. Can a relative discount be calculated?

Given

c=0; N=20; V=5.

To find

c*, P_L and whether d exists.

Formula and why it applies

\"c^*=(c+1)N/(N+V),\\qquad

The pseudo-count creates positive mass for the unseen vocabulary item.

Solution

Step 1. Adjusted count
\"c^*=(0+1)(20/25)=0.8\"
Step 2. Smoothed probability
\"P_L=0.8/20=0.04\"
Step 3. Relative discount

d would be 0.8/0. That is undefined, so do not report a finite discount.

Final answer. c*=0.8 and P_L=0.04; relative discount is undefined for c=0.
"},{"id":"evaluation-perplexity","title":"Perplexity: meaning, formula and four complete examples","html":"

Intrinsic evaluation evaluates a model directly on a held-out test corpus, for example using perplexity. Extrinsic evaluation places it inside a downstream application, such as a speech recognizer or translation system, and measures the application's performance. Extrinsic testing is closer to actual use but often slower and more expensive. A lower perplexity is not a guarantee of improvement on every downstream task.

Perplexity is the inverse probability of a test sequence, normalized by the number of prediction events. Intuitively, it is the model's effective average number of equally plausible choices. A model that always gives the correct next word probability 1/4 has perplexity 4. Lower is better when the models are tested on the same data, vocabulary, tokenization and boundary convention.

\"\\operatorname{PP}(W)=P(W)^{-1/N}=\\sqrt[N]{\\frac1{P(W)}}\"
SymbolMeaning
WThe complete test sequence.
P(W)Product of the probabilities assigned to the actual next tokens.
NNumber of scored prediction events under the stated convention.
Exponent −1/NTake the reciprocal, then the Nth root.

For calculator entry, use parentheses: P raised to (−1 ÷ N), or (1 ÷ P) raised to (1 ÷ N). Retain several digits in intermediate products. The ordinary start marker is usually given context, not a predicted event; the end marker counts if its probability is included.

Question. A model assigns probability 0.0625 to a test sequence containing four scored tokens. Compute its perplexity.

Given

P(W)=0.0625=1/16; N=4.

To find

PP(W).

Formula and why it applies

\"\\operatorname{PP}(W)=\\left(\\frac1{P(W)}\\right)^{1/N}\"

The root removes the effect of sequence length; the reciprocal makes lower surprise correspond to lower perplexity.

Solution

Step 1. Take the reciprocal
\"1/P(W)=1/0.0625=16\"
Step 2. Use the fourth root
\"\\operatorname{PP}=16^{1/4}\"
Step 3. Evaluate
\"2^4=16\\quad\\Rightarrow\\quad\\operatorname{PP}=2\"
Final answer. Perplexity = 2.

Question. Two language models score the same three-token test sequence. Model A gives next-token probabilities 0.5, 0.5, 0.5. Model B gives 0.5, 0.25, 0.25. Compute their perplexities and select the better intrinsic result.

Given

Same sequence and N=3 for both models. A factors: 0.5,0.5,0.5. B factors: 0.5,0.25,0.25.

To find

Both sequence probabilities and perplexities.

Formula and why it applies

\"P(W)=\\prod_i

A fair comparison uses identical test tokens and scoring conventions.

Solution

Step 1. Calculate A's probability
\"P_A=0.5\\times0.5\\times0.5=0.125=1/8\"
Step 2. Calculate A's perplexity
\"\\operatorname{PP}_A=8^{1/3}=2\"
Step 3. Calculate B's probability
\"P_B=0.5\\times0.25\\times0.25=0.03125=1/32\"
Step 4. Calculate B's perplexity
\"\\operatorname{PP}_B=32^{1/3}\\approx3.174802\"
Step 5. Compare

2 is lower than 3.174802; A places more probability on the observed sequence.

Final answer. Model A is better by perplexity on this test: 2 versus about 3.174802.

Question. The bigram training corpus is “This is a dog”, “This is a cat”, “I love my cat”, “This is my name”. Score “This is my cat”, including an end marker, and compute perplexity using five predicted events.

Given

Each training sentence has one start and end marker. Test predictions: This, is, my, cat, </s>. Thus N=5.

To find

Bigram sentence probability and perplexity.

Formula and why it applies

\"P(w\\mid

Start is given context. Four words plus the end marker produce five probability factors.

Solution

Step 1. Count the required transitions
TransitionPair count / history count
<s>→This3/4
This→is3/3
is→my1/3
my→cat1/2
cat→</s>2/2
Step 2. Form the product
\"P=\\frac34\\times1\\times\\frac13\\times\\frac12\\times1=\\frac18\"
Step 3. Take the reciprocal
\"1/P=8\"
Step 4. Take the fifth root
\"\\operatorname{PP}=8^{1/5}\\approx1.515717\"
Final answer. P(W)=1/8 and perplexity≈1.515717 using five predicted events.

If the question explicitly states N = 6: use that supplied convention and report PP = 81/6 ≈ 1.414214. The class slide counts both boundary markers in N. Write the chosen N beside the formula; do not silently switch between 5 and 6. Also, This occurs three times and is follows it three times, so P(is|This)=3/3, not 3/4.

Question. A uniform model assigns probability 1/10 to each actual next word in a five-token test. Find its perplexity.

Given

Five scored tokens, each with assigned probability 0.1. N=5.

To find

The sentence probability and perplexity.

Formula and why it applies

\"P(W)=(1/10)^5,\\qquad

Uniform equal probabilities make the effective number of choices especially clear.

Solution

Step 1. Multiply the five factors
\"P(W)=0.1^5=0.00001=10^{-5}\"
Step 2. Invert
\"1/P(W)=10^5\"
Step 3. Take the fifth root
\"\\operatorname{PP}=(10^5)^{1/5}=10\"
Final answer. Perplexity = 10.

Yaad rakho: Probability high → surprise low → perplexity low.

"},{"id":"log-probabilities","title":"Log probabilities: replace long products with sums","html":"

Probabilities are at most 1, so multiplying many of them produces very small numbers. Computers can round extremely small positive products to zero: numerical underflow. Log probabilities convert multiplication into addition and are more stable for long sequences. Natural logs use base e; log₂ uses base 2. Choose one base consistently.

\"\\ln
\"P(W)=e^{\\ln

The log of a probability below 1 is negative. A value closer to zero means a larger probability: ln(0.5) ≈ −0.693 is larger than ln(0.1) ≈ −2.303. A zero probability has log −∞; taking logs does not solve the unseen-event problem. Smoothing handles that separate problem.

Question. The next-word probabilities are 0.5, 0.25 and 0.5. Calculate the natural log probability, ordinary probability and perplexity.

Given

p₁=0.5; p₂=0.25; p₃=0.5; N=3.

To find

ln P(W), P(W), and PP.

Formula and why it applies

\"\\ln

Logarithms preserve products as sums and the exponent converts the result back.

Solution

Step 1. Compute individual logs
\"\\ln0.5\\approx-0.693147,\\quad\\ln0.25\\approx-1.386294\"
Step 2. Add all three terms
\"\\ln
Step 3. Recover the probability
\"P=e^{-2.772589}\\approx0.0625\"
Step 4. Find average negative log probability
\"-\\ln
Step 5. Exponentiate
\"\\operatorname{PP}=e^{0.924196}\\approx2.519842\"
Final answer. ln P≈−2.772589, P=0.0625, PP≈2.519842.

Question. A model has a total natural log probability −12 over six scored tokens. Calculate its perplexity.

Given

ln P(W)=−12; N=6.

To find

PP without forming the tiny ordinary probability.

Formula and why it applies

\"\\operatorname{PP}=\\exp(-\\ln

The logarithmic formula directly uses the supplied log score.

Solution

Step 1. Negate the log score
\"-\\ln
Step 2. Divide by token count
\"12/6=2\"
Step 3. Apply exponential
\"\\operatorname{PP}=e^2\\approx7.389056\"
Final answer. Perplexity≈7.389056.

Question. A language model's average negative log₂ probability is 3 bits per token. Find its perplexity.

Given

Cross-entropy H=3 bits/token. Log base is 2.

To find

PP.

Formula and why it applies

\"\\operatorname{PP}=2^H\"

The inverse operation of log₂ is raising 2 to that power; use e only for natural logs.

Solution

Step 1. Identify the log base

Bits mean base 2.

Step 2. Raise the base to H
\"\\operatorname{PP}=2^3=8\"
Final answer. Perplexity = 8.

Yaad rakho: ln use kiya toh e se wapas; log₂ use kiya toh 2 se wapas.

"}],"scope":"Formulas explained through full worked examples"},{"id":"f-hmm","title":"HMM & decoding","unit":0,"sections":[{"id":"markov-chain","title":"Markov chains: read a transition table and multiply a path","html":"

A Markov chain is a probabilistic model of a sequence of states. Under the first-order Markov assumption, the next state's probability depends on the current state rather than the complete previous history. The initial probability tells us how likely each first state is. A transition probability tells us how likely a move from one state to another is.

\"P(s_t\\mid
\"P(s_1,\\ldots,s_T)=P(s_1)\\prod_{t=2}^{T}P(s_t\\mid

For T states in a path, use one initial factor and T−1 transition factors. A row-oriented transition matrix must have each complete row sum to 1. Always read the row/column labels; some exam diagrams use a transposed arrangement.

Current → nextRainDry
Rain0.30.7
Dry0.20.8

Initial probabilities: P(Rain)=0.4, P(Dry)=0.6. Rows describe the current state; columns describe the next state.

Question. For the Rain/Dry Markov chain with initial P(Rain)=0.4 and P(Dry)=0.6, and transitions Rain→Rain=0.3, Rain→Dry=0.7, Dry→Rain=0.2, Dry→Dry=0.8, find the probability of Dry, Dry, Rain, Rain.

Given

Current → nextRainDry
Rain0.30.7
Dry0.20.8

Initial probabilities: P(Rain)=0.4, P(Dry)=0.6. Rows describe the current state; columns describe the next state.

To find

P(Dry,Dry,Rain,Rain).

Formula and why it applies

\"P(D,D,R,R)=P(D)P(D\\mid

Each step conditions on the immediately preceding state. The sequence contains four states, so it needs one initial plus three transition factors.

Solution

Step 1. Start in Dry
\"P(D)=0.6\"
Step 2. Move Dry to Dry
\"0.6\\times0.8=0.48\"
Step 3. Move Dry to Rain
\"0.48\\times0.2=0.096\"
Step 4. Move Rain to Rain
\"0.096\\times0.3=0.0288\"
Final answer. Path probability = 0.0288.

Question. For initial probabilities Rain=0.4, Dry=0.6 and transition rows Rain=(0.3,0.7), Dry=(0.2,0.8) in column order Rain,Dry, find the probability of Rain, Dry, Rain.

Given

Current → nextRainDry
Rain0.30.7
Dry0.20.8

Initial probabilities: P(Rain)=0.4, P(Dry)=0.6. Rows describe the current state; columns describe the next state.

To find

P(Rain,Dry,Rain).

Formula and why it applies

\"P(R,D,R)=P(R)P(D\\mid

Do not replace Dry→Rain with Rain→Dry: the two directional probabilities differ.

Solution

Step 1. Initial Rain
\"P(R)=0.4\"
Step 2. First transition
\"P(D\\mid
Step 3. Second transition
\"P(R\\mid
Final answer. Path probability = 0.056.

Question. For the same fully specified chain—initial Rain=0.4, Dry=0.6; Rain→Rain=0.3, Rain→Dry=0.7, Dry→Rain=0.2, Dry→Dry=0.8—find P(Dry,Dry). Also find the conditional probability of next Dry given current Dry.

Given

Current → nextRainDry
Rain0.30.7
Dry0.20.8

Initial probabilities: P(Rain)=0.4, P(Dry)=0.6. Rows describe the current state; columns describe the next state.

To find

The two-state joint probability and the one-step conditional probability.

Formula and why it applies

\"P(D,D)=P(D)P(D\\mid

The joint path includes an uncertain initial state. The conditional transition assumes the current state is already Dry.

Solution

Step 1. Read the conditional transition
\"P(D\\mid
Step 2. Include the initial state for the joint path
\"P(D,D)=0.6\\times0.8=0.48\"
Final answer. P(Dry,Dry)=0.48; P(next Dry|current Dry)=0.8. These answer different questions.

Yaad rakho: Path: pehle start probability, phir har arrow ki probability.

"},{"id":"transition-emission-estimation","title":"Estimate transition and emission probabilities from tagged data","html":"
\"P(t'\\mid
\"P(w\\mid

In a transition denominator, count the previous tag's outgoing events. In an emission denominator, count all tokens bearing that tag. With explicit end transitions the totals align conveniently. Keep the two numerator types separate: tag→tag counts are not word→tag counts.

Question. A tagged corpus contains 116454 determiner tags DT, each counted with an outgoing event. DT is followed by NN 56509 times. Estimate P(NN|DT).

Given

C(DT)=116454; C(DT,NN)=56509.

To find

The DT→NN transition probability.

Formula and why it applies

\"P(NN\\mid

The denominator is the number of previous DT contexts because DT is the condition.

Solution

Step 1. Identify the transition count

DT followed by NN occurs 56509 times.

Step 2. Identify the conditional total

There are 116454 DT contexts.

Step 3. Divide
\"P(NN\\mid
Final answer. P(NN|DT)≈0.485247.

Question. The tag VBZ occurs 21627 times, and 10073 of those tokens are the word “is”. Estimate P(is|VBZ).

Given

C(VBZ)=21627; C(is,VBZ)=10073.

To find

The emission probability of is from VBZ.

Formula and why it applies

\"P(is\\mid

The condition is the tag VBZ; we ask which word it emits.

Solution

Step 1. Choose the word-tag count

The required word is is with tag VBZ: count 10073.

Step 2. Choose the tag total

All VBZ-tagged word tokens: count 21627.

Step 3. Divide
\"P(is\\mid
Final answer. P(is|VBZ)≈0.465760. This is different from P(VBZ|is).

Question. Use the tagged sentences “dogs/N chase/V cats/N” and “cats/N sleep/V”. Estimate the initial tag probabilities, all nonzero transitions including END, and emissions.

Given

Corpus: START dogs/N chase/V cats/N END; START cats/N sleep/V END.

To find

Initial distribution, transition rows and emission rows.

Formula and why it applies

\"\\pi_t=C(START,t)/2,\\quad

The corpus explicitly tells us the tags, so we can count training frequencies directly.

Solution

Step 1. Count tag tokens

N occurs 3 times: dogs, cats, cats. V occurs 2 times: chase, sleep.

Step 2. Count sentence starts

Both sentences start with N.

\"\\pi_N=2/2=1,\\quad
Step 3. Count transitions
PairCount
N→V2
N→END1
V→N1
V→END1
Step 4. Normalize transition rows
CurrentTo NTo VTo END
N02/31/3
V1/201/2
Step 5. Count and normalize emissions
Tagdogscatschasesleep
N1/32/300
V001/21/2
Final answer. π=(1,0); transitions and emissions are given in the completed tables. Each complete probability row sums to 1.

Yaad rakho: Tag→tag table aur tag→word table ko mix mat karna.

"},{"id":"hot-cold-paths","title":"HMM path probabilities: three complete numerical examples","html":"

The following model uses hidden weather states Hot (H) and Cold (C). Observations are numbers 1, 2 and 3. These numbers are observation labels, not quantities to multiply directly. Their emission probabilities come from the table.

Initial stateProbability
H0.8
C0.2
From → toHC
H0.70.3
C0.40.6
Emission stateObservation 1Observation 2Observation 3
H0.20.40.4
C0.50.40.1

Question. An HMM has π(H)=0.8, π(C)=0.2; H→H=0.7, H→C=0.3, C→H=0.4, C→C=0.6. Emissions for observations (1,2,3) are H:(0.2,0.4,0.4) and C:(0.5,0.4,0.1). Find the joint probability of hidden path H,C,H and observations 1,3,1.

Given

Hidden path H→C→H. Observations 1→3→1. Model probabilities are specified in the question. There is no END factor.

To find

P(H,C,H; 1,3,1).

Formula and why it applies

\"P=\\pi_H

There are three hidden states and observations: three emissions, two transitions, and one initial probability.

Solution

Step 1. Start Hot and emit 1
\"0.8\\times0.2=0.16\"
Step 2. Transition Hot to Cold
\"0.16\\times0.3=0.048\"
Step 3. Cold emits 3
\"0.048\\times0.1=0.0048\"
Step 4. Transition Cold to Hot
\"0.0048\\times0.4=0.00192\"
Step 5. Hot emits the last 1
\"0.00192\\times0.2=0.000384\"
Final answer. Joint probability = 0.000384 = 3.84×10⁻⁴.

Question. For π(H)=0.8, π(C)=0.2; H→H=0.7, H→C=0.3, C→H=0.4, C→C=0.6; emissions H:(0.2,0.4,0.4), C:(0.5,0.4,0.1) for labels (1,2,3), find the joint probability of path H,H,H and observations 1,3,1.

Given

State path H→H→H, observations 1→3→1; no END factor.

To find

P(H,H,H;1,3,1).

Formula and why it applies

\"P=\\pi_H

The state remains Hot; each transition still contributes its probability 0.7.

Solution

Step 1. Initial state and first emission
\"0.8(0.2)=0.16\"
Step 2. First self-transition
\"0.16(0.7)=0.112\"
Step 3. Emit observation 3
\"0.112(0.4)=0.0448\"
Step 4. Second self-transition
\"0.0448(0.7)=0.03136\"
Step 5. Emit final observation 1
\"0.03136(0.2)=0.006272\"
Final answer. Joint probability = 0.006272. A self-transition is not automatically probability 1.

Question. For π(H)=0.8, π(C)=0.2; transitions H:(0.7,0.3), C:(0.4,0.6) in next-state order H,C; emissions H:(0.2,0.4,0.4), C:(0.5,0.4,0.1) for observations 1,2,3, find the score of path C,C,H with observations 1,3,1.

Given

Hidden path C→C→H. Observation sequence 1→3→1. No end-state factor.

To find

The joint path-and-observation probability.

Formula and why it applies

\"P=\\pi_C

The initial state is Cold, so use π(C), not π(H). Each emission is selected from the state at the same position.

Solution

Step 1. Start Cold and emit 1
\"0.2(0.5)=0.1\"
Step 2. Stay Cold
\"0.1(0.6)=0.06\"
Step 3. Emit 3 from Cold
\"0.06(0.1)=0.006\"
Step 4. Move Cold to Hot
\"0.006(0.4)=0.0024\"
Step 5. Emit 1 from Hot
\"0.0024(0.2)=0.00048\"
Final answer. Joint probability = 0.00048.

Yaad rakho: Har column par: state ka emission. Columns ke beech: transition.

"},{"id":"tagging-corpus-case","title":"POS HMM worked corpus: Mary, Jane, can, will and spot","html":"

The class example contains words that can be either names or ordinary verbs/modal auxiliaries. The term is modal (M), not “model”. To avoid guessing tags from spelling, the training corpus below supplies them explicitly and lowercases words for the emission vocabulary.

Training sentenceGold tags
mary jane can see willN N M V N
spot will see maryN M V N
will jane spot maryM N V N
mary will pat spotN M V N

There are 9 noun tokens, 4 modal tokens and 4 verb tokens. Each sentence includes a start and end transition. For nouns, the next event can be another noun, modal, verb or END.

Current → nextNMVEND
START3/41/400
N1/93/91/94/9
M1/403/40
V4/4000
Tag → wordmaryjanecanseewillspotpat
N4/92/9001/92/90
M001/403/400
V0002/401/41/4

Question. Using the fully tagged four-sentence corpus above, calculate the joint score of “will jane spot mary” assigned tags M,N,V,N, including END.

Given

Required factors from the tables: START→M=1/4; emit will|M=3/4; M→N=1/4; jane|N=2/9; N→V=1/9; spot|V=1/4; V→N=1; mary|N=4/9; N→END=4/9.

To find

The specified tag-path score, including words and sentence end.

Formula and why it applies

\"P=\\pi_M

Both transitions and emissions are needed. A tag transition alone cannot tell us whether a particular word is plausible.

Solution

Step 1. Start and emit will
\"\\frac14\\times\\frac34=\\frac3{16}\"
Step 2. Move to N and emit jane
\"\\frac3{16}\\times\\frac14\\times\\frac29=\\frac1{96}\"
Step 3. Move to V and emit spot
\"\\frac1{96}\\times\\frac19\\times\\frac14=\\frac1{3456}\"
Step 4. Move to N and emit mary
\"\\frac1{3456}\\times1\\times\\frac49=\\frac1{7776}\"
Step 5. End the sentence
\"\\frac1{7776}\\times\\frac49=\\frac1{17496}\\approx0.0000571559\"
Final answer. The joint score is 1/17496 ≈ 0.0000571559.

Question. For the same tagged corpus and tables, explain the unsmoothed score of “will can spot mary” assigned tags M,V,N,N.

Given

The proposed second word is can with hidden tag V. In the training corpus, can occurs only with M, so b_V(can)=0.

To find

Whether the complete path score is zero and what this means.

Formula and why it applies

\"P(\\text{path,words})=(\\text{other

A product containing one zero factor is zero even if every other factor is positive.

Solution

Step 1. Find the critical emission
\"b_V(can)=C(can,V)/C(V)=0/4=0\"
Step 2. Use the full-product consequence
\"P(\\text{path,words})=0\"
Step 3. Interpret correctly

This unsmoothed corpus model has no evidence for can as a verb. It does not prove that a verb usage of can is impossible in English.

Final answer. The unsmoothed path score is zero.
"},{"id":"sequence-count","title":"Why exhaustive search grows exponentially","html":"

If each of T observed words can receive any of N tags, there are N choices at each position. Thus the total number of complete tag sequences is N multiplied by itself T times. This counts possible assignments before grammar rules or zero probabilities eliminate any.

\"\\text{Number

Question. A four-word sentence is tagged using three possible POS tags per word. How many complete tag sequences are possible?

Given

N=3 possible tags per position; T=4 word positions.

To find

The number of tag sequences.

Formula and why it applies

\"M=N^T\"

Every independent position contributes a factor of three choices.

Solution

Step 1. Write one factor per position
\"M=3\\times3\\times3\\times3\"
Step 2. Multiply
\"3\\times3=9,\\quad9\\times3=27,\\quad27\\times3=81\"
Final answer. 81 possible tag sequences.

Question. A five-word sentence has two candidate tags per word. Count its tag sequences.

Given

N=2; T=5.

To find

M.

Formula and why it applies

\"M=N^T\"

The length is the exponent, not a multiplier.

Solution

Step 1. Substitute
\"M=2^5\"
Step 2. Evaluate
\"2\\times2\\times2\\times2\\times2=32\"
Final answer. 32 possible sequences.

Question. Using 45 possible tags, compare the number of tag sequences for sentences of length 2 and 3.

Given

N=45. First T=2, then T=3.

To find

Both counts and the growth factor.

Formula and why it applies

\"M_2=45^2,\\quad

Adding one position multiplies the possibilities by the number of tag choices.

Solution

Step 1. Two words
\"45^2=2025\"
Step 2. Three words
\"45^3=2025\\times45=91125\"
Step 3. Compare
\"91125/2025=45\"
Final answer. 2 words: 2025 paths. 3 words: 91125 paths. One additional word multiplies the count by 45.

The slides introduce Viterbi to avoid enumerating every path. It reuses the best prefix score for each ending state. For a dense first-order HMM, it needs O(TN²) recurrence work instead of enumerating NT complete paths.

Yaad rakho: Choices per word ki power = sentence length.

"},{"id":"viterbi-explained","title":"Viterbi decoding: compare incoming paths, keep the best, trace back","html":"

The Viterbi algorithm finds the most likely hidden-state sequence for a fixed observation sequence. At each position, it keeps the best path ending in each state. This is safe because, under the HMM assumptions, future extensions depend on that ending state, not the discarded details of the prefix.

\"\\delta_1(j)=\\pi_jb_j(o_1)\"
\"\\delta_t(j)=b_j(o_t)\\max_i[\\delta_{t-1}(i)a_{ij}]\"
\"\\psi_t(j)=\\arg\\max_i[\\delta_{t-1}(i)a_{ij}]\"
SymbolMeaning in ordinary words
δₜ(j)Score of the best prefix that ends in state j at position t.
ψₜ(j)Previous state that supplied that best prefix; keep it as a backpointer.
maxChoose the larger candidate score; do not add.
argmaxRemember which previous state won, rather than only its value.
Initialize: start × first emission
→
For each next state: compare incoming scores
→
Multiply winner by this state's emission
→
Keep score + winning predecessor
→
Choose best final state and trace arrows backward

Do not choose the best state independently at each word. The best last state can come from a different best predecessor than a local guess. Backpointers recover one consistent complete path. If scores tie, either tied path is optimal under the model; state the tie.

Question. Use an HMM with π(H)=0.8, π(C)=0.2, transitions H:(0.7,0.3), C:(0.4,0.6) in H,C order, and emissions H:(0.2,0.4,0.4), C:(0.5,0.4,0.1) for observations 1,2,3. Find the most likely hidden path for observations 1,3,1 using Viterbi.

Given

Initial probabilities: π(H)=0.8, π(C)=0.2. Transition matrix: H→H=0.7, H→C=0.3, C→H=0.4, C→C=0.6. Emissions for labels (1,2,3): H=(0.2,0.4,0.4), C=(0.5,0.4,0.1). There is no END-state factor.

To find

The best hidden-state sequence and its joint score.

Formula and why it applies

\"\\delta_1(j)=\\pi_jb_j(o_1),\\quad\\delta_t(j)=b_j(o_t)\\max_i[\\delta_{t-1}(i)a_{ij}]\"

Viterbi keeps two best prefix scores per column, one ending H and one ending C.

Solution

Step 1. Initialize at observation 1
\"\\delta_1(H)=0.8(0.2)=0.16\"
\"\\delta_1(C)=0.2(0.5)=0.10\"
Step 2. At observation 3, calculate the H cell
\"H\\to
\"C\\to
\"\\delta_2(H)=\\max(0.112,0.040)(0.4)=0.0448\"

Backpointer for this H cell: H.

Step 3. At observation 3, calculate the C cell
\"H\\to
\"C\\to
\"\\delta_2(C)=\\max(0.048,0.060)(0.1)=0.006\"

Backpointer for this C cell: C.

Step 4. At final observation 1, calculate H
\"H\\to
\"C\\to
\"\\delta_3(H)=0.03136(0.2)=0.006272\"

Backpointer: H.

Step 5. At final observation 1, calculate C
\"H\\to
\"C\\to
\"\\delta_3(C)=0.01344(0.5)=0.00672\"

Backpointer: H.

Step 6. Select the final state

0.00672 > 0.006272, so the best final state is C.

Step 7. Trace back

Final C came from H at position2. That H came from H at position1. Reading forward gives H,H,C.

Step 8. Verify the recovered complete path
\"0.8(0.2)(0.7)(0.4)(0.3)(0.5)=0.00672\"
Final answer. Best path H→H→C with joint score 0.00672.

Question. For π(H)=0.8, π(C)=0.2; transitions H:(0.7,0.3), C:(0.4,0.6); emissions H:(0.2,0.4,0.4), C:(0.5,0.4,0.1) for labels1,2,3, decode observations 3,1.

Given

Initial probabilities: π(H)=0.8, π(C)=0.2. Transition matrix: H→H=0.7, H→C=0.3, C→H=0.4, C→C=0.6. Emissions for labels (1,2,3): H=(0.2,0.4,0.4), C=(0.5,0.4,0.1). There is no END-state factor.

To find

Best two-state hidden path and score.

Formula and why it applies

\"\\delta_1(j)=\\pi_jb_j(3),\\quad\\delta_2(j)=b_j(1)\\max_i[\\delta_1(i)a_{ij}]\"

The same recurrence applies even though the observation order changed.

Solution

Step 1. Initialize observation 3
\"\\delta_1(H)=0.8(0.4)=0.32,\\quad\\delta_1(C)=0.2(0.1)=0.02\"
Step 2. Calculate the final H
\"\\max(0.32(0.7),0.02(0.4))(0.2)\"
\"=\\max(0.224,0.008)(0.2)=0.0448\"

Winning predecessor H.

Step 3. Calculate the final C
\"\\max(0.32(0.3),0.02(0.6))(0.5)\"
\"=\\max(0.096,0.012)(0.5)=0.048\"

Winning predecessor H.

Step 4. Select and trace

Final C wins with 0.048 and came from H. The path is H,C.

Final answer. Best path H→C, score 0.048.

Question. For π(H)=0.8, π(C)=0.2; transitions H:(0.7,0.3), C:(0.4,0.6); emissions H:(0.2,0.4,0.4), C:(0.5,0.4,0.1) for labels1,2,3, decode observations 2,2.

Given

Initial probabilities: π(H)=0.8, π(C)=0.2. Transition matrix: H→H=0.7, H→C=0.3, C→H=0.4, C→C=0.6. Emissions for labels (1,2,3): H=(0.2,0.4,0.4), C=(0.5,0.4,0.1). There is no END-state factor.

To find

Best path and score.

Formula and why it applies

\"\\delta_1(j)=\\pi_jb_j(2),\\quad\\delta_2(j)=b_j(2)\\max_i[\\delta_1(i)a_{ij}]\"

Both states emit2 with probability 0.4, but their initial and transition probabilities still differ.

Solution

Step 1. Initialize
\"\\delta_1(H)=0.8(0.4)=0.32,\\quad\\delta_1(C)=0.2(0.4)=0.08\"
Step 2. Final H
\"\\max(0.32(0.7),0.08(0.4))(0.4)\"
\"=\\max(0.224,0.032)(0.4)=0.0896\"

Predecessor H.

Step 3. Final C
\"\\max(0.32(0.3),0.08(0.6))(0.4)\"
\"=\\max(0.096,0.048)(0.4)=0.0384\"

Predecessor H.

Step 4. Select and trace

The final H score 0.0896 is larger. Its predecessor is H.

Final answer. Best path H→H with score 0.0896.

Yaad rakho: Har destination ke liye incoming paths compare; winner ka arrow yaad.

"},{"id":"joint-vs-posterior","title":"A path score is not the probability of the observed sentence","html":"

Question. An HMM has π(H)=0.8, π(C)=0.2; transitions H:(0.7,0.3), C:(0.4,0.6); emissions H:(0.2,0.4,0.4), C:(0.5,0.4,0.1) for labels1,2,3. For observations 1,3, calculate the probability of the observations by summing all four two-state paths.

Given

Initial probabilities: π(H)=0.8, π(C)=0.2. Transition matrix: H→H=0.7, H→C=0.3, C→H=0.4, C→C=0.6. Emissions for labels (1,2,3): H=(0.2,0.4,0.4), C=(0.5,0.4,0.1). There is no END-state factor.

To find

P(observations 1,3), not just the best-path score.

Formula and why it applies

\"P(o_1,o_2)=\\sum_{s_1,s_2}\\pi_{s_1}b_{s_1}(o_1)a_{s_1s_2}b_{s_2}(o_2)\"

The hidden path is unspecified, so its mutually exclusive alternatives must be summed.

Solution

Step 1. Score H,H
\"0.8(0.2)(0.7)(0.4)=0.0448\"
Step 2. Score H,C
\"0.8(0.2)(0.3)(0.1)=0.0048\"
Step 3. Score C,H
\"0.2(0.5)(0.4)(0.4)=0.016\"
Step 4. Score C,C
\"0.2(0.5)(0.6)(0.1)=0.006\"
Step 5. Add all paths
\"P(1,3)=0.0448+0.0048+0.016+0.006=0.0716\"
Final answer. Observation probability =0.0716. The best single-path score 0.0448 is a different quantity.

If asked for the posterior of path H,H given observations 1,3, normalize: 0.0448/0.0716 ≈0.625698. Viterbi chooses the best path using joint scores because the observation denominator is common to every candidate. It does not compute that denominator by taking a maximum.

"}],"scope":"Formulas explained through full worked examples"},{"id":"f-wsd","title":"WSD: Lesk & Naive Bayes","unit":0,"sections":[{"id":"u5-lesk","title":"Lesk algorithm, from words to overlap scores","html":"

Lesk is a knowledge-based WSD method that chooses senses whose dictionary descriptions have the greatest overlap with contextual evidence. Original Lesk compares definitions of neighbouring words' senses. A common simplified version compares each target-sense gloss directly with the sentence context. State which version you use.

Simplified Lesk procedure

  1. Remove the target word from the context; tokenize and normalize the remaining content words.
  2. Obtain the gloss for each candidate sense and apply the same normalization.
  3. Count shared words between the context and each gloss.
  4. Select the sense with maximum overlap. If scores tie, report the tie or use a specified tie-break rule.
\"\\operatorname{score}(s)=|C\\cap

\"C\" is the set of context words, \"G_s\" the set of words in sense \"s\"'s gloss, \"\\cap\" means common elements, \"|\\cdot|\" means count, and \"\\arg\\max\" means “the sense giving the largest score”. This version counts unique matching words once.

Question. Use simplified Lesk to disambiguate “bank” in “The river water rose near the bank.” Use C = {river, water, rose, near}; financial gloss = {institution, money, deposit}; river gloss = {land, river, water, edge}.

Given

The normalized context and two gloss sets are given. Count exact shared words.

To find

The score of each sense and the selected meaning.

Formula and why it applies

Use \"\\operatorname{score}(s)=|C\\cap because the task specifies context–gloss overlap.

Step 1. Compare the financial sense

No word occurs in both {river, water, rose, near} and {institution, money, deposit}. Overlap = {}.

Step 2. Count the financial score

\"\\operatorname{score}(F)=0\".

Step 3. Compare the river sense

Shared words are {river, water}. “Rose” and “near” are not in the gloss.

Step 4. Count and select

\"\\operatorname{score}(R)=2\". Since \"2>0\", choose the river-edge sense.

Answer. Bank means land beside a river.

Question. Use simplified Lesk for “The dog barked loudly.” Treat barked as bark. Context C = {dog, loudly}; sound gloss = {sharp, cry, dog}; tree gloss = {outer, covering, tree}; ship gloss = {sailing, ship}.

Given

Three glosses and normalized context {dog, loudly}.

To find

Select the sense of bark.

Formula and why it applies

\"\\operatorname{score}(s)=|C\\cap. Lemmatization lets barked match the target lemma bark; it does not create additional overlap words.

Step 1. Sound sense

Intersection = {dog}; score = 1.

Step 2. Tree sense

Intersection = {}; score = 0.

Step 3. Ship sense

Intersection = {}; score = 0.

Step 4. Compare

The sound sense has the unique maximum: 1 > 0.

Answer. Bark is the dog's cry.

Question. Apply definition–definition Lesk to “pine cone”. Pine senses: P1 = {evergreen, tree, needle, leaf}; P2 = {waste, sorrow, illness}. Cone senses: C1 = {solid, shape, circular}; C2 = {fruit, evergreen, tree}.

Given

Two candidate senses for each word. Tokens are already normalized; tree and trees would be reduced to the same form if necessary.

To find

The most compatible pair of senses.

Formula and why it applies

Use \"|G_{P_i}\\cap for every pair because this is original-style gloss–gloss comparison.

Step 1. P1 with C1

No shared word → 0.

Step 2. P1 with C2

Shared {evergreen, tree} → 2.

Step 3. P2 with each cone sense

P2–C1 has no overlap → 0; P2–C2 has no overlap → 0.

Step 4. Choose the best pair

The four scores are 0, 2, 0, 0. Maximum 2 belongs to P1–C2.

Answer. Pine is the evergreen tree; cone is its fruit-like reproductive structure.

Limitations: exact overlap misses synonyms, short glosses may produce zeros, and different preprocessing choices change the score. Extended Lesk may include related synset definitions or longer phrase overlaps; do not silently substitute it for a simple question.

Lesk: context aur dictionary mein common words gino; sabse bada overlap jeeta.

"},{"id":"u5-walker","title":"Thesaurus / Walker approach, with three complete examples","html":"

The Walker approach uses thesaurus categories as semantic evidence. Associate each candidate sense with a category. Count or weight context words that support that category. Choose the largest total.

\"\\operatorname{score}(s)=\\sum_{w\\in

\"I=1\" if the supplied category mapping says the context word supports the sense, otherwise \"I=0\". The summation symbol means add all those contributions. Weighted questions may replace 1 with a supplied weight. Category membership must be supplied or justified; it is not guessed from word frequency.

Question. Disambiguate bank using Walker scoring. Context words are money, interest, annum, fetch. The finance category supports money, interest and annum; the location category supports none of these words.

Given

Two categories: Finance and Location. Each supported word contributes 1.

To find

The two totals and selected bank sense.

Formula and why it applies

Sum the category indicators for each sense.

Step 1. Make a contribution table

WordFinanceLocation
money10
interest10
annum10
fetch00

Step 2. Add Finance

\"1+1+1+0=3\".

Step 3. Add Location

\"0+0+0+0=0\".

Step 4. Select

3 > 0, so select the financial institution.

Answer. Finance, with score 3 versus 0.

Question. Use Walker scoring for “The fisherman caught bass beside the lake.” Context {fisherman,caught,lake}. The Fish category contains all three clues; the Music category contains none.

Given

Fish and Music senses, binary category membership given.

To find

The correct bass sense.

Formula and why it applies

Score = sum of context-category matches.

Step 1. Record Fish matches

fisherman → 1; caught → 1; lake → 1.

Step 2. Add Fish score

\"1+1+1=3\".

Step 3. Add Music score

\"0+0+0=0\".

Step 4. Choose

Fish has the larger score.

Answer. Bass refers to a fish.

Question. Score “The player used a bat in the match.” Context {player,match}. Sport supports both words with weights 2 and 1; Animal supports neither. Use the supplied weighted Walker scheme.

Given

Sport weights {player:2, match:1}; Animal weights {player:0, match:0}.

To find

Weighted sense scores.

Formula and why it applies

Use the sum of supplied weights, not an unweighted count, because the question explicitly gives weights.

Step 1. Sport contribution of player

Add 2.

Step 2. Sport contribution of match

Add 1. Sport total = 2 + 1 = 3.

Step 3. Animal total

0 + 0 = 0.

Step 4. Compare

3 > 0 selects the sporting implement.

Answer. Bat is the sports implement.

Walker: dictionary ke exact words nahi, context kis topic/category ko support karta hai woh count.

"},{"id":"u5-bayes-steps","title":"Multinomial Naive Bayes: the exact solving method","html":"

Naive Bayes selects the sense with the largest prior × likelihood product. The context evidence denominator is identical across candidate senses, so it can be omitted for comparison.

\"\\operatorname{score}(s)=P(s)\\prod_j

For add-one multinomial smoothing:

\"P(w\\mid

\"C(w,s)\" counts occurrences of word \"w\" in training contexts of sense \"s\"; \"N_s\" is the total number of word occurrences in those contexts; \"V\" is the shared vocabulary size. It is not the number of training sentences. Repeated query words contribute repeated factors.

Always keep these counts separate

Class priors use number of examples. Word likelihoods in this multinomial model use number of word occurrences. The vocabulary counts unique words across all classes.

  1. List the class examples and calculate priors.
  2. Write one shared vocabulary and count its size.
  3. Count total word occurrences in each class.
  4. Count each query word in each class.
  5. Apply the given smoothing rule.
  6. Multiply each prior by all its query likelihoods.
  7. Compare scores; divide each score by their sum only if a normalized posterior is requested.
\"P(s\\mid

Why smoothing is needed: an unseen clue gives \"0/N_s=0\"; one zero makes the whole product zero. Add-one smoothing redistributes probability across every vocabulary word and prevents these zero factors. Use it only when requested, taught for the task, or state the assumption.

Different model warning: a Bernoulli Naive Bayes question counts presence/absence per document and may include absence factors. Do not use its denominator in a multinomial word-count question.

Pehle sentences se prior; phir words se likelihood; dono counts mix mat kar.

"},{"id":"u5-bayes-mam","title":"Mam's bank numerical—every count and every multiplication","html":"

Question. Classify the sense of “bank” using the training table below. The sentence is: “As the stream flowed gently past the shore, we set up camp by the bank of the river, discussing the possibility of borrowing a loan to buy a boat for future trips.” Use only the highlighted feature words stream, shore and loan, and add-one multinomial smoothing.

Given

Training rowContext wordsSense
1money, deposit, loanFinance
2river, water, shoreRiverbank
3account, savings, investmentFinance
4boat, river, streamRiverbank
5branch, deposit, checkFinance
6fishing, shore, riverRiverbank

Query feature set: {stream, shore, loan}. Other sentence words are not included as features in this supplied example.

To find

The most probable bank sense and normalized posteriors.

Formula and why it applies

Naive Bayes selects the sense with the largest prior × likelihood product. The context evidence denominator is identical across candidate senses, so it can be omitted for comparison.

\"\\operatorname{score}(s)=P(s)\\prod_j

For add-one multinomial smoothing:

\"P(w\\mid

\"C(w,s)\" counts occurrences of word \"w\" in training contexts of sense \"s\"; \"N_s\" is the total number of word occurrences in those contexts; \"V\" is the shared vocabulary size. It is not the number of training sentences. Repeated query words contribute repeated factors.

Step 1. Calculate priors

Finance appears in 3 of the 6 rows; Riverbank appears in 3 of 6.

\"P(F)=\\frac36=\\frac12,\\qquad

Step 2. Construct the vocabulary

{money, deposit, loan, river, water, shore, account, savings, investment, boat, stream, fishing, branch, check}. Thus V=14. Repeated deposit or river is counted once in vocabulary.

Step 3. Count word occurrences per class

There are 3 finance rows × 3 words = 9 word occurrences. There are 3 river rows × 3 words = 9. Thus \"N_F=N_R=9\", and each smoothed denominator is \"9+14=23\".

Step 4. Count the three query words

WordFinance countRiverbank count
stream01
shore02
loan10

Step 5. Calculate finance likelihoods

\"P(\\text{stream}\\mid
\"P(\\text{shore}\\mid
\"P(\\text{loan}\\mid

Step 6. Multiply the finance score

\"\\operatorname{score}(F)=\\frac12\\times\\frac1{23}\\times\\frac1{23}\\times\\frac2{23}=\\frac1{23^3}=\\frac1{12167}\\approx0.00008219.\"

Step 7. Calculate river likelihoods

\"P(\\text{stream}\\mid
\"P(\\text{shore}\\mid
\"P(\\text{loan}\\mid

Step 8. Multiply the river score

\"\\operatorname{score}(R)=\\frac12\\times\\frac2{23}\\times\\frac3{23}\\times\\frac1{23}=\\frac3{12167}\\approx0.00024657.\"

Step 9. Compare scores

3/12167 is larger than 1/12167. Riverbank wins even though “loan” is financial evidence: the combined clues matter.

Step 10. Normalize if a posterior is requested

\"P(F\\mid
\"P(R\\mid

Answer. Bank is the riverbank sense. Scores are 1/12167 and 3/12167; normalized model posteriors are 0.25 and 0.75.

Without smoothing: stream and shore are absent in Finance; loan is absent in Riverbank. Both unsmoothed scores become zero, so there is no valid winning class from those products. Also, unnormalized scores should not be labelled as final posterior probabilities.

Zero word? Add-one: count+1 upar, total words+vocabulary neeche. Last mein score bada wala sense.

"},{"id":"u5-bayes-practice","title":"More WSD numericals: given likelihoods, unequal priors and repeated clues","html":"

Question. For bass, let P(Fish)=0.6 and P(Music)=0.4. Given Fish likelihoods P(caught|Fish)=0.5, P(lake|Fish)=0.4 and Music likelihoods P(caught|Music)=0.05, P(lake|Music)=0.1, classify context {caught,lake} and normalize.

Given

Priors and likelihoods are provided directly; no smoothing or training-count estimation is requested.

To find

The winning sense and its posterior.

Formula and why it applies

\"\\operatorname{score}(s)=P(s)P(\\text{caught}\\mid, because the two features are treated as conditionally independent.

Step 1. Fish multiplication

\"0.6\\times0.5=0.3\", then \"0.3\\times0.4=0.12\".

Step 2. Music multiplication

\"0.4\\times0.05=0.02\", then \"0.02\\times0.1=0.002\".

Step 3. Total evidence

\"0.12+0.002=0.122\".

Step 4. Normalize

\"P(Fish\\mid
\"P(Music\\mid

Answer. Fish, with model posterior approximately 0.983607.

Question. Disambiguate bank from the repeated context tokens {loan, loan, river}. Priors are Finance=2/3 and River=1/3. Vocabulary V=4. Finance has N=8 total tokens, loan count 3 and river count 0; River has N=4, loan count 0 and river count 2. Apply add-one smoothing.

Given

Multinomial counts are supplied. The two appearances of loan must both contribute.

To find

Scores and normalized Finance posterior.

Formula and why it applies

Naive Bayes selects the sense with the largest prior × likelihood product. The context evidence denominator is identical across candidate senses, so it can be omitted for comparison.

\"\\operatorname{score}(s)=P(s)\\prod_j

For add-one multinomial smoothing:

\"P(w\\mid

\"C(w,s)\" counts occurrences of word \"w\" in training contexts of sense \"s\"; \"N_s\" is the total number of word occurrences in those contexts; \"V\" is the shared vocabulary size. It is not the number of training sentences. Repeated query words contribute repeated factors.

Step 1. Finance denominator

\"8+4=12\". \"P(loan\\mid; \"P(river\\mid.

Step 2. River denominator

\"4+4=8\". \"P(loan\\mid; \"P(river\\mid.

Step 3. Finance product

\"\\operatorname{score}(F)=\\frac23\\times\\frac13\\times\\frac13\\times\\frac1{12}=\\frac1{162}\\approx0.00617284.\"

Step 4. River product

\"\\operatorname{score}(R)=\\frac13\\times\\frac18\\times\\frac18\\times\\frac38=\\frac1{512}\\approx0.001953125.\"

Step 5. Compare and normalize

\"P(F\\mid

Answer. Finance wins. Repeated loan contributes a squared likelihood; do not reduce the query to a unique-word set.

Answer-writing order in an exam

Write the class priors, vocabulary size, class token totals, a likelihood table, both score products and the final class. Keep fractions until the comparison where possible. If the question only asks classification, normalization is optional; if it asks posterior probabilities, normalization is required.

Repeated word aaye toh factor bhi repeat. Sirf word-set bana diya toh multinomial answer badal jayega.

"}],"scope":"Formulas explained through full worked examples"},{"id":"f-vectors","title":"Representation & similarity","unit":0,"sections":[{"id":"u2-label-onehot","title":"Label and one-hot encoding: three complete examples","html":"

Label encoding assigns each category or word a unique integer ID. One-hot encoding assigns a length-\"V\" vector with one 1 at the word's vocabulary position and zeros elsewhere.

\"\\operatorname{onehot}(w)_j=\\begin{cases}1,&j=\\operatorname{index}(w)\\\\0,&\\text{otherwise.}\\end{cases}\"

The index is an ID, not a meaning score. One-hot represents one token/category; a multiword document generally needs multiple token vectors or an aggregation scheme.

Question. Encode “dog cat dog” using vocabulary [cat,dog,fish], IDs starting at 0. Give label IDs and a one-hot vector for every token.

Given

cat→0, dog→1, fish→2. Vector columns have the same order.

To find

The ID sequence and three one-hot rows.

Formula and why it applies

Replace each word by its ID for labels; set its vocabulary position to 1 for one-hot.

Step 1. Label IDs

dog→1, cat→0, dog→1 → [1,0,1].

Step 2. First dog

[0,1,0].

Step 3. Cat

[1,0,0].

Step 4. Second dog

[0,1,0]. Repeated tokens keep the same representation.

Answer. IDs [1,0,1]; one-hot rows [0,1,0], [1,0,0], [0,1,0].

Question. Encode the city sequence Mumbai, Delhi, Mumbai, Pune using vocabulary [Delhi,Mumbai,Pune] and label IDs starting at 1.

Given

Delhi→1, Mumbai→2, Pune→3.

To find

IDs and one-hot rows.

Formula and why it applies

One-hot uses three coordinates regardless of whether label IDs begin at 0 or 1.

Step 1. IDs

[2,1,2,3].

Step 2. Mumbai rows

Each Mumbai → [0,1,0].

Step 3. Delhi and Pune

Delhi→[1,0,0]; Pune→[0,0,1].

Answer. Rows [0,1,0], [1,0,0], [0,1,0], [0,0,1].

Question. One-hot encode categories [red,green,blue] for a linear regression with an intercept. Then give dummy coding after dropping blue.

Given

Full column order [red,green,blue]. Blue is the baseline.

To find

Full and reduced codes.

Formula and why it applies

Full one-hot rows sum to 1. With an intercept this creates a linear dependency; dropping one category column gives k−1 independent dummy indicators.

Step 1. Full encoding

Categoryredgreenblue
red100
green010
blue001

Step 2. Drop blue column

Categoryredgreen
red10
green01
blue00

Step 3. Interpret zero row

[0,0] now identifies the baseline blue, not missing data.

Answer. Full width 3; reduced dummy width 2. Dropping a column is model-dependent, not a requirement for every neural embedding input.

Limitations: one-hot is large and sparse and does not express semantic similarity. Unseen words need a defined unknown-token or vocabulary-update policy. Label IDs are compact but should not be treated as an ordered scalar feature for nominal words.

Label = ID number. One-hot = us ID ke box mein 1, baaki 0.

"},{"id":"u2-bow","title":"Bag of words and bag of n-grams","html":"

Bag of words (BoW) represents a document by the counts of vocabulary terms, discarding their positions. A binary BoW variant stores only presence/absence; a count BoW stores frequencies. State which is used.

\"x_{d,j}=C(v_j,d).\"

\"x_{d,j}\" is document \"d\"'s coordinate for vocabulary term \"v_j\". \"C(v_j,d)\" is its occurrence count.

A bag of n-grams uses short contiguous token sequences as features. Bigrams can distinguish “not good” from “good”, but increasing n enlarges the vocabulary and creates more rare features.

Question. Build count and binary BoW for D1=“cat cat dog” and D2=“dog fish”. Use vocabulary [cat,dog,fish] and no stopword removal.

Given

Two tokenized documents, fixed vocabulary order.

To find

Count vectors and binary vectors.

Formula and why it applies

Count BoW uses \"C(v_j,d)\". Binary BoW is 1 when that count is positive and 0 otherwise.

Step 1. D1 counts

cat=2, dog=1, fish=0 → [2,1,0].

Step 2. D2 counts

cat=0, dog=1, fish=1 → [0,1,1].

Step 3. Convert to presence

D1 [1,1,0]; D2 [0,1,1].

Answer. Count rows [2,1,0], [0,1,1]; binary rows [1,1,0], [0,1,1].

Question. After lowercasing and removing {is,a,means}, obtain BoW vectors for D1=“Continuous learning is learning lifetime”, D2=“Continuous learning means lifetime learning”, D3=“Learning is a good practice”. Use [continuous,good,learning,lifetime,practice].

Given

Three documents and exact stopword set from the supplied slides.

To find

The document–term count matrix.

Formula and why it applies

Remove only the listed stopwords, then count every remaining token in the fixed column order.

Step 1. Clean D1

[continuous,learning,learning,lifetime] → [1,0,2,1,0].

Step 2. Clean D2

[continuous,learning,lifetime,learning] → [1,0,2,1,0].

Step 3. Clean D3

[learning,good,practice] → [0,1,1,0,1].

Step 4. Compare D1 and D2

They have identical BoW vectors despite different original wording/order.

Answer. Rows [1,0,2,1,0], [1,0,2,1,0], [0,1,1,0,1].

Question. Build a bag of bigrams for “not good but not bad” with vocabulary [not good,good but,but not,not bad]. Do not add sentence boundary tokens.

Given

Five tokens; use adjacent pairs, stride 1.

To find

The bigram count vector.

Formula and why it applies

A sequence of \"L\" tokens has \"L-1\" adjacent bigrams when no boundaries are added.

Step 1. Enumerate pairs

(not,good), (good,but), (but,not), (not,bad).

Step 2. Count each vocabulary bigram

Each appears once.

Step 3. Write vector

[1,1,1,1].

Answer. Four bigram features, each with count 1.

BoW: order bhool, word-count rakh. Bag of bigrams: neighbouring word-pair count.

"},{"id":"u2-tfidf","title":"TF–IDF from the exact slide convention","html":"

TF–IDF weights a term highly when it is frequent in a particular document but uncommon across the corpus. The supplied Unit 2 slides use normalized term frequency and base-10 inverse document frequency:

\"\\operatorname{TF}(t,d)=\\frac{C(t,d)}{N_d}\"
\"\\operatorname{IDF}(t)=\\log_{10}\\frac{D}{\\operatorname{df}(t)}\"
\"\\operatorname{TFIDF}(t,d)=\\operatorname{TF}(t,d)\\operatorname{IDF}(t).\"

\"N_d\" is the document's token count after the specified preprocessing; \"D\" is number of documents; \"\\operatorname{df}(t)\" is the number of documents containing the term at least once. A word repeated ten times in one document contributes one to document frequency.

Why the log? It compresses the ratio's scale. If a term occurs in all documents, \"D/df=1\", so IDF=0 under this unsmoothed convention. Other libraries use smoothed IDF, different TF or normalization. Use the question's formula; do not silently substitute a library default.

Question. A corpus contains 10 documents. The word model appears in 2 documents and occurs 3 times in a 12-token document. Compute TF, IDF and TF–IDF using the slide convention.

Given

D=10, df(model)=2, C(model,d)=3, Nd=12; log base 10.

To find

The TF–IDF weight.

Formula and why it applies

\"TF=C/N_d\", \"IDF=\\log_{10}(D/df)\", then multiply.

Step 1. TF

\"3/12=0.25\".

Step 2. Document ratio

\"10/2=5\".

Step 3. IDF

\"\\log_{10}5\\approx0.698970\".

Step 4. Weight

\"0.25\\times0.698970\\approx0.174743\".

Answer. TF=0.25, IDF≈0.698970, TF–IDF≈0.174743.

Question. A term occurs 4 times in a 20-token document and is present in all 8 corpus documents. Calculate its TF–IDF.

Given

C=4, Nd=20, D=8, df=8.

To find

Its weight under unsmoothed base-10 IDF.

Formula and why it applies

\"TF=C/N_d\", \"IDF=\\log_{10}(D/df)\".

Step 1. TF

\"4/20=0.2\".

Step 2. IDF

\"\\log_{10}(8/8)=\\log_{10}1=0\".

Step 3. Multiply

\"0.2\\times0=0\".

Answer. Weight 0: this term does not distinguish documents under this convention.

Question. Find TF–IDF vectors for D1=“cat cat dog”, D2=“dog fish”, D3=“cat fish fish”, using [cat,dog,fish], normalized TF and log10 IDF.

Given

No further preprocessing. Each term occurs in exactly two of the three documents.

To find

All three vectors.

Formula and why it applies

First count df by documents, then compute each document's TF, then multiply by its term's IDF.

Step 1. Common IDF

\"D=3\", \"df(cat)=df(dog)=df(fish)=2\". Each IDF \"=\\log_{10}(3/2)\\approx0.176091\".

Step 2. D1 term frequencies

Length 3 → \"[2/3,1/3,0]\".

Step 3. D1 weights

\"[(2/3)0.176091,(1/3)0.176091,0]\\approx[0.117394,0.058697,0]\".

Step 4. D2 weights

Length 2; TF \"[0,1/2,1/2]\". Vector \"\\approx[0,0.088046,0.088046]\".

Step 5. D3 weights

Length 3; TF \"[1/3,0,2/3]\". Vector \"\\approx[0.058697,0,0.117394]\".

Answer. D1≈[0.117394,0.058697,0]; D2≈[0,0.088046,0.088046]; D3≈[0.058697,0,0.117394].

TF: iss document mein kitna. IDF: poore corpus mein kitna rare. Dono multiply.

"},{"id":"u2-cosine","title":"Cosine: products, lengths and division","html":"

Cosine similarity compares the direction of two nonzero vectors. Multiply matching coordinates and add to obtain the dot product. To obtain each vector's norm, square its coordinates, add them and take a square root.

\"\\cos(x,y)=\\frac{\\sum_i

The vectors must use the same coordinate/vocabulary order. For nonnegative BoW or TF–IDF vectors, the result lies from 0 to 1; for general real-valued vectors, it can range from −1 to 1. A zero vector makes this formula undefined.

Question. Find cosine similarity for x=[1,1] and y=[1,0].

Given

Nonzero vectors, matching coordinate order.

To find

Find cosine similarity for x=[1,1] and y=[1,0].

Formula and why it applies

Use \"\\cos(x,y)=(x\\cdot because direction similarity is required.

Step 1. Multiply matching coordinates

1 × 1 = 1; 1 × 0 = 0.

Step 2. Add products

1 + 0 = 1.

Step 3. Length of x

\"\\sqrt{(1)^2+(1)^2}=\\sqrt{2}.\"

Step 4. Length of y

\"\\sqrt{(1)^2+(0)^2}=\\sqrt{1}.\"

Step 5. Divide

\"\\frac{1}{\\sqrt{2}\\sqrt{1}}\\approx

Answer. Cosine≈0.707107.

Question. Find cosine similarity for x=[1,2] and y=[2,4].

Given

Nonzero vectors, matching coordinate order.

To find

Find cosine similarity for x=[1,2] and y=[2,4].

Formula and why it applies

Use \"\\cos(x,y)=(x\\cdot because direction similarity is required.

Step 1. Multiply matching coordinates

1 × 2 = 2; 2 × 4 = 8.

Step 2. Add products

2 + 8 = 10.

Step 3. Length of x

\"\\sqrt{(1)^2+(2)^2}=\\sqrt{5}.\"

Step 4. Length of y

\"\\sqrt{(2)^2+(4)^2}=\\sqrt{20}.\"

Step 5. Divide

\"\\frac{10}{\\sqrt{5}\\sqrt{20}}\\approx

Answer. Cosine≈1.000000.

Question. Find cosine similarity for x=[1,2,0] and y=[0,1,2].

Given

Nonzero vectors, matching coordinate order.

To find

Find cosine similarity for x=[1,2,0] and y=[0,1,2].

Formula and why it applies

Use \"\\cos(x,y)=(x\\cdot because direction similarity is required.

Step 1. Multiply matching coordinates

1 × 0 = 0; 2 × 1 = 2; 0 × 2 = 0.

Step 2. Add products

0 + 2 + 0 = 2.

Step 3. Length of x

\"\\sqrt{(1)^2+(2)^2+(0)^2}=\\sqrt{5}.\"

Step 4. Length of y

\"\\sqrt{(0)^2+(1)^2+(2)^2}=\\sqrt{5}.\"

Step 5. Divide

\"\\frac{2}{\\sqrt{5}\\sqrt{5}}\\approx

Answer. Cosine≈0.400000.

Proportional vectors [1,2] and [2,4] have cosine 1 but nonzero Euclidean distance. Direction similarity does not require identical magnitudes.

Cosine: matching products add upar; dono lengths multiply neeche.

"},{"id":"u2-jaccard","title":"Jaccard: intersection, union and distance","html":"

Jaccard similarity is the fraction of shared distinct elements among all distinct elements in either set.

\"J(A,B)=\\frac{|A\\cap

Intersection means common elements; union means all distinct elements. Repetitions do not count again. Jaccard distance is \"1-J\", not the same quantity as similarity. Two empty sets require a declared convention because the fraction becomes 0/0.

Question. Find Jaccard similarity and distance for A={cat, dog} and B={dog, fish}.

Given

Sets contain unique elements.

To find

Find Jaccard similarity and distance for A={cat, dog} and B={dog, fish}.

Formula and why it applies

\"J=|A\\cap; \"d=1-J\".

Step 1. Find intersection

{dog}; count 1.

Step 2. Find union

{cat, dog, fish}; count 3.

Step 3. Divide

J=1/3≈0.333333.

Step 4. Subtract from one

Distance=1−0.333333≈0.666667.

Answer. Similarity≈0.333333; distance≈0.666667.

Question. Find Jaccard similarity and distance for A={red, blue} and B={red, blue}.

Given

Sets contain unique elements.

To find

Find Jaccard similarity and distance for A={red, blue} and B={red, blue}.

Formula and why it applies

\"J=|A\\cap; \"d=1-J\".

Step 1. Find intersection

{red, blue}; count 2.

Step 2. Find union

{red, blue}; count 2.

Step 3. Divide

J=2/2≈1.000000.

Step 4. Subtract from one

Distance=1−1.000000≈0.000000.

Answer. Similarity≈1.000000; distance≈0.000000.

Question. Find Jaccard similarity and distance for A={apple} and B={orange, pear}.

Given

Sets contain unique elements.

To find

Find Jaccard similarity and distance for A={apple} and B={orange, pear}.

Formula and why it applies

\"J=|A\\cap; \"d=1-J\".

Step 1. Find intersection

{}; count 0.

Step 2. Find union

{apple, orange, pear}; count 3.

Step 3. Divide

J=0/3≈0.000000.

Step 4. Subtract from one

Distance=1−0.000000≈1.000000.

Answer. Similarity≈0.000000; distance≈1.000000.

Jaccard = common unique ÷ total unique. Distance = 1 minus similarity.

"},{"id":"u2-distances","title":"Manhattan, Euclidean and Minkowski: three complete comparisons","html":"

Distance measures separation; smaller means closer. Absolute values prevent positive and negative differences from cancelling.

\"d_1=\\sum_i|x_i-y_i|\\quad\\text{Manhattan}\"
\"d_2=\\sqrt{\\sum_i(x_i-y_i)^2}\\quad\\text{Euclidean}\"
\"d_p=\\left(\\sum_i|x_i-y_i|^p\\right)^{1/p},\\quad

Manhattan is p=1; Euclidean is p=2. For p=3, cube absolute differences, add, then take a cube root.

Question. Find Manhattan, Euclidean and Minkowski p=3 distances for x=[1,2], y=[4,6].

Given

Vectors share coordinate order; negative inputs are allowed.

To find

Find Manhattan, Euclidean and Minkowski p=3 distances for x=[1,2], y=[4,6].

Formula and why it applies

Use \"d_1=\\sum|\\Delta_i|\", \"d_2=\\sqrt{\\sum|\\Delta_i|^2}\", \"d_3=\\sqrt[3]{\\sum|\\Delta_i|^3}\".

Step 1. Absolute differences

|1 − (4)| = 3; |2 − (6)| = 4.

Step 2. Manhattan

3 + 4 = 7.

Step 3. Square and add

3²=9; 4²=16. Sum=25.

Step 4. Euclidean root

\"\\sqrt{25}\\approx

Step 5. Cube and add

3³=27; 4³=64. Sum=91.

Step 6. Minkowski root

\"\\sqrt[3]{91}\\approx

Answer. Manhattan=7; Euclidean≈5.000000; p=3≈4.497941.

Question. Find Manhattan, Euclidean and Minkowski p=3 distances for x=[0,1,2], y=[2,1,0].

Given

Vectors share coordinate order; negative inputs are allowed.

To find

Find Manhattan, Euclidean and Minkowski p=3 distances for x=[0,1,2], y=[2,1,0].

Formula and why it applies

Use \"d_1=\\sum|\\Delta_i|\", \"d_2=\\sqrt{\\sum|\\Delta_i|^2}\", \"d_3=\\sqrt[3]{\\sum|\\Delta_i|^3}\".

Step 1. Absolute differences

|0 − (2)| = 2; |1 − (1)| = 0; |2 − (0)| = 2.

Step 2. Manhattan

2 + 0 + 2 = 4.

Step 3. Square and add

2²=4; 0²=0; 2²=4. Sum=8.

Step 4. Euclidean root

\"\\sqrt{8}\\approx

Step 5. Cube and add

2³=8; 0³=0; 2³=8. Sum=16.

Step 6. Minkowski root

\"\\sqrt[3]{16}\\approx

Answer. Manhattan=4; Euclidean≈2.828427; p=3≈2.519842.

Question. Find Manhattan, Euclidean and Minkowski p=3 distances for x=[-1,3], y=[2,-1].

Given

Vectors share coordinate order; negative inputs are allowed.

To find

Find Manhattan, Euclidean and Minkowski p=3 distances for x=[-1,3], y=[2,-1].

Formula and why it applies

Use \"d_1=\\sum|\\Delta_i|\", \"d_2=\\sqrt{\\sum|\\Delta_i|^2}\", \"d_3=\\sqrt[3]{\\sum|\\Delta_i|^3}\".

Step 1. Absolute differences

|-1 − (2)| = 3; |3 − (-1)| = 4.

Step 2. Manhattan

3 + 4 = 7.

Step 3. Square and add

3²=9; 4²=16. Sum=25.

Step 4. Euclidean root

\"\\sqrt{25}\\approx

Step 5. Cube and add

3³=27; 4³=64. Sum=91.

Step 6. Minkowski root

\"\\sqrt[3]{91}\\approx

Answer. Manhattan=7; Euclidean≈5.000000; p=3≈4.497941.

Manhattan: absolute-add. Euclidean: square-add-root. Minkowski: p-power-add-p-root.

"},{"id":"u5-distribution","title":"Distributional hypothesis and a co-occurrence matrix","html":"

The distributional hypothesis states that words occurring in similar contexts tend to have similar meanings or semantic behaviour. “Cat” and “dog” often occur near pet, feed and animal. Their context patterns can therefore be represented and compared numerically.

A vector is an ordered list of numbers. Each position must have a fixed meaning. For context order [pet, food, road], cat=[4,3,0] means four co-occurrences with pet, three with food, none with road. A matrix stacks vectors as rows in a table.

A word–context co-occurrence matrix records how often a target word occurs with each context word under a specified context definition. A window may be a sentence, document, or a fixed number of neighbouring tokens. State its width, direction and whether targets count themselves.

Choose target words
→
Choose context/window
→
Count co-occurrences
→
Store each row as a vector
→
Compare vectors

Question. Build the symmetric co-occurrence matrix for “cats eat fish” using a window of one token on either side, no self-counts and vocabulary [cats,eat,fish].

Given

One sentence, one occurrence of each token. Left and right adjacent tokens are contexts.

To find

A 3×3 target-by-context count matrix.

Formula and why it applies

\"X_{ij}\" counts the number of times context word \"j\" occurs within the chosen window of target \"i\".

Step 1. Target cats

Only eat is adjacent. Row cats = [0,1,0].

Step 2. Target eat

cats is left and fish is right. Row eat = [1,0,1].

Step 3. Target fish

Only eat is adjacent. Row fish = [0,1,0].

Step 4. Assemble the matrix

Target / contextcatseatfish
cats010
eat101
fish010

Answer. Rows are [0,1,0], [1,0,1], [0,1,0]. The symmetry follows from the symmetric window.

Question. Build a target–context matrix for sentences “cats eat fish” and “dogs eat fish”. Target words are cats and dogs; context columns [eat,fish]; use the entire sentence excluding the target.

Given

Each sentence contains one target. Count context token occurrences.

To find

Two context vectors.

Formula and why it applies

Count each context word in every sentence containing the target.

Step 1. Count cats contexts

In its sentence: eat occurs once, fish once → [1,1].

Step 2. Count dogs contexts

In its sentence: eat occurs once, fish once → [1,1].

Step 3. Compare

Both targets have identical observed context patterns in this very small corpus.

Answer. cats=[1,1], dogs=[1,1]. This corpus supplies no context evidence to distinguish them.

Question. For “a b a”, use a right-only window of one token. Build the matrix with rows/columns [a,b].

Given

Count directed target→next-word events, not both directions.

To find

The co-occurrence matrix.

Formula and why it applies

\"X_{ij}=C(\\text{target.

Step 1. First target a

Next token is b → add 1 to X(a,b).

Step 2. Target b

Next token is a → add 1 to X(b,a).

Step 3. Final a

No following token → no addition.

Step 4. Write matrix

Target / nextab
a01
b10

Answer. a=[0,1], b=[1,0]. A different context window would produce a different matrix.

Word ka meaning: uske aas-paas kaunse words kitni baar aaye, uski number-list.

"},{"id":"u5-word2vec","title":"Word2Vec: CBOW and skip-gram from training pairs","html":"

Word2Vec learns dense word embeddings by predicting words from nearby context. Its two common architectures are CBOW and skip-gram. A vocabulary ID or one-hot vector is the input representation; the learned embedding table converts that identity into a short trainable vector.

ArchitectureInputPredictionMemory cue
CBOWSurrounding context wordsCentre/target wordContext → centre
Skip-gramCentre/target wordSurrounding wordsCentre → context
CBOW: the + sat
→
Average context embeddings
→
Predict: cat
Skip-gram: cat
→
Embedding of cat
→
Predict: the and sat

For vocabulary size \"V\" and embedding dimension \"d\", an input one-hot row is \"1\\times; the input embedding matrix is \"V\\times; the hidden row is \"1\\times; the output matrix is \"d\\times; output scores are \"1\\times. Multiplying a one-hot row by the embedding matrix simply selects one row.

\"P(w_o\\mid

\"v_{w_i}\" is the input word embedding, \"u_w\" an output-context embedding, and the dot product is a compatibility score. Softmax turns all scores into probabilities that sum to one. Training increases the probability of observed context words. Negative sampling or hierarchical softmax can reduce computation in larger vocabularies; use the algorithm specified by the question.

Question. For the sentence “the cat sat on mat”, generate CBOW and skip-gram training data for centre word sat with a window of one token on each side.

Given

Tokens indexed 1 to 5: the, cat, sat, on, mat. Centre = sat.

To find

The CBOW example and the skip-gram pairs.

Formula and why it applies

CBOW predicts the centre from its neighbours; skip-gram predicts each neighbour from the centre.

Step 1. Identify neighbours

Left neighbour = cat; right neighbour = on. “the” and “mat” are outside this window.

Step 2. CBOW example

Input [cat,on] → target sat.

Step 3. Skip-gram examples

(sat,cat) and (sat,on), with first item as input and second as predicted context.

Answer. CBOW: [cat,on]→sat. Skip-gram: sat→cat and sat→on.

Question. The embeddings of context words cat and on are [1,2] and [3,0]. Compute the CBOW hidden vector using mean aggregation. Output vectors for [sat,ran] are [1,0] and [0,1]. Compute softmax probabilities.

Given

Context embeddings [1,2], [3,0]; mean aggregation; two candidate output vectors.

To find

The hidden vector, logits and probabilities.

Formula and why it applies

Mean context vector \"h=(v_1+v_2)/2\". Logit \"z_w=u_w^Th\". Softmax \"P(w)=e^{z_w}/\\sum_j.

Step 1. Add coordinates

\"[1+3,\\,2+0]=[4,2]\".

Step 2. Divide by number of words

\"h=[4/2,2/2]=[2,1]\".

Step 3. Compute logits

\"z_{sat}=1(2)+0(1)=2\". \"z_{ran}=0(2)+1(1)=1\".

Step 4. Exponentials

\"e^2\\approx7.389056\", \"e^1\\approx2.718282\"; sum \"\\approx10.107338\".

Step 5. Divide

\"P(sat)\\approx0.731059\", \"P(ran)\\approx0.268941\".

Answer. h=[2,1]; sat is more probable (0.731059).

Question. A skip-gram centre embedding is v=[1,−1]. Output embeddings for [eat,sleep,drive] are [1,1], [0,1], [1,0]. Calculate the predicted context probabilities.

Given

One centre vector and three output vectors; full softmax over these three words.

To find

Three logits and three probabilities.

Formula and why it applies

\"z_j=u_j^Tv\", then \"P(j)=e^{z_j}/\\sum_k.

Step 1. Eat dot product

\"1(1)+1(-1)=0\".

Step 2. Sleep and drive dot products

\"z_{sleep}=0(1)+1(-1)=-1\". \"z_{drive}=1(1)+0(-1)=1\".

Step 3. Exponentiate and sum

\"e^0=1\", \"e^{-1}\\approx0.367879\", \"e^1\\approx2.718282\". Sum \"\\approx4.086161\".

Step 4. Normalize

\"P(eat)\\approx0.244728\"; \"P(sleep)\\approx0.090031\"; \"P(drive)\\approx0.665241\".

Answer. drive has the highest probability in this toy learned model.

Limitations: a standard static Word2Vec vector merges a word's senses; it has limited representation of subword morphology and out-of-vocabulary words; its local prediction objective does not directly fit a full global co-occurrence table. Similarity is learned from data and is not a guarantee of factual or causal understanding.

CBOW = neighbours se missing centre. Skip-gram = centre se neighbours.

"},{"id":"u5-sentence","title":"Word analogies and sentence embeddings","html":"

A sentence embedding represents an entire sentence by one fixed-length vector. A basic method averages word embeddings. More advanced encoders learn a sentence representation from context and task objectives.

\"v_{\\text{sentence}}=\\frac1n\\sum_{i=1}^{n}v_{w_i}.\"

This means add corresponding coordinates and divide each coordinate by the number of words. Averaging ignores order: “dog bites man” and “man bites dog” have the same average if their words are identical.

Question. Average the word vectors [1,0], [0,2] and [2,1] to represent a three-word sentence.

Given

Three two-dimensional vectors with equal weights.

To find

The two-dimensional sentence vector.

Formula and why it applies

Use coordinate-wise mean \"v=(v_1+v_2+v_3)/3\".

Step 1. First coordinate

\"1+0+2=3\"; \"3/3=1\".

Step 2. Second coordinate

\"0+2+1=3\"; \"3/3=1\".

Answer. Sentence vector = [1,1].

Question. Toy word vectors are king=[3,2], man=[1,0], woman=[1,1]. Find king−man+woman and compare to queen=[3,3].

Given

All vectors use the same two-coordinate space.

To find

The analogy vector and its distance from queen.

Formula and why it applies

Vector arithmetic is coordinate by coordinate. Euclidean distance is the square root of the sum of squared coordinate differences.

Step 1. Subtract man

\"[3-1,2-0]=[2,2]\".

Step 2. Add woman

\"[2+1,2+1]=[3,3]\".

Step 3. Compare with queen

\"\\sqrt{(3-3)^2+(3-3)^2}=0\".

Answer. The toy analogy exactly equals queen. Real learned embeddings only approximate some analogies.

Contextual encoders can assign “bank” different representations in different sentences. BERT and RoBERTa produce contextual token representations; obtaining strong sentence similarity vectors requires an appropriate pooling/training method, not simply assuming every raw token vector is a sentence embedding. Universal Sentence Encoder is designed to encode sentence-level meaning. Skip-thought predicts surrounding sentences; InferSent learns from natural-language-inference supervision.

Sentence average: har column add karo, words ki count se divide. Word order ka loss yaad rakh.

"},{"id":"u5-glove-loss","title":"The GloVe objective and numerical interpretation","html":"
\"J=\\sum_{i,j:X_{ij}>0}f(X_{ij})\\left(w_i^T\\widetilde

What each piece means: \"w_i\" is a target vector; \"\\widetilde is a separate context vector; the dot product estimates their compatibility; biases adjust word-level baselines. The desired score is the natural log of the observed count. Squaring penalizes both positive and negative errors. Weighting prevents unreliable rare pairs or extremely frequent pairs from dominating inappropriately.

\"f(x)=\\begin{cases}(x/x_{\\max})^\\alpha,&0<x<x_{\\max}\\\\1,&x\\ge

The original commonly used settings are \"x_{\\max}=100\", \"\\alpha=0.75\". They are hyperparameters, not learned word embeddings. Zero-count pairs do not require evaluating \"\\ln in the displayed nonzero-count objective.

Question. For one nonzero GloVe pair, X=4, weighting f(X)=0.5, target vector [1,2], context vector [0.5,0], and both biases 0. Find its contribution to J.

Given

Use natural logarithm. Only one pair is being evaluated.

To find

Weighted squared residual.

Formula and why it applies

\"J_{ij}=f(X)(w^T\\widetilde.

Step 1. Dot product

\"1(0.5)+2(0)=0.5\".

Step 2. Log-count target

\"\\ln4\\approx1.386294\".

Step 3. Residual

\"0.5+0+0-1.386294=-0.886294\".

Step 4. Square and weight

\"(-0.886294)^2\\approx0.785518\". Multiply by 0.5 → approximately 0.392759.

Answer. The pair contributes approximately 0.392759 to the objective.

Question. A GloVe pair has X=e², weight 1, dot product 1.5 and biases 0.2 and 0.3. Calculate its objective contribution.

Given

Natural logarithm, so ln(e²)=2.

To find

The squared weighted error.

Formula and why it applies

\"J_{ij}=f(X)(\\text{dot}+b+\\widetilde.

Step 1. Predicted score

\"1.5+0.2+0.3=2\".

Step 2. Target score

\"\\ln(e^2)=2\".

Step 3. Residual and square

\"2-2=0\"; \"1\\times0^2=0\".

Answer. Contribution 0: this pair is fitted exactly.

Question. Use xmax=100 and α=0.75. A GloVe pair has X=16 and residual 2. Find its weighting and loss contribution.

Given

16 is below xmax, so use the power branch.

To find

f(16) and weighted squared residual.

Formula and why it applies

\"f(16)=(16/100)^{0.75}\", and \"J_{ij}=f(16)\\times2^2\".

Step 1. Ratio

\"16/100=0.16\".

Step 2. Power

\"0.16^{0.75}=0.16^{3/4}\\approx0.252982\".

Step 3. Square residual

\"2^2=4\".

Step 4. Multiply

\"0.252982\\times4\\approx1.011929\".

Answer. f≈0.252982 and contribution≈1.011929.

Six steps to explain the algorithm

  1. Collect a corpus and choose a context window.
  2. Build the global co-occurrence matrix.
  3. Study row-normalized context probabilities and their ratios.
  4. Define the weighted log-count fitting objective.
  5. Optimize word/context vectors and biases.
  6. Use the learned embeddings for similarity, analogy or downstream tasks.

GloVe: global counts → log target → dot product prediction → weighted squared error kam karo.

"}],"scope":"Formulas explained through full worked examples"},{"id":"f-applications","title":"Application calculations","unit":0,"sections":[{"id":"classification","title":"Text classification and its full pipeline","html":"

Text classification assigns a document to one or more predefined categories. Spam detection usually uses two classes; news classification may use several topics. In single-label multiclass classification, choose one of several categories. In multilabel classification, several labels may apply to the same document.

Define labels
→
Prepare labelled examples
→
Train representation + model
→
Evaluate on unseen text
→
Predict new labels

An exam-ready pipeline

StageWhat you doWhy it matters
Task definitionDecide what counts as spam or which news topics exist.Unclear labels create inconsistent training data.
Data collection and annotationCollect representative examples and assign correct labels.The model learns from these examples.
Split dataSeparate training, validation and test data.Test examples must not influence training choices.
PreprocessTokenize and normalise where suitable; retain meaningful negation.Reduce irrelevant variation without deleting signal.
RepresentUse BoW, TF-IDF or learned embeddings.Models need numerical features.
TrainFit Naive Bayes, logistic regression or another appropriate model.Learn how features predict categories.
EvaluateMeasure relevant precision, recall and F1 on held-out examples.Accuracy alone may hide poor minority-class performance.
Deploy and monitorApply the same preprocessing and monitor changing language.Incoming data can differ from old examples.

Naive Bayes is especially useful for hand calculations. With a bag-of-words representation, its class score is the prior multiplied by the conditional probability of each token; repeated tokens repeat the factor. The “naive” assumption is conditional independence of features given the class, not that words are independent in real language.

\"s(c)=P(c)\\prod_{w\\in
\"P(w\\mid

C(w,d) counts word w in document d. C(w,c) counts it in training documents belonging to class c. Nc is the total token count for class c. Add-one smoothing adds one to every vocabulary count and therefore adds |V| to the denominator. This is the same arithmetic taught for WSD, with document classes instead of senses.

Worked classification · two-token message

A spam classifier has vocabulary {win, prize, meeting}. Priors are P(spam)=P(ham)=0.5. Spam token counts are [3,2,0]; ham counts are [0,0,5]. With add-one smoothing, classify “win prize”.

Given

|V|=3; N(spam)=5; N(ham)=5; message has win once and prize once.

To find

Class scores and predicted label.

Formula and why it applies

\"s(c)=P(c)P(\\text{win}\\mid
\"P(w\\mid

The model is multinomial Naive Bayes; multiply the two token likelihoods and the class prior.

Solution

Step 1. Compute the smoothed denominators

\"5+3=8\"

Step 2. Compute spam likelihoods

\"P(\\text{win}\\mid
\"P(\\text{prize}\\mid

Step 3. Compute ham likelihoods

\"P(\\text{win}\\mid

Step 4. Multiply the spam score

\"s(S)=\\frac12\\times\\frac12\\times\\frac38=\\frac{3}{32}=0.09375\"

Step 5. Multiply the ham score

\"s(H)=\\frac12\\times\\frac18\\times\\frac18=\\frac{1}{128}=0.0078125\"

Step 6. Compare

0.09375 > 0.0078125. These are unnormalised scores; the label is chosen by the larger score.

Predicted label: spam.

Yaad rakho: Class prior × har word ki class-likelihood; sabse bada score wins.

"},{"id":"evaluation","title":"Precision, recall, F1 and accuracy","html":"

Evaluation compares predicted results with known answers. A true positive is a correctly predicted positive; a false positive is a negative item predicted positive; a false negative is a missed positive; a true negative is a correctly rejected negative.

\"\\text{Precision}=\\frac{TP}{TP+FP}\"
\"\\text{Recall}=\\frac{TP}{TP+FN}\"
\"F_1=\\frac{2PR}{P+R}=\\frac{2TP}{2TP+FP+FN}\"
\"\\text{Accuracy}=\\frac{TP+TN}{TP+TN+FP+FN}\"

Precision asks how many predicted positives were correct. Recall asks how many actual positives were found. F1 balances them by a harmonic mean. If a denominator is zero, state the convention rather than dividing by zero.

Example 1 · evaluate a classifier

A binary text classifier gives TP=6, FP=2, FN=4, TN=8. Compute precision, recall, F1 and accuracy.

Given

TP=6; FP=2; FN=4; TN=8.

To find

Four evaluation measures.

Formula and why it applies

\"P=\\frac{TP}{TP+FP},\\quad
\"F_1=\\frac{2TP}{2TP+FP+FN},\\quad

Use predicted positives in precision, actual positives in recall, and all examples in accuracy.

Solution

Step 1. Precision

\"P=\\frac{6}{6+2}=0.750000\"

Step 2. Recall

\"R=\\frac{6}{6+4}=0.600000\"

Step 3. F1

\"F_1=\\frac{12}{12+2+4}=0.666667\"

Step 4. Accuracy

\"A=\\frac{6+8}{6+2+4+8}=0.700000\"
Precision 0.7500; recall 0.6000; F1 0.6667; accuracy 0.7000.

Example 2 · evaluate a classifier

A binary text classifier gives TP=9, FP=3, FN=1, TN=7. Compute precision, recall, F1 and accuracy.

Given

TP=9; FP=3; FN=1; TN=7.

To find

Four evaluation measures.

Formula and why it applies

\"P=\\frac{TP}{TP+FP},\\quad
\"F_1=\\frac{2TP}{2TP+FP+FN},\\quad

Use predicted positives in precision, actual positives in recall, and all examples in accuracy.

Solution

Step 1. Precision

\"P=\\frac{9}{9+3}=0.750000\"

Step 2. Recall

\"R=\\frac{9}{9+1}=0.900000\"

Step 3. F1

\"F_1=\\frac{18}{18+3+1}=0.818182\"

Step 4. Accuracy

\"A=\\frac{9+7}{9+3+1+7}=0.800000\"
Precision 0.7500; recall 0.9000; F1 0.8182; accuracy 0.8000.

Example 3 · evaluate a classifier

A binary text classifier gives TP=4, FP=0, FN=4, TN=12. Compute precision, recall, F1 and accuracy.

Given

TP=4; FP=0; FN=4; TN=12.

To find

Four evaluation measures.

Formula and why it applies

\"P=\\frac{TP}{TP+FP},\\quad
\"F_1=\\frac{2TP}{2TP+FP+FN},\\quad

Use predicted positives in precision, actual positives in recall, and all examples in accuracy.

Solution

Step 1. Precision

\"P=\\frac{4}{4+0}=1.000000\"

Step 2. Recall

\"R=\\frac{4}{4+4}=0.500000\"

Step 3. F1

\"F_1=\\frac{8}{8+0+4}=0.666667\"

Step 4. Accuracy

\"A=\\frac{4+12}{4+0+4+12}=0.800000\"
Precision 1.0000; recall 0.5000; F1 0.6667; accuracy 0.8000.
"},{"id":"spelling","title":"Spelling correction and edit distance","html":"

Spelling correction proposes the intended word or sentence when the observed text contains an error. A non-word error produces an invalid vocabulary item (“recieve”); a real-word error produces an existing word in the wrong context (“I will meat you”). Dictionary lookup can detect many non-words but cannot reliably detect real-word errors.

Detect suspicious text
→
Generate candidates
→
Score edit + context fit
→
Choose correction

Minimum edit distance

Levenshtein distance is the minimum total cost of insertions, deletions and substitutions needed to transform one string into another. In the examples below, each edit costs 1; matching characters cost 0. Transposition is not a single Levenshtein operation. State different costs explicitly if the question gives them.

\"D(i,j)=\\min\\begin{cases}D(i-1,j)+1\\\\D(i,j-1)+1\\\\D(i-1,j-1)+[a_i\\ne

Rows represent prefixes of the source; columns represent prefixes of the target. D(i,j) is the best cost for transforming the first i source characters to the first j target characters. Initialise D(i,0)=i and D(0,j)=j. The three choices are deletion, insertion and diagonal match/substitution.

Example 1 · cat → cut

Calculate the unit-cost Levenshtein distance from “cat” to “cut”. Show the dynamic-programming table.

Given

Source=cat; target=cut; insertion=deletion=substitution=1; match=0.

To find

Minimum edit cost.

Formula and why it applies

\"D(i,j)=\\min(D(i-1,j)+1,D(i,j-1)+1,D(i-1,j-1)+c)\"

c = 0 for matching characters, 1 otherwise. Compare all three possible ways to reach each cell.

Solution

Step 1. Initialise empty prefixes

First row counts insertions; first column counts deletions.

Step 2. Fill row 1: source character “c”

\"D(1,1)=\\min(1+1,\\
\"D(1,2)=\\min(2+1,\\
\"D(1,3)=\\min(3+1,\\

Step 3. Fill row 2: source character “a”

\"D(2,1)=\\min(0+1,\\
\"D(2,2)=\\min(1+1,\\
\"D(2,3)=\\min(2+1,\\

Step 4. Fill row 3: source character “t”

\"D(3,1)=\\min(1+1,\\
\"D(3,2)=\\min(1+1,\\
\"D(3,3)=\\min(2+1,\\

Step 5. Read the completed matrix

Source / target∅cut
∅0123
c1012
a2112
t3221
Minimum edit distance = 1.

Example 2 · at → cat

Calculate the unit-cost Levenshtein distance from “at” to “cat”. Show the dynamic-programming table.

Given

Source=at; target=cat; insertion=deletion=substitution=1; match=0.

To find

Minimum edit cost.

Formula and why it applies

\"D(i,j)=\\min(D(i-1,j)+1,D(i,j-1)+1,D(i-1,j-1)+c)\"

c = 0 for matching characters, 1 otherwise. Compare all three possible ways to reach each cell.

Solution

Step 1. Initialise empty prefixes

First row counts insertions; first column counts deletions.

Step 2. Fill row 1: source character “a”

\"D(1,1)=\\min(1+1,\\
\"D(1,2)=\\min(2+1,\\
\"D(1,3)=\\min(3+1,\\

Step 3. Fill row 2: source character “t”

\"D(2,1)=\\min(1+1,\\
\"D(2,2)=\\min(1+1,\\
\"D(2,3)=\\min(2+1,\\

Step 4. Read the completed matrix

Source / target∅cat
∅0123
a1112
t2221
Minimum edit distance = 1.

Example 3 · cats → cat

Calculate the unit-cost Levenshtein distance from “cats” to “cat”. Show the dynamic-programming table.

Given

Source=cats; target=cat; insertion=deletion=substitution=1; match=0.

To find

Minimum edit cost.

Formula and why it applies

\"D(i,j)=\\min(D(i-1,j)+1,D(i,j-1)+1,D(i-1,j-1)+c)\"

c = 0 for matching characters, 1 otherwise. Compare all three possible ways to reach each cell.

Solution

Step 1. Initialise empty prefixes

First row counts insertions; first column counts deletions.

Step 2. Fill row 1: source character “c”

\"D(1,1)=\\min(1+1,\\
\"D(1,2)=\\min(2+1,\\
\"D(1,3)=\\min(3+1,\\

Step 3. Fill row 2: source character “a”

\"D(2,1)=\\min(0+1,\\
\"D(2,2)=\\min(1+1,\\
\"D(2,3)=\\min(2+1,\\

Step 4. Fill row 3: source character “t”

\"D(3,1)=\\min(1+1,\\
\"D(3,2)=\\min(0+1,\\
\"D(3,3)=\\min(1+1,\\

Step 5. Fill row 4: source character “s”

\"D(4,1)=\\min(2+1,\\
\"D(4,2)=\\min(1+1,\\
\"D(4,3)=\\min(0+1,\\

Step 6. Read the completed matrix

Source / target∅cat
∅0123
c1012
a2101
t3210
s4321
Minimum edit distance = 1.

Noisy-channel correction

\"\\hat

x is the observed misspelling and w a candidate intended word. P(x | w) is the error-model likelihood; P(w) is the language-model prior. In context, replace the isolated-word prior with an appropriate contextual probability. A frequent word with a plausible typo may beat a rarer candidate, but context can reverse the ranking.

Candidate ranking

For observed “teh”, candidate “the” has prior 0.05 and error likelihood 0.02. Candidate “ten” has prior 0.01 and likelihood 0.01. Select the intended word.

Given

Two candidates and their prior/error-model likelihoods are supplied.

To find

Largest noisy-channel score.

Formula and why it applies

\"s(w)=P(x\\mid

Multiply likelihood by prior; the same evidence normaliser is irrelevant to ranking.

Solution

Step 1. Score the

\"s(\\text{the})=0.02\\times0.05=0.001\"

Step 2. Score ten

\"s(\\text{ten})=0.01\\times0.01=0.0001\"

Step 3. Compare

0.001 is ten times 0.0001.

Choose “the”.
"},{"id":"retrieval","title":"Information retrieval","html":"

Information retrieval (IR) locates and ranks items relevant to an information need, usually expressed as a query. The collection may contain documents, product descriptions or scientific papers. Retrieval estimates relevance; it does not itself establish that every retrieved claim is true.

Documents → cleaning
→
Build index
→
Query → same representation
→
Match & rank
→
Return relevant results

The inverted index

An inverted index maps a term to the documents containing it, often with positions or counts. For D1 “cats drink milk” and D2 “dogs drink water”, the posting list for drink is {D1,D2}, while milk maps only to D1. This avoids rescanning every document for every query.

TermPosting list
catsD1
drinkD1, D2
milkD1
dogsD2
waterD2

Matching and ranking

Boolean retrieval combines term membership using AND, OR and NOT. Vector-space retrieval represents documents and queries as weighted vectors, for example with TF-IDF, and compares them with cosine similarity. Learned retrieval can use dense embeddings for meaning-based matching. The representation used at query time must be compatible with the document index.

Search engines and recommendation

A search engine receives an explicit query and ranks matching indexed pages. A recommender may derive an implicit query from a user’s liked items or profile and retrieve similar candidates. Both require candidate retrieval and ranking, but the source of the information need differs.

Evaluate the results

For retrieval, TP is the number of retrieved relevant documents, FP retrieved irrelevant documents, and FN relevant documents not retrieved. Precision and recall use the same formulas as classification. Ranking measures also consider the positions of relevant items; a relevant result at rank 1 is more useful than one far below.

Retrieval evaluation

A search system retrieves 8 documents; 6 are relevant. There are 10 relevant documents in the collection. Find precision, recall and F1.

Given

Retrieved=8; relevant retrieved=6; all relevant=10.

To find

P, R and F1.

Formula and why it applies

\"P=\\frac{\\text{relevant
\"F_1=\\frac{2PR}{P+R}\"

The denominator is 8 for precision and 10 for recall because they ask different questions.

Solution

Step 1. Identify the confusion counts

TP=6; FP=8−6=2; FN=10−6=4.

Step 2. Precision

\"P=\\frac68=0.75\"

Step 3. Recall

\"R=\\frac6{10}=0.6\"

Step 4. F1

\"F_1=\\frac{2(0.75)(0.6)}{0.75+0.6}=\\frac{0.9}{1.35}=\\frac23\\approx0.666667\"
Precision=0.75, recall=0.60, F1≈0.666667.

Yaad rakho: Precision: jo laaye, unmein kitne sahi? Recall: jo sahi the, unmein kitne laaye?

"}],"scope":"Formulas explained through full worked examples"},{"id":"f-deep-u3-smoothing-extensions","title":"Faculty topics: interpolation and Stupid Backoff","unit":3,"mode":"tee","order":100,"scope":"Unit 3 PDF p147 names these methods. Neither supplied TEE paper asks their numerical calculation; the worked values below are authored examples.","sections":[{"id":"interpolation","title":"N-gram interpolation: weighted evidence from several histories","html":"

Faculty topic; not an observed numerical PYQ. Unit 3 PDF p147 names these methods. Neither supplied TEE paper asks their numerical calculation; the worked values below are authored examples.

Interpolation combines several context lengths for every prediction. A trigram estimate is specific but may have little evidence. Bigram and unigram estimates use less context and can support unseen longer sequences. A weighted mixture combines their information.

\"P_{\\mathrm{mix}}(w\\mid
\"\\lambda_1,\\lambda_2,\\lambda_3\\ge0,\\qquad\\lambda_1+\\lambda_2+\\lambda_3=1.\"

Each component must be a normalized distribution over the same vocabulary. Then summing the mixture over all possible words gives λ₁+λ₂+λ₃=1. A zero higher-order count can be rescued by positive lower-order evidence. Unlike simple backoff, interpolation can combine all levels even when the trigram was seen.

Question. For predicting tea after I like, suppose trigram, bigram and unigram probabilities are 0, 0.2 and 0.1. Use weights λ₃=0.5, λ₂=0.3 and λ₁=0.2 to calculate the interpolated probability. Repeat for coffee with probabilities 0.4, 0.1 and 0.05. [Authored worked example]

Answer

Given

The three distributions use one vocabulary. Nonnegative weights sum to 1. These are supplied probabilities; no count estimation is needed.

To find

Pmix(tea | I,like) and Pmix(coffee | I,like).

Formula and why it applies

\"P_{mix}=\\lambda_3P_3+\\lambda_2P_2+\\lambda_1P_1\"

We add weighted alternatives from models, rather than multiply sequential word events.

Solution

Step 1. Check the weights
\"0.5+0.3+0.2=1.\"
Step 2. Calculate the three tea contributions
\"0.5(0)=0,\\qquad0.3(0.2)=0.06,\\qquad0.2(0.1)=0.02.\"
Step 3. Add the tea contributions
\"P_{mix}(tea)=0+0.06+0.02=0.08.\"
Step 4. Calculate the coffee contributions
\"0.5(0.4)=0.20,\\qquad0.3(0.1)=0.03,\\qquad0.2(0.05)=0.01.\"
Step 5. Add and compare
\"P_{mix}(coffee)=0.20+0.03+0.01=0.24.\"

Coffee gets three times tea’s probability in this model. Other vocabulary outcomes carry the remaining mass; these two displayed entries are not the whole distribution.

Final answer. Tea=0.08 and coffee=0.24. A zero trigram probability does not force a zero mixture.

Yaad rakho: Interpolation = sab context-lengths ka weighted mix. Word-sequence product aur model-mixture sum alag operations hain.

"},{"id":"stupid-backoff","title":"Backoff and the important score-versus-probability distinction","html":"

Faculty topic; not an observed numerical PYQ. Unit 3 PDF p147 names these methods. Neither supplied TEE paper asks their numerical calculation; the worked values below are authored examples.

Backoff uses a shorter history when the longer-history estimate lacks evidence. A properly normalized backoff model discounts some seen-event probability and allocates the remaining mass across unseen continuations using lower-order probabilities and normalization factors.

Stupid Backoff is a specifically named efficient scoring method in the slides’ list. It uses an observed n-gram relative frequency when available; otherwise it multiplies the shorter-history score by a fixed penalty, commonly 0.4. Its output is generally an unnormalized score, not a conditional probability distribution. Do not insert raw Stupid Backoff scores into standard probability-based perplexity and call the result a normalized-model perplexity.

\"S(w\\mid

Here h′ drops the oldest history word; α is the backoff penalty. At the unigram base, use C(w)/N. The letter α here is a penalty, not the forward-HMM variable introduced later.

Question. A bigram scorer has complete vocabulary {a,b}. After h, C(h,a)=2 and C(h,b)=0. The unigram probabilities are P(a)=P(b)=0.5. Apply Stupid Backoff with penalty 0.4 and show whether the scores form a normalized row. [Authored worked example]

Answer

Given

C(h)=2. The a bigram is observed; the b bigram is unseen. Unigram probabilities are supplied.

To find

S(a|h), S(b|h), their sum, and whether they are probabilities.

Formula and why it applies

Use relative frequency for an observed pair and α times the unigram base for an unseen pair. The method does not discount a’s observed score in this example.

Solution

Step 1. Score the observed pair
\"S(a\\mid
Step 2. Back off for the unseen pair
\"S(b\\mid
Step 3. Add the complete row
\"1+0.2=1.2\\ne1.\"
Step 4. Interpret the failure of normalization

The scores can rank a above b, but the raw row cannot be called a probability distribution. Merely labelling the method “smoothing” does not change this.

Final answer. Scores are 1 and 0.2; their sum is 1.2. They are useful ranking scores under this rule, not normalized probabilities.
MethodOperationNormalization
Add-one/add-kAdjust all allowed outcome counts and the totalNormalized with a fixed vocabulary
InterpolationMix normalized distributions using nonnegative weights summing to 1Normalized
Normalized backoffDiscount seen events and distribute remaining mass over unseen eventsNormalized when its factors are correctly computed
Stupid BackoffKeep seen relative frequencies; penalize shorter-history scores for unseen n-gramsGenerally not normalized

Scope: Unit 3 slide 147 lists Stupid Backoff and n-gram interpolation; the class calculations emphasize Laplace. These short worked extensions explain the listed methods without claiming extra official exam weight.

"}]},{"id":"f-deep-u4-forward-cky","title":"Faculty topic: CNF and CYK / CKY parsing","unit":4,"mode":"tee","order":100,"scope":"Unit 4 PDF p105 names CYK. The supplied papers ask top-down / bottom-up CFG trees, not CKY charts. These traces explain the taught topic.","sections":[{"id":"cnf","title":"Prepare a grammar for CYK / CKY","html":"

Faculty topic; not an observed numerical PYQ. Unit 4 PDF p105 names CYK. The supplied papers ask top-down / bottom-up CFG trees, not CKY charts. These traces explain the taught topic.

CYK and CKY are two names for the same dynamic-programming parsing family. A chart stores which grammatical categories can generate each contiguous span of the sentence. Small spans are solved first and reused to build larger spans. Basic CKY uses a context-free grammar in Chomsky normal form (CNF): a production has either two nonterminals on its right, A→B C, or one terminal, A→word. Empty-string handling is a separate special case; the examples here have no empty production.

RuleBasic CKY statusReason
S → NP VPAllowedTwo nonterminals
N → catAllowedOne terminal
NP → Det Adj NConvert firstThree nonterminals
NP → NEliminate or handle unary closureOne nonterminal, not a lexical terminal
VP → eats NPConvert firstMixed terminal and nonterminal

Question. Convert NP→Det Adj N, NP→N, N→cat and VP→eats NP into basic CNF-compatible productions. Preserve their possible word yields. [Authored worked example]

Answer

Given

The symbols NP,Det,Adj,N,VP are nonterminals; cat and eats are terminals. Other needed lexical rules remain as supplied.

To find

Equivalent binary or terminal productions for these cases.

Formula and why it applies

Introduce helper categories for a long right-hand side or a terminal mixed with nonterminals. Eliminate a unary nonterminal rule by inheriting the reachable non-unit productions.

Solution

Step 1. Binarize the three-part NP

Introduce X: NP→Det X and X→Adj N. X is a helper, not a word; the terminal order remains Det, Adj, N.

Step 2. Remove the unit production

NP→N and N→cat allow NP to derive cat. Add NP→cat, retain N→cat where needed, and remove NP→N. With more rules reachable through unit chains, inherit every corresponding non-unit rule, not only the first one.

Step 3. Replace the mixed terminal

Introduce Vlex→eats and replace VP→eats NP by VP→Vlex NP.

Step 4. Read the converted forms

NP→Det X | cat; X→Adj N; N→cat; VP→Vlex NP; Vlex→eats. All displayed productions are binary nonterminal or single-terminal rules.

Final answer. The converted rules preserve the original word sequences while satisfying the displayed basic CKY forms. Helper nodes change tree shape; they can be removed when presenting the original grammar’s tree.

General CNF conversion may also require eliminating nullable productions and useless symbols while preserving the intended language. Do not claim that simply splitting long rules handles every arbitrary CFG. An Earley parser instead supports general CFG rules using prediction, scanning and completion; it combines top-down expectations with bottom-up evidence. LR and operator-precedence parsers are other methods named in the faculty list, with grammar restrictions of their own.

"},{"id":"cky","title":"CYK / CKY: complete charts, backpointers and ambiguity","html":"

Faculty topic; not an observed numerical PYQ. Unit 4 PDF p105 names CYK. The supplied papers ask top-down / bottom-up CFG trees, not CKY charts. These traces explain the taught topic.

Number the words 1 to n. Let T[i,j] contain all categories that derive the span from word i through word j, inclusive. Start with one-word cells from lexical rules. For every longer span try every split k between i and j: if B belongs to T[i,k], C belongs to T[k+1,j], and A→B C is a rule, put A in T[i,j]. Store the rule and split as backpointers when a tree is required.

\"A\\in

Accept precisely when S is in T[1,n]. Empty cells are meaningful: no category in this grammar derives that span. Basic recognition takes O(n³|G|) time using the straightforward rule-loop analysis and O(n²|N|) category storage; packed backpointers can preserve alternative parses without listing all trees immediately.

Question. Use CKY to parse “the cat eats a fish” with S→NP VP, NP→Det N, VP→V NP; Det→the|a; N→cat|fish; V→eats. Show the complete span chart and reconstruct a tree. [Authored worked example]

Answer

Given

Words 1=the, 2=cat, 3=eats, 4=a, 5=fish. The grammar is already in the basic CNF form. ∅ denotes an empty cell.

To find

All chart cells, acceptance, and a parse tree.

Formula and why it applies

Fill lexical cells, then spans of length 2,3,4,5. For each span try all split points and combine only categories permitted by a binary grammar rule.

Solution

Step 1. Fill the five one-word cells

T[1,1]={Det}; T[2,2]={N}; T[3,3]={V}; T[4,4]={Det}; T[5,5]={N}.

Step 2. Fill length-two spans

T[1,2]={NP} because Det+N matches NP→Det N. T[2,3]=∅ (N V has no rule). T[3,4]=∅ (V Det has no rule). T[4,5]={NP} using Det+N.

Step 3. Fill length-three spans

T[1,3]=∅: neither Det+(N eats) nor NP+V matches a rule. T[2,4]=∅. For T[3,5], split after word 3: V from [3,3] plus NP from [4,5] gives VP. Thus T[3,5]={VP}; the other split has an empty left child.

Step 4. Fill length-four spans

T[1,4]=∅ because every split has an empty child or an unmatched category pair. T[2,5]=∅: the only wholly nonempty possibility is N+VP, and no rule produces a parent from that pair.

Step 5. Fill the complete span

For T[1,5], k=1 has an empty right cell; k=2 gives NP from [1,2] and VP from [3,5], hence S. k=3 and k=4 have empty left cells. Therefore T[1,5]={S}.

Step 6. Copy the completed upper-triangular chart
Start i / end j1: the2: cat3: eats4: a5: fish
1DetNP∅∅S
2—N∅∅∅
3——V∅VP
4———DetNP
5————N

A dash is outside the chart (i>j); ∅ is an evaluated but empty valid span. Read the chart by increasing span length, not left-to-right across a row.

Step 7. Follow the stored backpointers

S[1,5]→NP[1,2] VP[3,5]; NP[1,2]→Det[1,1] N[2,2]; VP[3,5]→V[3,3] NP[4,5]; NP[4,5]→Det[4,4] N[5,5].

S\n├── NP\n│   ├── Det: the\n│   └── N: cat\n└── VP\n    ├── V: eats\n    └── NP\n        ├── Det: a\n        └── N: fish
Final answer. Accepted because S∈T[1,5]. The reconstructed tree’s leaves are exactly the five input words in order.

Question. A grammar has S→A A, A→A A and A→a. Use CKY for “a a a” and explain how two parse trees are preserved. [Authored worked example]

Answer

Given

Three identical terminal tokens. S is the start symbol. No unit or empty rules are present.

To find

Chart categories and both successful split choices for the root.

Formula and why it applies

A chart cell is a set of possible categories. For each category keep all successful backpointers when the task requires every parse.

Solution

Step 1. Initialize the diagonal

Each of T[1,1], T[2,2], T[3,3] is {A}, from A→a.

Step 2. Fill two-word spans

T[1,2] and T[2,3] each contain {A,S}: A+A can use A→A A or S→A A.

Step 3. Try both root splits

k=1: A[1,1] + A[2,3] permits S. k=2: A[1,2] + A[3,3] also permits S. The complete cell is {A,S}, but S has two distinct backpointer alternatives.

Step 4. Draw both bracketed trees

[S [A a] [A [A a] [A a]]] and [S [A [A a] [A a]] [A a]]. Both yield a a a; the constituent grouping differs.

Final answer. Accepted, with two S-rooted parses. A category appearing once in a set does not mean there is only one derivation.

Question. With S→NP VP, NP→Det N, VP→V NP; Det→the|a; N→cat|fish; V→eats, test “the eats cat” using CKY. [Authored worked example]

Answer

Given

Tokens are the/eats/cat, with no punctuation. Use exactly the supplied grammar.

To find

Whether the grammar accepts the whole input.

Formula and why it applies

A parse exists only if the complete chart cell contains the start symbol.

Solution

Step 1. Initialize the diagonal

T[1,1]={Det}; T[2,2]={V}; T[3,3]={N}.

Step 2. Try length-two combinations

Det+V matches no binary rule; V+N also matches no rule (VP requires V+NP). Both two-word cells are empty.

Step 3. Try the two full-span splits

Each split has an empty child, so T[1,3]=∅ and S is absent.

Final answer. Rejected by this grammar. Do not invent NP→N to force acceptance. Grammar rejection and a universal judgment about every possible English grammar are different claims.

Yaad rakho: Diagonal par words ke tags; phir chhote spans jodo. Har split dekho. Full sentence cell mein S aaya toh parse possible.

"}]},{"id":"tee-formula-index","title":"TEE · additional formulas","unit":0,"sections":[{"id":"all-tee","title":"TEE formula sheet: vectors and applications","html":"

These formulas extend the M2 sheet to the complete TEE syllabus. Each coordinate, token-count convention and log base must be defined before calculation. Worked-example chapters follow this sheet.

Type–token ratio

\"TTR=\\frac{\\#\\text{types}}{\\#\\text{tokens}}\"

Distinct vocabulary types divided by all token occurrences. Often expressed as a percentage.

Normalized term frequency

\"TF(t,d)=\\frac{C(t,d)}{\\sum_w

The slides use count of the term divided by the document token total after the specified preprocessing.

Inverse document frequency

\"IDF(t)=\\log_{10}\\frac{D}{df(t)}\"

D is the number of documents; df counts documents containing the term at least once. This is the course slide convention.

TF–IDF

\"TF\\!\\operatorname{-}\\!IDF(t,d)=TF(t,d)IDF(t)\"

Multiply the local frequency weight by the global rarity weight.

Dot product

\"a\\cdot

Multiply corresponding coordinates, then add. Vectors need the same coordinate order.

Vector length

\"\\|a\\|_2=\\sqrt{\\sum_i

Square each coordinate, add, then take the square root.

Cosine similarity

\"\\cos(a,b)=\\frac{\\sum_i

Compare direction; undefined for a zero-length vector.

Jaccard similarity

\"J(A,B)=\\frac{|A\\cap

Set membership, not repeated word counts. Jaccard distance = 1 − J.

Manhattan distance

\"d_1(a,b)=\\sum_i|a_i-b_i|\"

Add coordinate-wise absolute differences.

Euclidean distance

\"d_2(a,b)=\\sqrt{\\sum_i(a_i-b_i)^2}\"

Square differences, add, square root.

Minkowski distance

\"d_p(a,b)=\\left(\\sum_i|a_i-b_i|^p\\right)^{1/p}\"

p≥1; Manhattan p=1, Euclidean p=2.

Mean sentence embedding

\"v_{sentence}=\\frac1n\\sum_{i=1}^n

Add word vectors coordinate by coordinate; divide each coordinate by n. This baseline loses word order.

GloVe objective

\"J=\\sum_{i,j}f(X_{ij})(w_i^T\\widetilde

X is a co-occurrence count. Compute the residual, square it and apply the weighting function. Use observed positive counts.

Precision and recall

\"P=\\frac{TP}{TP+FP},\\qquad

Precision checks predicted positives; recall checks actual positives.

F1 score

\"F_1=\\frac{2PR}{P+R}=\\frac{2TP}{2TP+FP+FN}\"

Harmonic mean of precision and recall.

Accuracy

\"A=\\frac{TP+TN}{TP+TN+FP+FN}\"

All correct predictions divided by all predictions.

Noisy-channel correction

\"\\hat

Rank intended word w using observed typo x; likelihood × prior.

Unit-cost edit distance

\"D(i,j)=\\min\\{D(i-1,j)+1,D(i,j-1)+1,D(i-1,j-1)+[a_i\\ne

Initialise first row/column by insertion/deletion counts; fill all cells using min.

"}],"scope":"TEE · used together with the M2 formula sheet"}],"papers":[{"id":"final-2025","title":"Solved final paper · December 2025","unit":"PYQ","scope":"Actual final examination · 2 December 2025 · 100 marks · 3 hours","sections":[{"id":"final-2025-q1a","title":"Q1A · NLU and NLG","html":"
Question. Differentiate between Natural Language Understanding (NLU) and Natural Language Generation (NLG) in NLP with suitable examples. [5 marks]

Answer

Natural Language Understanding (NLU) interprets language to identify its meaning, intent and relevant information. Natural Language Generation (NLG) produces language that expresses information in a useful form.

BasisNLUNLG
DirectionLanguage → meaning or structured informationMeaning or structured information → language
Typical inputA user utterance, document or sentenceFacts, a database record, a prompt or a representation
Typical outputIntent, entities, sentiment or a semantic representationA sentence, explanation, summary or dialogue response
Main difficultyResolving ambiguity and contextChoosing accurate content and producing coherent, fluent wording

Example: a weather assistant

User: “Will it rain in Mumbai tomorrow?” NLU recognises a weather-query intent, the location Mumbai and the date tomorrow. After a weather service supplies the forecast, NLG produces “Rain is likely in Mumbai tomorrow; carry an umbrella.”

User language → NLU: identify request → Retrieve forecast → NLG: express answer

The two functions often work together in a chatbot. Understanding the request and wording the reply are separate tasks; fluent generation alone does not prove correct understanding.

Yaad rakho: NLU mein meaning nikalo; NLG mein meaning ko sentence banao.

","m2":false,"units":"1","marks":5},{"id":"final-2025-q1b","title":"Q1B · Chunking and chinking","html":"
Question. Explain the concepts of chunking and chinking in Natural Language Processing (NLP). Discuss how Part-of-Speech (POS) tagging supports these processes, and illustrate your explanation with a suitable example. [5 marks]

Answer

Chunking groups adjacent tagged words into shallow phrases, such as noun phrases, without building a complete sentence parse. Chinking removes a specified sequence of tags or words from a chunk, splitting a larger candidate chunk when required.

POS tags provide the pattern

A POS tag describes a grammatical category: DT = determiner, JJ = adjective, NN = singular noun, VBD = past-tense verb and IN = preposition. A noun-phrase rule can be written as NP: {<DT>?<JJ>*<NN>+}: an optional determiner, zero or more adjectives, followed by one or more nouns.

Worked example

WordThesmallcatsatonthemat
TagDTJJNNVBDINDTNN

Applying the NP rule gives [The small cat] sat on [the mat]. The first chunk contains DT–JJ–NN; the second contains DT–NN. “sat on” is not included because it does not match the noun-phrase rule.

For chinking, begin with one broad chunk: [The small cat sat on the mat]. Remove the VBD and IN sequence “sat on”, for example with }<VBD|IN>+{. The result is [The small cat] sat on [the mat].

POS tagging therefore supplies grammatical labels; chunking uses labels to select phrases, while chinking uses labels to exclude unwanted material. Incorrect tags can lead to incorrect chunks.

Yaad rakho: Chunking = phrase ko jodo; chinking = beech ka unwanted part nikalo.

","m2":true,"units":"4","marks":5},{"id":"final-2025-q1c","title":"Q1C · Hyponymy, hypernymy and WSD","html":"
Question. Define hyponymy and hypernymy with suitable linguistic examples. How can hypernymy be applied in Word Sense Disambiguation (WSD) to resolve the correct meaning of polysemous words? Provide an example? [5 marks]

Answer

Hyponymy is the “is-a-kind-of” relation from a specific concept to a broader concept. Hypernymy is the inverse relation from the broader concept to the specific one. “Rose” is a hyponym of “flower”; “flower” is a hypernym of “rose”.

rose / tulip → flower → plant

The arrow points towards a broader category. Hyponymy is not a part–whole relation: a wheel is part of a car, but it is not a kind of car.

Using the hierarchy for WSD

WSD selects the intended sense of a word from its context. Obtain candidate senses from a lexical resource such as WordNet, inspect their hypernym chains, and favour the chain most compatible with nearby words.

  1. Sentence: “The mouse clicked the icon.”
  2. Candidate 1: mouse → rodent → mammal → animal. Candidate 2: mouse → pointing device → computer peripheral.
  3. The context words “clicked” and “icon” belong to the computer-use domain. They support the device hierarchy, rather than the animal hierarchy.
  4. Select the computer-pointing-device sense. In “The mouse ate cheese,” the animal hierarchy is more compatible.

Hypernym evidence supplies semantic generalisation; it is a useful signal rather than a guarantee, because short or unusual contexts may still be ambiguous.

Yaad rakho: Hyponym specific hai; hypernym uska bada category hai.

","m2":true,"units":"5","marks":5},{"id":"final-2025-q1d","title":"Q1D · Pragmatics and discourse","html":"
Question. Compare pragmatic and discourse analysis with the help of appropriate examples. [5 marks]

Answer

Pragmatic analysis interprets an utterance using the situation, speaker intention, shared knowledge and conversational expectations. Discourse analysis studies how multiple utterances form a connected text or conversation.

BasisPragmatic analysisDiscourse analysis
FocusIntended meaning in contextRelationships and coherence across utterances
Useful informationSpeaker, listener, place, time and common knowledgePrevious sentences, entities, topics and rhetorical relations
Typical tasksInterpret requests, implicatures and deixisResolve references and analyse cohesion, coherence and discourse structure
Example“It is cold here” may be a request to close a window“Riya bought a book. She read it.” links She to Riya and it to book

In “Can you pass the salt?”, literal syntax asks about ability; pragmatic interpretation usually identifies a polite request. In “The road flooded. Therefore, the school closed,” discourse analysis identifies a cause–result link between the sentences.

The levels interact: resolving “she” across a conversation may require both earlier text and real-world context. They are complementary rather than mutually exclusive.

Yaad rakho: Pragmatics: bolne ka intention. Discourse: sentences ka connection.

","m2":false,"units":"6","marks":5},{"id":"final-2025-q2a","title":"Q2A · Morpheme, lexeme and NER","html":"
Question. i. Differentiate between Morpheme and Lexeme with examples.
ii. Discuss the significance of NER in three different NLP applications. [10 marks]

Answer

i. Morpheme and lexeme

A morpheme is the smallest linguistic unit that contributes meaning or grammatical function. A lexeme is an abstract vocabulary item representing a family of inflected word forms.

BasisMorphemeLexeme
UnitA meaningful component of a wordAn abstract dictionary vocabulary item
Examplesun-, happy, -ness; cat and plural -sRUN covers run, runs, ran and running
IndependenceFree morphemes can stand alone; bound morphemes cannotA lexeme is realised through one or more word forms
NLP relevanceMorphological segmentation, stemming and analysisLemmatization and lexical meaning

“Cats” contains two morphemes: cat + plural -s. “cat” and “cats” are forms of the lexeme CAT. “Unhappiness” can be segmented into un + happy + ness, although its spelling changes at the boundary. A morpheme and a syllable are different: syllables concern pronunciation, while morphemes concern meaning or grammar.

ii. Named Entity Recognition in applications

NER locates spans that name entities and assigns types such as PERSON, ORGANIZATION, LOCATION, DATE or domain-specific categories.

ApplicationEntities and exampleWhy NER matters
News information extraction“Asha joined Infosys in Pune.” → Asha/PERSON, Infosys/ORG, Pune/LOCATIONCreates searchable people–organisation–location records and supports later relation extraction
Healthcare document processingFind disease, drug and dosage spans in a clinical noteSupports record indexing and medication extraction; specialist labels and validation are required
Customer support and commerceExtract product names, dates and order identifiers from “My Pixel arrived damaged on Monday.”Routes complaints, links the correct product and fills structured support fields

NER answers “Which entity spans occur here, and what type are they?” It does not by itself establish every relation between those entities or resolve all names to unique real-world identities.

Yaad rakho: Morpheme = word ka meaningful piece; lexeme = word family. NER = naam dhoondho aur type do.

","m2":false,"units":"1,4","marks":10},{"id":"final-2025-q2b","title":"Q2B · Laplace-smoothed bigram probability","html":"
Question. Consider the following training corpus:
1. artists paint murals in city
2. musicians play songs in city
3. painters paint murals on walls
4. signs are near city
Using a bigram language model with Laplace smoothing, evaluate the probability of the test sentence: “artists paint murals near city”. Assume the <s> and </s> for each sentence. [10 marks]

Answer

Given

Test: artists paint murals near city. Add one start token <s> and one end token </s> to every training and test sentence.

To find

The Laplace-smoothed bigram probability of the complete test sentence, including its end token.

Formula: which one and why

\"P_{+1}(w_i\\mid

C(previous,current) counts that adjacent pair in the training corpus. C(previous) counts its occurrences as a history. V is the vocabulary size. Add-one smoothing is necessary because an unseen pair must not force the whole sentence probability to zero.

\"P(w_1,\\ldots,w_n,\\langle/s\\rangle\\mid\\langle

Solution

Step 1. Fix the token and vocabulary convention

Lowercase words and remove the final punctuation. The supplied course key counts both boundary tokens in V. Following that convention: are, artists, city, in, murals, musicians, near, on, paint, painters, play, signs, songs, walls, <s>, </s> gives V = 16. The start marker supplies history; it is not an extra probability factor before the first word.

Step 2. List every required transition

The complete test chain is <s> → artists → paint → murals → near → city → </s>.

Step 3. Count first, then add one

Required bigramHistory countPair countSmoothed probability
<s> → artists41(1+1)/(4+16) = 1/10
artists → paint11(1+1)/(1+16) = 2/17
paint → murals22(2+1)/(2+16) = 1/6
murals → near20(0+1)/(2+16) = 1/18
near → city11(1+1)/(1+16) = 2/17
city → </s>33(3+1)/(3+16) = 4/19

Count only adjacent words, never words that merely appear in the same sentence. The unseen pairs have count zero, so their numerator becomes one.

Step 4. Multiply all six probabilities

\"P=\\frac{2}{20}\\times\\frac{2}{17}\\times\\frac{3}{18}\\times\\frac{1}{18}\\times\\frac{2}{17}\\times\\frac{4}{19}\"

Start with subtotal 1. At each row, multiply the previous subtotal by exactly one next factor. Simplify the fraction after multiplication, but do not round it.

Factor being includedPrevious subtotal × next factor = new subtotal
1. <s> → artists\"1\\times\\frac{1}{10}=\\frac{1}{10}\"
2. artists → paint\"\\frac{1}{10}\\times\\frac{2}{17}=\\frac{1}{85}\"
3. paint → murals\"\\frac{1}{85}\\times\\frac{1}{6}=\\frac{1}{510}\"
4. murals → near\"\\frac{1}{510}\\times\\frac{1}{18}=\\frac{1}{9180}\"
5. near → city\"\\frac{1}{9180}\\times\\frac{2}{17}=\\frac{1}{78030}\"
6. city → </s>\"\\frac{1}{78030}\\times\\frac{4}{19}=\\frac{2}{741285}\"

The last subtotal is the complete product. Only now convert the fraction to a decimal.

\"P=\\frac{2}{741285}\\approx

Step 5. Interpret the result

This is a joint sentence probability, so a small value is normal. Do not add transition probabilities; each word must occur in sequence, so the chain multiplies them.

Final answer

P ≈ 2.69801763 × 10-6 with the stated course vocabulary convention.

Yaad rakho: Pair count mein 1 jodo, previous-word count mein V; phir saare fractions multiply karo.

Convention note: Some implementations exclude <s> from the predicted vocabulary. Their V is one smaller and their numerical probability differs. In an exam, state V explicitly and follow the convention specified in the question or class solution.

","m2":true,"units":"3","marks":10},{"id":"final-2025-q3a","title":"Q3A · Ambiguity and bottom-up parsing","html":"
Question. i. Explain the various types of ambiguities encountered in Natural Language Processing (NLP). Illustrate each type with suitable examples.
ii. Compare top-down and bottom-up parsing and apply bottom-up syntax parsing approach to the sentence “A cat chased the mouse” and draw a parse tree using the grammar rules:
S → NP VP; NP → Pronoun | Proper-Noun | Det Nominal; Nominal → Noun | Nominal Noun | Nominal PP; VP → Verb | Verb NP | VP PP; PP → Prep NP.
Lexicon: Det → {A, the}; Noun → {cat, mouse}; Verb → {chased}. [10 marks]

Answer

i. Types of ambiguity

Ambiguity exists when the same language expression permits more than one interpretation. The correct interpretation usually requires grammatical, semantic or contextual evidence.

TypeMeaningExample and readings
LexicalOne word has multiple meanings“She visited the bank”: financial institution or riverbank
SyntacticThe sentence permits different structural attachments“I saw the man with a telescope”: I used the telescope, or the man had it
Semantic / scopeA structured sentence permits different interpretations“Every student read a book”: one shared book, or possibly a different book per student
ReferentialA referring expression has more than one candidate referent“Asha told Riya that she won”: she could refer to Asha or Riya
PragmaticIntention depends on the situation“Can you open the door?” can ask about ability or request an action

ii. Parsing approaches

AspectTop-downBottom-up
StartThe start symbol SThe input words
OperationExpand a nonterminal using grammar productionsCombine words and constituents using productions in reverse
SuccessLeaves match the complete inputAll input is reduced to S
Possible difficultyNaive recursive descent can loop on left recursion or need backtrackingA naive shift-reduce parser may face shift/reduce conflicts

Given

The five tokens are A / cat / chased / the / mouse. The supplied grammar permits determiner + nominal noun phrases and a verb + noun phrase verb phrase.

To find

A bottom-up sequence of reductions and a parse tree whose root is S and whose leaves preserve the input order.

Rule: which one and why

A production A → B C permits B C to be reduced to A. Bottom-up parsing begins with the words; lexical rules assign their categories, and phrase rules combine those categories.

Step 1. Assign lexical categories

A → Det; cat → Noun; chased → Verb; the → Det; mouse → Noun.

Step 2. Build the first noun phrase

cat is Noun. Reduce Noun to Nominal using Nominal → Noun. Then reduce Det Nominal, representing “A cat”, to NP.

Step 3. Build the object noun phrase

mouse is Noun. Reduce it to Nominal. Reduce “the” + “mouse”, now Det Nominal, to NP.

Step 4. Build the verb phrase

“chased” is Verb and “the mouse” is NP. Reduce Verb NP to VP.

Step 5. Build the sentence

The remaining constituents are NP (“A cat”) and VP (“chased the mouse”). Apply S → NP VP in reverse to obtain S. Every input token is used exactly once.

SNPVPDetNominalVerbNPNounDetNominalNounAcatchasedthemouse
Bottom-up parse tree for A cat chased the mouse.

Final answer: [S [NP [Det A] [Nominal [Noun cat]]] [VP [Verb chased] [NP [Det the] [Nominal [Noun mouse]]]]].

Yaad rakho: Bottom-up: words se phrases, phrases se S. Top-down: S se words.

","m2":false,"units":"1,4","marks":10},{"id":"final-2025-q3b","title":"Q3B · Co-occurrence matrix and cosine","html":"
Question. Consider the given paragraph: The teacher wrote on the board. The students listened to the teacher. The board was full of notes. The student asked a question. The teacher answered from the book.
Target words: teacher, student, board, book.
Context words: wrote, listened, notes, question.
i. Construct a co-occurrence matrix for the target and context words. Take the window size as one whole sentence.
ii. Evaluate the similarity between the vectors of the target words “teacher-student” and “teacher-board” using the cosine similarity.
iii. Plot the target word vectors by considering only two dimensions: “wrote” and “notes”. [10 marks]

Answer

Given

Five sentences, four target rows and four context columns. Treat students and student as the same target lemma, and count each target–context pair once when both occur in a sentence.

To find

The 4 × 4 sentence-window co-occurrence matrix, two cosine similarities and the requested 2-D plot.

Formula: which one and why

\"M_{ij}=\\sum_{s=1}^{5}\\mathbf{1}[\\text{target}_i\\text{

The indicator is 1 when both words appear in that sentence and 0 otherwise. Each matrix row is the target word’s vector: its list of four context counts.

\"\\cos(\\mathbf

The dot product multiplies corresponding coordinates and adds them. The length is the square root of the sum of squared coordinates. Cosine compares directions while allowing different vector lengths.

Solution

Step 1. Record the contribution of each sentence

SentenceTargets presentRelevant contextCounts added
1teacher, boardwroteteacher–wrote +1; board–wrote +1
2student, teacherlistenedstudent–listened +1; teacher–listened +1
3boardnotesboard–notes +1
4studentquestionstudent–question +1
5teacher, booknoneNo requested context count

Step 2. Construct every requested column

Targetwrotelistenednotesquestion
teacher1100
student0101
board1010
book0000

Step 3. Calculate teacher–student

\"\\mathbf
\"\\|\\mathbf
\"\\|\\mathbf
\"\\cos(\\mathbf

Step 4. Calculate teacher–board

\"\\mathbf
\"\\|\\mathbf
\"\\cos(\\mathbf

Step 5. Select only the two requested plot coordinates

Wordx = wrotey = notes
teacher10
student00
board11
book00
001122teacherstudent, bookboardwrotenotes
The plot uses only the requested two coordinates; cosine uses all context columns.

Final answer

Both requested cosine similarities are 0.5. Student and book overlap at the origin of the 2-D projection, but their complete four-dimensional vectors differ. The book row is zero, so a cosine involving book would be undefined without an explicit convention.

Yaad rakho: Har sentence mein target aur context saath? Ek count. Cosine ke liye poori row lo, plot ke liye sirf do columns.

","m2":false,"units":"5","marks":10},{"id":"final-2025-q4a","title":"Q4A · Three preprocessing techniques","html":"
Question. Explain any three basic text preprocessing techniques used in Natural Language Processing (NLP). Discuss the purpose of each technique and illustrate them with suitable examples. [10 marks]

Answer

Text preprocessing transforms raw text into a representation suitable for a chosen NLP task. The best steps depend on the task; removing information indiscriminately can reduce performance.

1. Tokenization

Tokenization divides text into units called tokens, such as words, subwords or sentences. These units are required for counting words and building many model inputs. “The cats are sleeping.” can become [The, cats, are, sleeping, .] under a punctuation-preserving word tokenizer. A different tokenizer may split contractions or punctuation differently, so the convention must be consistent during training and prediction.

2. Lowercasing

Lowercasing maps uppercase letters to lowercase so that variants such as “Book”, “BOOK” and “book” can share one feature. This reduces vocabulary duplication in a case-insensitive search system. However, “US” and “us” are different, and capitals help NER recognise names. Therefore lowercasing should be chosen according to the application.

3. Lemmatization

Lemmatization maps an inflected word to its dictionary base form, often using its POS and context. Examples: cats → cat; running/VBG → run; better/JJR → good. It allows a search for “run” to match relevant inflections. A stemmer instead uses simpler truncation rules and can produce a non-word stem; lemmatization is linguistically informed and can need more processing.

Combined example

StageRepresentation
RawThe cats are running.
TokenizedThe | cats | are | running | .
Lowercasedthe | cats | are | running | .
Lemmatized with correct POSthe | cat | be | run | .

These steps do not automatically remove stop words or punctuation. Those are separate choices. For sentiment analysis, preserve negation such as “not”; for syntax parsing, function words and word order are often essential.

Yaad rakho: Preprocess task dekh ke karo: har word ko blindly delete mat karo.

","m2":false,"units":"2","marks":10},{"id":"final-2025-q4b","title":"Q4B · HMM emission, transition and sequence","html":"
Question. Consider the following POS tagged corpus. Apply Hidden Markov Model (HMM) approach to:
1. Cat/Noun chase/Verb mice/Noun
2. The/Det dog/Noun runs/Verb
3. A/Det cat/Noun eats/Verb cheese/Noun
i. Construct the emission probability matrix.
ii. Construct the transmission probability matrix.
iii. Calculate the probability of sequence of events: The/Det cat/Noun eats/Verb mice/Noun.
Note: Include start and end characters for all the sentences. [10 marks]

Answer

Given

SentenceTagged sequence
1cat/Noun chase/Verb mice/Noun
2the/Det dog/Noun runs/Verb
3a/Det cat/Noun eats/Verb cheese/Noun

Test words: the cat eats mice. Given tag sequence: Det → Noun → Verb → Noun. Include Start and End for all sentences. Lowercase the corpus so sentence-initial Cat and cat are one word.

To find

Emission probabilities, transition probabilities and the joint probability of the specified tag-and-word sequence.

Formula: which one and why

\"P(w\\mid

An emission probability asks: among tokens carrying this tag t, what fraction are the word w?

\"P(t_j\\mid

A transition probability asks: among transitions leaving tag tᵢ, what fraction go to tⱼ? Include End in the outgoing counts. The paper says “transmission”; the standard HMM term is transition.

\"P(\\mathbf

Solution

Step 1. Count tokens for each tag

Det has 2 tokens; Noun has 5 tokens; Verb has 3 tokens. The repeated noun cat occurs twice with Noun; every other displayed noun occurs once. The three verb types each occur once.

Step 2. Build the emission matrix

Tag / wordacatchasecheesedogeatsmicerunsthe
Det1/200000001/2
Noun02/501/51/501/500
Verb001/3001/301/30

Each row sums to one. A zero means no occurrence under that tag in this unsmoothed training corpus; no add-one smoothing was requested.

Step 3. Write the three tag chains

Start → Noun → Verb → Noun → End
Start → Det → Noun → Verb → End
Start → Det → Noun → Verb → Noun → End

Step 4. Count outgoing transitions

FromTo DetTo NounTo VerbTo EndOutgoing total
Start21003
Det02002
Noun00325
Verb02013

Step 5. Divide each row by its outgoing total

From / ToDetNounVerbEnd
Start2/31/300
Det0100
Noun003/52/5
Verb02/301/3

End is terminal, so it has no estimated outgoing row. The four nonterminal rows each sum to one.

Step 6. Choose the factors for the given sequence

Event in orderProbability
Start → Det2/3
Det emits the1/2
Det → Noun1
Noun emits cat2/5
Noun → Verb3/5
Verb emits eats1/3
Verb → Noun2/3
Noun emits mice1/5
Noun → End2/5

Step 7. Multiply, including the end transition

\"P=\\frac23\\times\\frac12\\times1\\times\\frac25\\times\\frac35\\times\\frac13\\times\\frac23\\times\\frac15\\times\\frac25\"

Start with subtotal 1. At each row, multiply the previous subtotal by exactly one next factor. Simplify the fraction after multiplication, but do not round it.

Factor being includedPrevious subtotal × next factor = new subtotal
1. Start → Det\"1\\times\\frac{2}{3}=\\frac{2}{3}\"
2. Det emits the\"\\frac{2}{3}\\times\\frac{1}{2}=\\frac{1}{3}\"
3. Det → Noun\"\\frac{1}{3}\\times1=\\frac{1}{3}\"
4. Noun emits cat\"\\frac{1}{3}\\times\\frac{2}{5}=\\frac{2}{15}\"
5. Noun → Verb\"\\frac{2}{15}\\times\\frac{3}{5}=\\frac{2}{25}\"
6. Verb emits eats\"\\frac{2}{25}\\times\\frac{1}{3}=\\frac{2}{75}\"
7. Verb → Noun\"\\frac{2}{75}\\times\\frac{2}{3}=\\frac{4}{225}\"
8. Noun emits mice\"\\frac{4}{225}\\times\\frac{1}{5}=\\frac{4}{1125}\"
9. Noun → End\"\\frac{4}{1125}\\times\\frac{2}{5}=\\frac{8}{5625}\"

The last subtotal is the complete product. Only now convert the fraction to a decimal.

\"P=\\frac8{5625}\\approx0.0014222222\"

Final answer

Joint sequence probability = 8/5625 ≈ 0.0014222222. This is the probability of the specified words and tags together. It is not the probability of the word sequence summed over every possible tag path.

Yaad rakho: Tag se tag = transition; tag se word = emission. Dono alternate multiply, end ko mat bhoolo.

Boundary reminder: The initial factor is P(Det | Start) = 2/3, based on three sentence starts, and the final factor is P(End | Noun) = 2/5. The overall fraction of Det tokens, 2/10, is not the start probability. Using it and omitting End, as in the handwritten key’s last line, computes a different quantity.

","m2":true,"units":"4","marks":10},{"id":"final-2025-q5a","title":"Q5A · TF–IDF matrix","html":"
Question. For the following two sentences:
Sentence A: “The student read a science book in the library”
Sentence B: “The teacher read a history book in the classroom”
1. Construct the TF-IDF matrix for these sentences after removing the stop words {The, a, in}.
2. Identify and comment on the important terms in each sentence based on their TF-IDF scores. [10 marks]

Answer

Given

Two documents, N = 2. Remove the listed stop words case-insensitively, so both “The” and “the” are removed. Use raw count TF and natural-log IDF, the convention used in the course key.

To find

The term-by-document TF–IDF matrix and the terms that distinguish each sentence.

Formula: which one and why

\"\\mathrm{TF}(t,d)=C(t,d),\\qquad
\"\\mathrm{TFIDF}(t,d)=\\mathrm{TF}(t,d)\\,\\mathrm{IDF}(t)\"

C(t,d) is the number of occurrences of term t in document d. df(t) is the number of documents containing t, not its total token count. N is the number of documents. IDF downweights terms shared by many documents.

Solution

Step 1. Clean and list the tokens

A: student, read, science, book, library.
B: teacher, read, history, book, classroom. Each retained token appears once in its sentence.

Step 2. Fix one common vocabulary order

student, read, science, book, library, teacher, history, classroom. Each row must refer to the same term in both documents.

Step 3. Count document frequency and calculate IDF

Term typedfIDF calculation
read, book2ln(2/2) = ln(1) = 0
Every other term1ln(2/1) = ln(2) ≈ 0.693147

A term absent from a document has TF = 0. A present term has TF = 1 here.

Step 4. Multiply TF by IDF

TermTF in ATF in BIDFTF–IDF ATF–IDF B
student100.6931470.6931470
read11000
science100.6931470.6931470
book11000
library100.6931470.6931470
teacher010.69314700.693147
history010.69314700.693147
classroom010.69314700.693147

Step 5. Interpret the weights

A is distinguished by student, science and library. B is distinguished by teacher, history and classroom. “read” and “book” have zero IDF in this tiny unsmoothed two-document collection because they appear in every document; that does not mean those words are meaningless in English.

Final answer

All six document-specific terms receive 0.693147 in their own document and 0 in the other. Shared read and book receive 0 in both.

Yaad rakho: TF: is document mein kitni baar? IDF: kitne documents mein common? Rare-across-documents term ko zyada weight.

If normalized TF is explicitly requested, divide each present-token count by five; each nonzero weight becomes 0.693147/5 = 0.1386294. Do not silently switch conventions.

","m2":false,"units":"2","marks":10},{"id":"final-2025-q5b","title":"Q5B · Naïve Bayes word-sense classification","html":"
Question. Consider the sentence: “The bat hit the ball hard on the cricket field.” Two possible senses of “bat”: Sense 1: Bat (Sports equipment – cricket/baseball); Sense 2: Bat (Nocturnal flying mammal).
Sentence IDContext wordsSense
1ball, cricket, score, stadiumSports
2night, cave, wings, flyAnimal
3player, run, match, batSports
4nocturnal, vampire, insectAnimal
5team, field, bowl, batSports
6dark, flying, cave, nightAnimal
a) Using the Naïve Bayes algorithm, classify the sense of “bat” in the given sentence. Apply Laplace Smoothing wherever applicable.
b) Discuss the significance of Laplace Smoothing in Naïve Bayes Classification. [10 marks]

Answer

Given

There are three Sports examples and three Animal examples. Treat training context lists as bag-of-words token counts. Exclude the ambiguous target “bat” from the test context. Use the fixed training vocabulary; ignore out-of-vocabulary test words hit and hard. The usable test features are ball, cricket and field.

To find

The posterior-score comparison for Sports and Animal, and the role of smoothing.

Formula: which one and why

\"\\mathrm{score}(s)=P(s)\\prod_{w\\in\\mathrm{context}}P(w\\mid
\"P(w\\mid

P(s) is the class prior, C(w,s) counts occurrences of context word w in class s, Nₛ is the total number of context tokens in that class and V is the shared vocabulary size. Conditional independence lets us multiply word probabilities. The common denominator P(context) is unnecessary when only comparing classes.

Solution

Step 1. Calculate the priors

\"P(S)=\\frac36=0.5,\\qquad

Step 2. Count class tokens and vocabulary

Sports has 4 + 4 + 4 = 12 tokens. Animal has 4 + 3 + 4 = 11 tokens. The shared training vocabulary has 20 distinct words: ball, cricket, score, stadium, player, run, match, bat, team, field, bowl, night, cave, wings, fly, nocturnal, vampire, insect, dark, flying. “bat”, “night” and “cave” repeat but count only once each towards V.

Step 3. Look up each usable context word

FeatureSports countAnimal countP(word | Sports)P(word | Animal)
ball10(1+1)/(12+20) = 2/32(0+1)/(11+20) = 1/31
cricket102/321/31
field102/321/31

Step 4. Multiply the Sports score

\"\\mathrm{score}(S)=0.5\\times\\frac2{32}\\times\\frac2{32}\\times\\frac2{32}=0.0001220703125\"

Step 5. Multiply the Animal score

\"\\mathrm{score}(A)=0.5\\times\\frac1{31}\\times\\frac1{31}\\times\\frac1{31}\\approx0.00001678359236\"

Step 6. Compare the scores

0.0001220703125 is greater than 0.00001678359236. Therefore select Sports. Optional normalization gives P(Sports | retained features) ≈ 0.87913; normalization is not needed for the decision.

Final answer

The intended sense is bat as sports equipment. Ball, cricket and field favour that sense.

Yaad rakho: Pehle class prior, phir har context-word likelihood multiply; bigger score wali sense choose karo.

Why Laplace smoothing matters

Without smoothing, all three test features have zero count in Animal. One zero factor makes the entire class score zero, even if other evidence could support that class. Add-one smoothing assigns nonzero probability to every word in the defined vocabulary and redistributes probability mass. It does not mean every unseen word outside the vocabulary can be inserted without changing the vocabulary model.

If following the supplied paper key exactly

The key also multiplies probabilities for hit and hard using the same denominators 32 and 31. That convention produces Sports = 1.19209290 × 10⁻⁷ and Animal ≈ 1.74647163 × 10⁻⁸, again selecting Sports. Its treatment of out-of-vocabulary words is not a normalized fixed-vocabulary model. State your feature convention clearly; the complete calculation above uses a consistent fixed training vocabulary.

","m2":true,"units":"5","marks":10},{"id":"final-2025-q6a","title":"Q6A · Coreference, pronouns and Word2Vec","html":"
Question. i. Differentiate between co-reference resolution and pronoun resolution with suitable examples.
ii. Explain the working principle of the Continuous Bag of Words (CBOW) model and Skipgram model in Word2Vec. Illustrate how it predicts the target word from its surrounding context. [10 marks]

Answer

i. Coreference and pronoun resolution

Coreference resolution groups expressions that refer to the same entity, including names, descriptions and pronouns. Pronoun resolution identifies the referent of a pronoun; it is narrower in scope.

Expression / taskExample
Coreference cluster“Dr Rao arrived. The surgeon opened her bag.” → {Dr Rao, the surgeon, her}
Pronoun resolutionFor the same text, resolve “her” to Dr Rao.
DifferenceResolving the description “the surgeon” is coreference work even when no pronoun is involved.

ii. Word2Vec

Word2Vec learns dense word embeddings by predicting words from local context. A dense embedding is a short learned list of real numbers for a word; words used in similar contexts can obtain related vectors. CBOW and Skip-gram use opposite prediction directions.

FeatureCBOWSkip-gram
InputSurrounding context wordsOne centre word
Prediction targetThe centre wordSurrounding context words
Training example[the, cat, on, the] → satsat → the; sat → cat; sat → on; sat → the
RepresentationCombine context embeddings, often by averagingUse the centre-word embedding for each context prediction

Example sentence: “The cat sat on the mat.” With a radius-two window around sat, the context is [The, cat, on, the]. In CBOW, obtain the four context embeddings, average or sum them, calculate scores for vocabulary words, and train the model to assign high probability to sat. In Skip-gram, use sat as input and create four centre–context training pairs.

CBOW: nearby words → combine embeddings → predict centre word
Skip-gram: centre word → its embedding → predict nearby words

Training adjusts embedding parameters to improve prediction; the final word vectors are the learned representation. Thus the question’s phrase “predicts the target word from surrounding context” describes CBOW specifically; Skip-gram reverses that direction.

Yaad rakho: CBOW: neighbours se centre. Skip-gram: centre se neighbours.

","m2":false,"units":"5,6","marks":10},{"id":"final-2025-q6b","title":"Q6B · Hobbs: Ravi and Suresh","html":"
Question. i. Explain Hobb’s Algorithm. Discuss the steps with help of example and state its limitation.
ii. Sentence: Ravi met Suresh at the station after he arrived from Delhi. Using Hobbs’ algorithm, determine the most likely antecedent of the pronoun “he”. Show the steps of traversal according to the algorithm. [10 marks]

Answer

i. Method

Hobbs’ algorithm is a syntax-based pronoun-resolution procedure that searches constituency parse trees in a specified order. It proposes noun phrases as antecedents and filters incompatible candidates, for example by number and gender agreement.

Read the tree labels first: NP means noun phrase; S means sentence/clause; VP means verb phrase. X is the current NP or S search node, and p is the upward path used to reach it. Breadth-first means shallower nodes before deeper nodes; left-to-right fixes the order at the same depth. An N-bar (N′) node is the noun-centred part of an NP in a parse using that label.

Complete ordered procedure

  1. Locate the pronoun. Start at the NP immediately dominating the pronoun: the smallest NP containing it.
  2. Make the first climb. Move upward to the first NP or S above the starting NP. Call it X and record the path p.
  3. Search initially to the left. Traverse branches below X to the left of p, breadth-first and left-to-right. Propose an NP only if another NP or S lies between that candidate and X. Check agreement and other applicable constraints for each proposed candidate; accept the first eligible compatible one.
  4. Check the highest sentence. If X is the highest S and no candidate has been accepted, search previous sentence trees, most recent first. In each, traverse breadth-first and left-to-right and propose NPs. Accept the first eligible compatible candidate; if the available context contains none, report no resolved antecedent.
  5. Otherwise climb again. Move from X to the next NP or S above it. Make that node the new X and record the new upward path p.
  6. Test a higher NP. If the new X is an NP, propose X only when p did not pass through its nominal N-bar child. Apply the same compatibility checks. Use the labels in the supplied parse; do not invent a missing N-bar branch.
  7. Search left again. Traverse the new X’s branches to the left of p, breadth-first and left-to-right, proposing NPs. The extra intervening-NP/S requirement in Step 3 belongs to the initial search.
  8. Search permitted right branches. If X is an S, traverse branches to the right of p, breadth-first and left-to-right, but do not descend below any NP or S encountered. Propose the encountered eligible NPs and test their compatibility.
  9. Repeat or finish. Return to Step 4 until a compatible candidate is accepted or the available sentence trees are exhausted. Record the candidate order and why each candidate was accepted or rejected.

Why the initial restriction matters: in “John saw him”, the subject NP John is directly under the clause S, with no additional NP or S between them. Step 3 therefore does not propose John as the ordinary local antecedent of him. A reflexive such as himself is a different grammatical case. The worked question below shows the specific parse and traversal; selecting an antecedent is not simply choosing the nearest name.

ii. Apply the method

Given: the pronoun he occurs as the subject of the subordinate clause “after he arrived from Delhi”. Assume that this clause is attached to the main verb phrase. The relevant main-clause NPs are Ravi, Suresh and the station.

To find: the first eligible singular-male antecedent in Hobbs’ syntactic search order.

S (main)\n├── NP: Ravi                    ← first higher-left candidate\n└── VP\n    ├── VP: met [NP Suresh] at [NP the station]\n    └── SBAR: after\n        └── S (embedded)        ← first X\n            ├── NP: he          ← start\n            └── VP: arrived from Delhi

Step 1. Start at the pronoun NP

Locate NP “he” inside the after-clause. Move to that clause’s S node. No earlier noun phrase in this small clause provides a candidate.

Step 2. Move to the higher sentence

The embedded S is not the highest S, so Step 5 climbs through SBAR and VP to the main S. Step 7 now searches to the left of that new path. This is the later left search, so the extra initial-search restriction from Step 3 is not imposed again.

Step 3. Examine candidates in structural order

The main subject NP Ravi is encountered before the deeper object NP Suresh in the relevant breadth-first search. Ravi is singular and compatible with “he”. “the station” would fail the ordinary person constraint.

Step 4. Choose the first compatible candidate

Select Ravi under the assumed parse and agreement settings. Suresh remains a possible interpretation in ordinary language, but the syntactic preference chooses Ravi first.

Limitations

Final answer: Hobbs’ syntactic preference is Ravi for this analysis. This is an algorithmic selection, not proof of the speaker’s intended referent.

Yaad rakho: Nearest naam blindly nahi: tree ki search order aur agreement follow karo.

","m2":false,"units":"6","marks":10},{"id":"final-2025-q7a","title":"Q7A · Text classification pipeline","html":"
Question. Explain the working of text classification in Natural Language Processing (NLP) and discuss the key steps involved in building a text classification system in context of real-world applications such as spam detection or news categorization. [10 marks]

Answer

Text classification assigns one or more predefined labels to a document or message based on its content. In spam detection the labels may be spam and not-spam; in news categorization they may be sports, business and politics.

Labelled text → Prepare and split data → Represent text → Train classifier → Evaluate → Classify new text
  1. Define the labels and task. Decide whether each text gets one label or multiple labels. Establish what counts as spam so annotators use a consistent rule.
  2. Collect and label representative examples. Gather messages from the intended application and assign reliable labels. Include enough examples of each class and protect private information.
  3. Split training, validation and test data. Use training data to learn, validation data to choose settings and an untouched test set for final evaluation. Keep duplicate or closely related texts from leaking between splits.
  4. Preprocess consistently. Tokenize and normalize only as appropriate. In spam detection, URLs, currency symbols and capitalisation may contain useful information.
  5. Create features. A classical baseline uses BoW or TF–IDF. A neural system may use embeddings or a pretrained contextual model. Fit the vocabulary and IDF on training data only.
  6. Train a classifier. For example, multinomial Naïve Bayes estimates class priors and word likelihoods; logistic regression learns a decision boundary. Training uses the labelled examples.
  7. Evaluate. Inspect a confusion matrix, precision, recall and F1 as well as accuracy. False spam flags may hide legitimate mail, so the costs of errors matter.
  8. Deploy and monitor. Apply the same preprocessing and feature mapping to new texts. Monitor changing vocabulary and error patterns, and retrain when suitable new labelled data becomes available.

Example prediction

A message such as “Claim your free prize now” is transformed into its feature representation. The classifier compares the evidence for the two labels and outputs spam if its decision score crosses the chosen threshold. The label comes from learned patterns; the mere presence of one word should not be treated as a universally correct rule.

Yaad rakho: Labelled examples → features → model → evaluation → new text ka label.

","m2":false,"units":"7","marks":10},{"id":"final-2025-q7b","title":"Q7B · Corpora and Jaccard similarity","html":"
Question. i. Differentiate between a flat text corpus and an annotated corpus, and discuss how each can be applied in various NLP tasks with suitable examples.
ii. Explain a real-world scenario where the Jaccard Similarity measure is more appropriate than Euclidean or Manhattan distance, and justify why Jaccard is the best choice in that case. [10 marks]

Answer

i. Flat and annotated corpora

A corpus is a collection of language data. A flat text corpus stores text without the linguistic labels needed by a particular task. An annotated corpus adds labels or structure such as POS tags, entity spans, parse trees or sentence sentiment.

BasisFlat text corpusAnnotated corpus
ExampleA collection of plain news articlesThe same articles with PERSON/ORG spans marked
InformationThe original text and possibly simple metadataText plus explicit task-related linguistic labels
ApplicationsWord frequency, collocations and unsupervised language-model trainingSupervised POS tagging, NER or parser training and evaluation
PreparationRelatively inexpensive to gather, but still needs cleaningRequires annotation guidelines, labour and quality checks

“Flat” describes the lack of the annotations being discussed, not an absence of all file organization. Both corpus types may still have document boundaries and metadata.

ii. Jaccard in a set-based application

Consider recommending articles using their topic-tag sets. Article A has {AI, NLP, ethics}; article B has {AI, NLP, healthcare}. We care about overlap among present tags, not how many irrelevant tags are absent from both.

Given

A = {AI, NLP, ethics}; B = {AI, NLP, healthcare}.

To find

The fraction of distinct present tags shared by the articles.

Formula and why

\"J(A,B)=\\frac{|A\\cap

The intersection counts common items; the union counts all distinct items present in either set.

Step 1. Find common tags

A ∩ B = {AI, NLP}, so the intersection size is 2.

Step 2. Find all distinct present tags

A ∪ B = {AI, NLP, ethics, healthcare}, so the union size is 4.

Step 3. Divide

\"J(A,B)=\\frac24=0.5\"

Jaccard directly measures proportional overlap and ignores joint absences. Euclidean or Manhattan distance on binary tag vectors counts mismatches but does not by itself normalize by the union size. If frequency or magnitude were meaningful, those distance measures could be more appropriate. Here, membership-only tags and sparse sets make Jaccard a suitable choice.

Yaad rakho: Jaccard: common present items / total distinct present items.

","m2":false,"units":"2,3","marks":10}]},{"id":"reexam-2026","title":"Solved re-exam · February 2026","unit":"PYQ","scope":"Actual re-examination · 9 February 2026 · 100 marks · 3 hours","sections":[{"id":"reexam-2026-q1a","title":"Q1A · Generic NLP pipeline","html":"
Question. Draw block diagram of generic NLP pipeline and explain the task associated with each block. [5 marks]

Answer

An NLP pipeline is an ordered workflow that transforms language data into a task-specific result. The exact components depend on the application; a generic supervised pipeline is shown below.

Collect text → Clean and preprocess → Represent text → Train / apply model → Evaluate and deliver output
BlockTaskExample for sentiment analysis
Data acquisitionCollect representative text and any required labelsReviews labelled positive or negative
Cleaning and preprocessingHandle noise, tokenize and normalize appropriatelyRemove HTML while preserving “not”
Text representationConvert tokens into usable model featuresTF–IDF values or contextual embeddings
ModelingLearn patterns from training data and predict new labelsTrain a classifier on labelled review features
EvaluationMeasure performance on unseen held-out data and inspect errorsPrecision, recall, F1 and confusion matrix
Deployment and feedbackDeliver predictions and monitor changing dataShow review sentiment and monitor new product vocabulary

Training/validation/test separation must precede fitting the learned preprocessing and feature statistics. New inputs must use the same feature mapping as training. Not every task uses every linguistic level explicitly: a modern end-to-end model may combine several operations internally.

Yaad rakho: Text lo → prepare karo → numbers banao → model lagao → result check aur use karo.

","m2":false,"units":"1","marks":5},{"id":"reexam-2026-q1b","title":"Q1B · Content and function words","html":"
Question. i. Define function words and content words.
ii. Which category (function words and content words) belong to open class and closed class? Justify your answer.
iii. For the sentence “The small cat is sleeping under the tree”, identify the function words and content words. [5 marks]

Answer

Content words carry much of a sentence’s lexical meaning, usually including nouns, main verbs, adjectives and many adverbs. Function words primarily express grammatical relationships, including determiners, prepositions, conjunctions, pronouns and auxiliary verbs.

CategoryTypical classReason
Content wordsOpen classNew nouns, verbs and adjectives enter the language readily, such as new technology names
Function wordsClosed classThe grammatical inventory is relatively stable; new determiners or pronouns are much less frequently added
WordRole in this sentenceCategory
TheDeterminerFunction
smallAdjectiveContent
catNounContent
isAuxiliary supporting progressive tenseFunction
sleepingMain verbContent
underPrepositionFunction
theDeterminerFunction
treeNounContent

Final answer: content words = small, cat, sleeping, tree; function words = The, is, under, the. “is” is auxiliary here; a word’s role must be judged in context. “Closed” means relatively resistant to new members, not that change is logically impossible.

Yaad rakho: Content words scene batate hain; function words grammar ka connection banate hain.

","m2":true,"units":"4","marks":5},{"id":"reexam-2026-q1c","title":"Q1C · Word2Vec and GloVe","html":"
Question. Differentiate between word2vec and GloVe in terms of their learning strategies. Which type of corpus would be more suitable for each method, and why? [5 marks]

Answer

Both Word2Vec and GloVe learn dense word vectors from distributional information: words occurring in related contexts tend to receive related representations. They differ mainly in the training objective and how corpus co-occurrence evidence is used.

BasisWord2VecGloVe
Learning strategyPredictive: CBOW predicts a centre word; Skip-gram predicts nearby wordsCount-based objective fitted to global word–context co-occurrence statistics
Training evidenceLocal context windows converted into prediction examplesA corpus-wide co-occurrence matrix and relationships among counts
Parameters learnedWord/context embeddings that make context predictions accurateWord/context embeddings and biases that fit weighted log co-occurrence counts
Corpus considerationsLarge token sequences with useful local context, including domain-specific textA sufficiently large corpus from which reliable global co-occurrence statistics can be collected

For a domain such as medicine, either method can benefit from a representative medical corpus. Word2Vec is convenient when generating local training windows directly from a stream of tokens. GloVe is natural when a global sparse co-occurrence matrix can be constructed and stored. Corpus size, domain match, preprocessing and available compute matter; there is no rule that Word2Vec requires informal text while GloVe requires formal text.

Both standard forms produce one static vector per vocabulary word, so multiple senses are blended. They do not automatically provide a different vector for the same word in each sentence.

Yaad rakho: Word2Vec predicts local neighbours; GloVe fits global co-occurrence counts.

","m2":false,"units":"5","marks":5},{"id":"reexam-2026-q1d","title":"Q1D · Anaphora, cataphora and coreference","html":"
Question. Differentiate between cataphora and anaphora in terms of direction of reference. How coreference resolution is different from anaphora reference? [5 marks]

Answer

Anaphora points backward to an expression or information introduced earlier. Cataphora points forward to an expression introduced later.

RelationExampleDirection
Anaphora“Riya arrived. She smiled.” She refers to the earlier Riya.Backward
Cataphora“Before she spoke, Riya checked the microphone.” She refers to the later Riya.Forward

Coreference means two or more expressions refer to the same entity. Coreference resolution groups such expressions, regardless of whether they are names, descriptions or pronouns and regardless of which is mentioned first.

For “Dr Sen arrived. The surgeon smiled. She waved,” {Dr Sen, the surgeon, She} is a coreference cluster. The latter two mentions are also backward-looking references. Coreference is an identity relation; anaphora describes a dependency on prior discourse and is not always identity. For example, “I bought a house. The roof leaks” contains a bridging reference: the roof depends on the house for interpretation, but roof and house are not the same entity.

Yaad rakho: Ana = pehle ki taraf; cata = aage ki taraf; coreference = same entity.

","m2":false,"units":"6","marks":5},{"id":"reexam-2026-q2a","title":"Q2A · NLP tasks and syntax versus semantics","html":"
Question. i. Discuss five different tasks associated with Natural Language Processing.
ii. Compare and contrast syntactic analysis and semantic analysis in NLP. Support your explanation with examples. [10 marks]

Answer

i. Five NLP tasks

TaskWhat it doesExample
Machine translationExpresses source-language content in a target languageEnglish “Good morning” → Hindi greeting
Sentiment analysisIdentifies the opinion or attitude expressed“The service was excellent” → positive sentiment
Named Entity RecognitionFinds and types entity spans“Asha joined Google” → PERSON and ORGANIZATION
Question answeringReturns an answer supported by a question’s relevant informationFrom a passage, identify who discovered a described result
Text classificationAssigns predefined category labelsClassify an email as spam or legitimate

ii. Syntactic and semantic analysis

BasisSyntactic analysisSemantic analysis
Main questionHow are words grammatically organised?What do the words and sentence mean?
Typical outputPOS tags, constituency or dependency structureWord senses, semantic relations or a meaning representation
Knowledge usedGrammar rules and structural patternsLexical meaning, context and semantic constraints
ExampleIdentify “The boy” as subject NP and “ate an apple” as VPInterpret the boy as eater and apple as the thing eaten

“Colourless green ideas sleep furiously” can have a plausible English grammatical structure while remaining semantically unusual. Syntax alone therefore does not guarantee ordinary meaning. Conversely, “boy eat apple yesterday” conveys a possible event despite grammatical errors.

The analyses support each other. In “I saw the man with a telescope,” syntactic PP attachment determines which meaning is available, and semantic knowledge can help choose a plausible attachment. A full NLP system often needs both.

Yaad rakho: Syntax = sentence ka structure; semantics = us structure ka meaning.

","m2":false,"units":"1,4,5","marks":10},{"id":"reexam-2026-q2b","title":"Q2B · Laplace smoothing on a second corpus","html":"
Question. Consider the following training corpus:
1. cats chase mice in the garden
2. dogs bark loudly at night
3. birds sing in the morning
4. children play games in the park
Using a bigram language model with Laplace smoothing, evaluate the probability of the test sentence: “cats bark in the park.” Assume <s> and </s> as start and end of the sentence. [10 marks]

Answer

Given

Test: cats bark in the park. Add one start token <s> and one end token </s> to every training and test sentence.

To find

The Laplace-smoothed bigram probability of the complete test sentence, including its end token.

Formula: which one and why

\"P_{+1}(w_i\\mid

C(previous,current) counts that adjacent pair in the training corpus. C(previous) counts its occurrences as a history. V is the vocabulary size. Add-one smoothing is necessary because an unseen pair must not force the whole sentence probability to zero.

\"P(w_1,\\ldots,w_n,\\langle/s\\rangle\\mid\\langle

Solution

Step 1. Fix the token and vocabulary convention

Lowercase words and remove the final punctuation. The supplied course key counts both boundary tokens in V. Following that convention: at, bark, birds, cats, chase, children, dogs, games, garden, in, loudly, mice, morning, night, park, play, sing, the, <s>, </s> gives V = 20. The start marker supplies history; it is not an extra probability factor before the first word.

Step 2. List every required transition

The complete test chain is <s> → cats → bark → in → the → park → </s>.

Step 3. Count first, then add one

Required bigramHistory countPair countSmoothed probability
<s> → cats41(1+1)/(4+20) = 1/12
cats → bark10(0+1)/(1+20) = 1/21
bark → in10(0+1)/(1+20) = 1/21
in → the33(3+1)/(3+20) = 4/23
the → park31(1+1)/(3+20) = 2/23
park → </s>11(1+1)/(1+20) = 2/21

Count only adjacent words, never words that merely appear in the same sentence. The unseen pairs have count zero, so their numerator becomes one.

Step 4. Multiply all six probabilities

\"P=\\frac{2}{24}\\times\\frac{1}{21}\\times\\frac{1}{21}\\times\\frac{4}{23}\\times\\frac{2}{23}\\times\\frac{2}{21}\"

Start with subtotal 1. At each row, multiply the previous subtotal by exactly one next factor. Simplify the fraction after multiplication, but do not round it.

Factor being includedPrevious subtotal × next factor = new subtotal
1. <s> → cats\"1\\times\\frac{1}{12}=\\frac{1}{12}\"
2. cats → bark\"\\frac{1}{12}\\times\\frac{1}{21}=\\frac{1}{252}\"
3. bark → in\"\\frac{1}{252}\\times\\frac{1}{21}=\\frac{1}{5292}\"
4. in → the\"\\frac{1}{5292}\\times\\frac{4}{23}=\\frac{1}{30429}\"
5. the → park\"\\frac{1}{30429}\\times\\frac{2}{23}=\\frac{2}{699867}\"
6. park → </s>\"\\frac{2}{699867}\\times\\frac{2}{21}=\\frac{4}{14697207}\"

The last subtotal is the complete product. Only now convert the fraction to a decimal.

\"P=\\frac{4}{14697207}\\approx

Step 5. Interpret the result

This is a joint sentence probability, so a small value is normal. Do not add transition probabilities; each word must occur in sequence, so the chain multiplies them.

Final answer

P ≈ 2.72160554 × 10-7 with the stated course vocabulary convention.

Yaad rakho: Pair count mein 1 jodo, previous-word count mein V; phir saare fractions multiply karo.

Convention note: Some implementations exclude <s> from the predicted vocabulary. Their V is one smaller and their numerical probability differs. In an exam, state V explicitly and follow the convention specified in the question or class solution.

","m2":true,"units":"3","marks":10},{"id":"reexam-2026-q3a","title":"Q3A · Top-down parsing","html":"
Question. In the context of natural language processing, explain the concept of top-down and bottom-up parsing approaches. Apply top-down syntax parsing approach to the sentence “The tall boy saw a girl in the park.” and draw a parse tree using the CFG grammar rules:
S → NP VP; NP → Det Adj N | Det N; VP → V NP | VP PP; PP → P NP.
Lexicon: Det → {the | a}; Adj → {tall}; N → {boy | girl | park}; V → {saw}; P → {in}. [10 marks]

Answer

AspectTop-downBottom-up
StartThe start symbol SThe input words
OperationExpand a nonterminal using grammar productionsCombine words and constituents using productions in reverse
SuccessLeaves match the complete inputAll input is reduced to S
Possible difficultyNaive recursive descent can loop on left recursion or need backtrackingA naive shift-reduce parser may face shift/reduce conflicts

Given

The sentence contains the tokens the / tall / boy / saw / a / girl / in / the / park. Lowercase the initial “The” to match the lexicon. The full stop is punctuation outside the listed grammar.

To find

A derivation beginning at S and a parse tree matching the whole sentence.

Rule: which one and why

A context-free production replaces one nonterminal with its right-hand sequence. A top-down derivation begins at S and expands the desired structure. The supplied grammar attaches the PP through VP → VP PP; it does not provide NP → NP PP.

Step 1. Expand the sentence

S ⇒ NP VP. Choose NP → Det Adj N for “the tall boy”.

Step 2. Match the subject words

Det ⇒ the, Adj ⇒ tall, N ⇒ boy. The remaining material must be generated by VP.

Step 3. Allow a verb phrase followed by a PP

Expand the outer VP using VP → VP PP. Then expand the inner VP using VP → V NP. This finite derivation selects the recursive rule once.

Step 4. Match the verb and object

V ⇒ saw. Choose NP → Det N, then Det ⇒ a and N ⇒ girl.

Step 5. Expand the prepositional phrase

PP ⇒ P NP. P ⇒ in. NP ⇒ Det N, with Det ⇒ the and N ⇒ park.

Step 6. Check the complete leaf order

The leaves read “the tall boy saw a girl in the park”, exactly the normalized input. The PP “in the park” attaches to the VP under this supplied grammar.

SNPVPDetAdjNVPPPVNPPNPDetNDetNThetallboysawagirlinthepark
Top-down parse tree showing the PP attached to VP.

Complete derivation

S ⇒ NP VP ⇒ Det Adj N VP ⇒ the tall boy VP ⇒ the tall boy VP PP ⇒ the tall boy V NP PP ⇒ the tall boy saw Det N PP ⇒ the tall boy saw a girl P NP ⇒ the tall boy saw a girl in Det N ⇒ the tall boy saw a girl in the park.

Implementation caution: VP → VP PP is left-recursive. A naive recursive-descent implementation may loop; it must transform or otherwise handle the grammar. A valid finite top-down derivation, as shown above, still exists.

Yaad rakho: Rule jo diya hai wahi use karo: yahan “in the park” VP ka part hai.

","m2":true,"units":"4","marks":10},{"id":"reexam-2026-q3b","title":"Q3B · Lesk word-sense disambiguation","html":"
Question. Consider the sentence: “The bank can ensure your deposits are safe.” The word “bank” has the following two possible senses (WordNet-style glosses):
Bank₁: Sloping land especially beside a body of water.
Bank₂: A financial institution that accepts deposits and channels the money into lending activities.
i. Apply the LESK algorithm to determine the correct sense of the word “bank” in the given sentence. Show the steps clearly.
ii. Briefly discuss limitations of the LESK algorithm in handling ambiguous words in real-world contexts. [10 marks]

Answer

Given

Context sentence and the two supplied dictionary glosses. Use a simplified Lesk method: lowercase, remove function words, exclude the target bank and compare distinct content-word overlap. Do not add synonyms that the question does not supply.

To find

Which candidate gloss has the greatest overlap with the local context.

Formula: which one and why

\"\\mathrm{score}(s)=|C\\cap

C is the set of context content words. Gₛ is the content-word set of the gloss for sense s. The intersection contains words appearing in both; its size is the score. This is an overlap count, not a probability.

Solution

Step 1. Prepare the context

Remove The, can, your, are and the target bank. A reasonable content set is C = {ensure, deposits, safe}. If “can” is retained under another fixed stop-list it matches neither gloss and does not change the result.

Step 2. Prepare Bank₁’s gloss

G₁ = {sloping, land, especially, beside, body, water}. There is no shared word with {ensure, deposits, safe}.

Step 3. Calculate Bank₁’s score

\"C\\cap

Step 4. Prepare Bank₂’s gloss and count overlap

G₂ = {financial, institution, accepts, deposits, channels, money, lending, activities}. The shared word is deposits.

Step 5. Calculate Bank₂’s score and select

\"C\\cap

1 > 0, so choose the financial-institution sense.

Final answer

Bank₂: financial institution. The explicit overlap “deposits” supports this choice.

Yaad rakho: Lesk: context aur har meaning ki definition mein common words count karo; highest overlap choose karo.

ii. Limitations

Lemmatization, richer glosses and semantic similarity can improve variants, but the exact variant and tie-breaking rule should be stated.

","m2":true,"units":"5","marks":10},{"id":"reexam-2026-q4a","title":"Q4A · Bag of Words and three cosine similarities","html":"
Question. Consider the following three documents:
Document A: “The dog chased the cat.”
Document B: “The cat chased the mouse.”
Document C: “The dog ran after the cat.”
Using the Bag of Words (BoW) model, represent each document as a vector after removing the stop words {The, by, was}.
i. Construct the term-document matrix.
ii. Compute the cosine similarity between each pair of documents (A–B, A–C, and B–C).
iii. Interpret and comment on the results obtained in terms of document similarity. [10 marks]

Answer

Given

Three documents. Lowercase and remove punctuation. Remove the specified stop words case-insensitively; “the” is removed everywhere. Keep “after” because it is not in the supplied stop list. Use raw word counts.

To find

A common-vocabulary count matrix, three cosine values and their interpretation.

Formula: which one and why

\"\\mathrm{BoW}_j(d)=C(v_j,d)\"

vⱼ is vocabulary term j. The coordinate stores its count in document d; order in the sentence is not retained.

\"\\cos(\\mathbf

Solution

Step 1. Clean each document

A: dog chased cat.
B: cat chased mouse.
C: dog ran after cat.

Step 2. Fix vocabulary and matrix

Vocabulary: dog, chased, cat, mouse, ran, after.

TermABC
dog101
chased110
cat111
mouse010
ran001
after001

A = [1,1,1,0,0,0]; B = [0,1,1,1,0,0]; C = [1,0,1,0,1,1].

Step 3. Calculate vector lengths

\"\\|A\\|=\\sqrt{1^2+1^2+1^2}=\\sqrt3\"
\"\\|B\\|=\\sqrt{1^2+1^2+1^2}=\\sqrt3\"
\"\\|C\\|=\\sqrt{1^2+1^2+1^2+1^2}=2\"

Step 4. Calculate A–B

\"A\\cdot
\"\\cos(A,B)=\\frac2{\\sqrt3\\sqrt3}=\\frac23\\approx0.666667\"

Step 5. Calculate A–C

\"A\\cdot
\"\\cos(A,C)=\\frac2{\\sqrt3\\cdot2}=\\frac1{\\sqrt3}\\approx0.577350\"

Step 6. Calculate B–C

\"B\\cdot
\"\\cos(B,C)=\\frac1{\\sqrt3\\cdot2}\\approx0.288675\"

Final answer

A–B = 0.666667; A–C = 0.577350; B–C = 0.288675. A and B have the highest lexical overlap under this representation. B and C share only cat. BoW similarity reflects token overlap, not full semantic equivalence or word order.

Yaad rakho: Same vocabulary order rakho. Corresponding counts multiply-add; phir dono lengths se divide.

","m2":false,"units":"2","marks":10},{"id":"reexam-2026-q4b","title":"Q4B · A second HMM sequence","html":"
Question. Consider the following POS tagged corpus. Apply Hidden Markov Model (HMM) approach to:
1. Dog/Noun eats/Verb bread/Noun
2. The/Det cat/Noun sleeps/Verb
3. A/Det dog/Noun eats/Verb fish/Noun
i. Construct the emission probability matrix.
ii. Construct the transmission probability matrix.
iii. Calculate the probability of the given sequence of events: The/Det dog/Noun eats/Verb bread/Noun.
Note: Include start and end characters for all the sentences. [10 marks]

Answer

The repeated verb eats has a different emission count from the final paper; recalculate it rather than copying the previous numerical result.

Given

SentenceTagged sequence
1dog/Noun eats/Verb bread/Noun
2the/Det cat/Noun sleeps/Verb
3a/Det dog/Noun eats/Verb fish/Noun

Lowercase the corpus. Test sequence: the/Det dog/Noun eats/Verb bread/Noun. Add Start and End markers.

To find

Emission and transition matrices, then the probability of the supplied tagged sequence.

Formula: which one and why

\"P(w\\mid
\"P(t_j\\mid

Emission divides the word’s tag-specific count by the total tokens with that tag. Transition divides the tag-pair count by all transitions leaving the previous tag, including End.

\"P(\\mathbf

Solution

Step 1. Count tag totals and word frequencies

Det: the=1, a=1, total2. Noun: dog=2, bread=1, cat=1, fish=1, total5. Verb: eats=2, sleeps=1, total3.

Step 2. Construct emission matrix

Tag / wordabreadcatdogeatsfishsleepsthe
Det1/20000001/2
Noun01/51/52/501/500
Verb00002/301/30

Step 3. Count transitions from the three chains

Start → Noun → Verb → Noun → End
Start → Det → Noun → Verb → End
Start → Det → Noun → Verb → Noun → End

FromDet countNoun countVerb countEnd countTotal
Start21003
Det02002
Noun00325
Verb02013

Step 4. Divide outgoing counts

From / toDetNounVerbEnd
Start2/31/300
Det0100
Noun003/52/5
Verb02/301/3

All nonterminal rows sum to one. No smoothing is requested.

Step 5. Write every sequence factor

EventProbability
Start → Det2/3
Det emits the1/2
Det → Noun1
Noun emits dog2/5
Noun → Verb3/5
Verb emits eats2/3
Verb → Noun2/3
Noun emits bread1/5
Noun → End2/5

Step 6. Multiply all factors

\"P=\\frac23\\times\\frac12\\times1\\times\\frac25\\times\\frac35\\times\\frac23\\times\\frac23\\times\\frac15\\times\\frac25\"

Start with subtotal 1. At each row, multiply the previous subtotal by exactly one next factor. Simplify the fraction after multiplication, but do not round it.

Factor being includedPrevious subtotal × next factor = new subtotal
1. Start → Det\"1\\times\\frac{2}{3}=\\frac{2}{3}\"
2. Det emits the\"\\frac{2}{3}\\times\\frac{1}{2}=\\frac{1}{3}\"
3. Det → Noun\"\\frac{1}{3}\\times1=\\frac{1}{3}\"
4. Noun emits dog\"\\frac{1}{3}\\times\\frac{2}{5}=\\frac{2}{15}\"
5. Noun → Verb\"\\frac{2}{15}\\times\\frac{3}{5}=\\frac{2}{25}\"
6. Verb emits eats\"\\frac{2}{25}\\times\\frac{2}{3}=\\frac{4}{75}\"
7. Verb → Noun\"\\frac{4}{75}\\times\\frac{2}{3}=\\frac{8}{225}\"
8. Noun emits bread\"\\frac{8}{225}\\times\\frac{1}{5}=\\frac{8}{1125}\"
9. Noun → End\"\\frac{8}{1125}\\times\\frac{2}{5}=\\frac{16}{5625}\"

The last subtotal is the complete product. Only now convert the fraction to a decimal.

\"P=\\frac{16}{5625}\\approx0.0028444444\"

Final answer

P = 16/5625 ≈ 0.0028444444 for these words and specified tags jointly. The two occurrences of eats give P(eats | Verb) = 2/3.

Yaad rakho: Same tag structure ho sakta hai, lekin word emission counts dobara count karna.

Boundary reminder: Include P(Det | Start) = 2/3 and P(End | Noun) = 2/5. The handwritten key’s final line instead inserts the overall Det fraction 2/10 and omits End; the complete boundary-aware probability requested here is 0.0028444444.

","m2":true,"units":"4","marks":10},{"id":"reexam-2026-q5a","title":"Q5A · Why TF–IDF is useful","html":"
Question. Justify with an appropriate example the importance of TF-IDF technique in the document representation and information retrieval task. [10 marks]

Answer

TF–IDF represents each document using weights that combine within-document term frequency with how unusual a term is across the collection. It supports information retrieval by giving larger weight to terms that are frequent in a document but not present in every document.

Given

Use three short documents: D₁ = “solar energy solar”; D₂ = “wind energy”; D₃ = “energy policy”. Use raw-count TF and natural-log unsmoothed IDF. N = 3.

To find

A TF–IDF representation illustrating why distinctive terms help retrieve relevant documents.

Formula: which one and why

\"\\mathrm{TFIDF}(t,d)=C(t,d)\\ln\\frac{N}{\\mathrm{df}(t)}\"

C counts occurrences within a document; df counts documents containing the term. The logarithm reduces the scale of very large rarity ratios.

Solution

Step 1. Count term occurrences

TermD₁D₂D₃df
solar2001
energy1113
wind0101
policy0011

Step 2. Calculate IDF

\"\\mathrm{IDF}(solar)=\\mathrm{IDF}(wind)=\\mathrm{IDF}(policy)=\\ln3\\approx1.098612\"
\"\\mathrm{IDF}(energy)=\\ln(3/3)=0\"

Step 3. Multiply each count by its IDF

TermD₁ weightD₂ weightD₃ weight
solar2 × 1.098612 = 2.19722400
energy000
wind01.0986120
policy001.098612

Step 4. Connect the representation to retrieval

A query containing “solar” matches D₁’s high solar coordinate. A query containing “wind” matches D₂. “energy” alone cannot distinguish these documents under this unsmoothed collection because it occurs in all three. A retrieval system can compare query and document vectors, commonly by cosine similarity.

Final answer

TF–IDF preserves distinctive lexical evidence and reduces dominance by collection-wide frequent terms. D₁ is most relevant to a solar query under these features.

Yaad rakho: Document mein frequent + collection mein rare = high TF–IDF.

Strengths and limits

The representation is sparse, interpretable and useful as a fast baseline. However, basic TF–IDF ignores word order and does not recognise synonyms or multiple word senses automatically. Its weights depend on the collection and on the chosen TF/IDF convention. Fit IDF on training or indexed documents, not on a hidden test set.

","m2":false,"units":"2","marks":10},{"id":"reexam-2026-q5b","title":"Q5B · Sentence-window vectors and cosine","html":"
Question. Consider the paragraph: The farmer planted seeds in the soil. The rain watered the soil. The sun helped the plants grow. The farmer harvested the plants. The soil became rich and fertile.
Target words: farmer, soil, sun, plant.
Context words: plant, water, grow, harvest.
i. Construct a co-occurrence matrix for the target and context words. Take the window size as one whole sentence.
ii. Evaluate the similarity between the vectors of the target words “farmer-soil”, “farmer-sun”, and “farmer-plant” using the cosine similarity.
iii. Plot the target word vectors by considering only two dimensions: “planted” and “harvested”. [10 marks]

Answer

Given

Five sentence windows. For alignment with the supplied matrix, target plant means the noun plants, occurring in sentences 3 and 4. Context plant covers planted and plants; watered → water; harvested → harvest. Count a target–context pair once per sentence in which both are present, including the plant–plant sentence-presence diagonal.

To find

The target–context matrix, three cosine similarities and a two-coordinate plot.

Formula: which one and why

\"M_{ij}=\\sum_s\\mathbf1[\\text{target}_i\\text{
\"\\cos(\\mathbf

This is a sentence-presence matrix, not a position-based sliding window that excludes a token from its own context. Stating the convention is necessary because those alternatives give a different diagonal.

Solution

Step 1. Normalize and identify each sentence’s contribution

SentenceTargets presentContext lemmas present
1farmer, soilplant
2soilwater
3sun, plantplant, grow
4farmer, plantplant, harvest
5soilnone

Step 2. Fill all matrix rows

Targetplantwatergrowharvest
farmer2001
soil1100
sun1010
plant2011

Farmer meets context plant in sentences 1 and 4. Soil meets plant in sentence 1 and water in sentence 2. Sun meets plant and grow in sentence 3. The noun plant occurs in sentences 3 and 4, producing its last row.

Step 3. Compute farmer–soil

\"\\mathbf
\"\\|\\mathbf
\"\\cos(\\mathbf

Step 4. Compute farmer–sun

\"\\mathbf
\"\\|\\mathbf

Step 5. Compute farmer–plant

\"\\mathbf
\"\\|\\mathbf
\"\\cos(\\mathbf

Step 6. Plot the requested two coordinates

Interpret the axis names planted and harvested as their normalized context features plant and harvest. Farmer=(2,1), soil=(1,0), sun=(1,0), plant=(2,1). Farmer and plant overlap; soil and sun overlap.

001122farmer, plantsoil, sunplant (planted)harvest (harvested)
The plot uses only the requested two coordinates; cosine uses all context columns.

Final answer

Using the stated noun-target convention: farmer–soil = 0.632456, farmer–sun = 0.632456, farmer–plant = 0.912871.

Yaad rakho: Plant noun aur plant verb ko same maana ya separate? Pehle convention likho, phir count karo.

Two details that prevent an incorrect answer

Arithmetic: 5/√30 = 0.912871; the source key’s printed approximation 0.9444 does not equal its own fraction. Keep the exact fraction until the last step.

Normalization: If the question instead requires all noun and verb forms of plant to be merged as targets, sentence 1 also contains the plant target. Its row becomes [3,0,1,1], and farmer–plant becomes 7/√55 ≈ 0.943880. The paper leaves this choice unstated; the main worked solution explicitly uses the noun-target interpretation that reproduces its supplied matrix.

","m2":false,"units":"5","marks":10},{"id":"reexam-2026-q6a","title":"Q6A · Pronoun approaches and lexical relations","html":"
Question. i. Briefly describe any two approaches to pronoun resolution and mention one advantage of each.
ii. Describe the difference between Homonymy and Synonymy with suitable examples. Discuss how these two relations differently affect natural language processing applications? [10 marks]

Answer

i. Two approaches to pronoun resolution

ApproachHow it worksAdvantage
Rule-based / syntacticUses parse structure, agreement, recency and linguistic constraints. Hobbs’ algorithm searches parse trees for eligible antecedents.Interpretable: a proposed antecedent can be traced to explicit rules.
Statistical or neuralLearns candidate–pronoun compatibility or whole-document coreference patterns from labelled examples, using context and linguistic features or contextual embeddings.Can learn interactions and contextual cues that are difficult to enumerate as hand-written rules.

For “Asha dropped the glass because it was slippery,” a system should evaluate candidate referents and context rather than merely choose the most recent noun. Rules can constrain number and syntax; learned models can use compatibility patterns, though neither approach is guaranteed to resolve every ambiguity.

ii. Homonymy and synonymy

RelationDefinitionExample
HomonymySame spelling or sound, unrelated meaningsbank as financial institution versus bank as river edge
SynonymyDifferent words with the same or closely related meanings in a contextcar and automobile; big and large

Homonymy creates an ambiguity problem: a search for bank may retrieve finance or rivers. WSD uses context to choose the intended sense. Synonymy creates a vocabulary-matching problem: a document saying automobile may be relevant to a query saying car. Query expansion or semantic representations can improve recall by connecting related terms.

The remedies differ. Do not merge unrelated homonym senses simply because their spelling matches. Do not insist on exact surface-form matching when different words express the same concept. Synonyms are also context-sensitive: “big mistake” and “large mistake” are not always equally natural.

Yaad rakho: Homonym: same word-form, different meaning. Synonym: different words, similar meaning.

","m2":false,"units":"5,6","marks":10},{"id":"reexam-2026-q6b","title":"Q6B · Hobbs: Ramesh and Shyam","html":"
Question. i. Explain Hobbs’ Algorithm for pronoun resolution. State its main steps and limitations.
ii. Apply Hobbs’ Algorithm to the sentence: “When Ramesh gave Shyam a book, he smiled.” Identify the antecedent of “he” by showing traversal steps. [10 marks]

Answer

Hobbs’ algorithm is a syntax-based pronoun-resolution procedure that searches constituency parse trees in a specified order. It proposes noun phrases as antecedents and filters incompatible candidates, for example by number and gender agreement.

Read the tree labels first: NP means noun phrase; S means sentence/clause; VP means verb phrase. X is the current NP or S search node, and p is the upward path used to reach it. Breadth-first means shallower nodes before deeper nodes; left-to-right fixes the order at the same depth. An N-bar (N′) node is the noun-centred part of an NP in a parse using that label.

Complete ordered procedure

  1. Locate the pronoun. Start at the NP immediately dominating the pronoun: the smallest NP containing it.
  2. Make the first climb. Move upward to the first NP or S above the starting NP. Call it X and record the path p.
  3. Search initially to the left. Traverse branches below X to the left of p, breadth-first and left-to-right. Propose an NP only if another NP or S lies between that candidate and X. Check agreement and other applicable constraints for each proposed candidate; accept the first eligible compatible one.
  4. Check the highest sentence. If X is the highest S and no candidate has been accepted, search previous sentence trees, most recent first. In each, traverse breadth-first and left-to-right and propose NPs. Accept the first eligible compatible candidate; if the available context contains none, report no resolved antecedent.
  5. Otherwise climb again. Move from X to the next NP or S above it. Make that node the new X and record the new upward path p.
  6. Test a higher NP. If the new X is an NP, propose X only when p did not pass through its nominal N-bar child. Apply the same compatibility checks. Use the labels in the supplied parse; do not invent a missing N-bar branch.
  7. Search left again. Traverse the new X’s branches to the left of p, breadth-first and left-to-right, proposing NPs. The extra intervening-NP/S requirement in Step 3 belongs to the initial search.
  8. Search permitted right branches. If X is an S, traverse branches to the right of p, breadth-first and left-to-right, but do not descend below any NP or S encountered. Propose the encountered eligible NPs and test their compatibility.
  9. Repeat or finish. Return to Step 4 until a compatible candidate is accepted or the available sentence trees are exhausted. Record the candidate order and why each candidate was accepted or rejected.

Why the initial restriction matters: in “John saw him”, the subject NP John is directly under the clause S, with no additional NP or S between them. Step 3 therefore does not propose John as the ordinary local antecedent of him. A reflexive such as himself is a different grammatical case. The worked question below shows the specific parse and traversal; selecting an antecedent is not simply choosing the nearest name.

Given

The pronoun he is the subject of the main clause. The preceding subordinate clause is “When Ramesh gave Shyam a book”. Assume the temporal clause attaches to the whole main sentence.

To find

The first suitable antecedent in the left-to-right breadth-first search of the preceding clause.

S (main)                        ← first X\n├── SBAR: When\n│   └── S (embedded)             ← intervening S\n│       ├── NP: Ramesh           ← first candidate\n│       └── VP\n│           ├── V: gave\n│           ├── NP: Shyam\n│           └── NP: a book\n├── NP: he                      ← start\n└── VP: smiled

Step 1. Locate the pronoun NP

Start at NP “he” and move upward to its S ancestor. Record the path from he.

Step 2. Search branches before the pronoun

The initial when-clause lies to the left of the path. Apply Step 3: its embedded S lies between each of its noun phrases and the main S, so these NPs satisfy the initial intervening-NP/S requirement. Traverse left-to-right and breadth-first.

Step 3. Propose noun phrases

The subject NP Ramesh is encountered before the deeper object NP Shyam and object NP a book. Ramesh agrees with the assumed singular-male pronoun.

Step 4. Accept the first compatible antecedent

Select Ramesh. Shyam is also a semantically possible referent in ordinary discourse, but the algorithm’s stated syntactic preference selects Ramesh first.

Limitations

Final answer: Ramesh, under the shown parse and ordinary agreement assumptions. A syntactic preference is not a guarantee about the author’s intended meaning.

Yaad rakho: Left branches ko breadth-first dekho; pehla eligible matching NP choose karo.

","m2":false,"units":"6","marks":10},{"id":"reexam-2026-q7a","title":"Q7A · Information retrieval","html":"
Question. i. Explain the concept of Information Retrieval, highlighting its role in Recommender Systems and Search Engines.
ii. Discuss the various steps involved in the Information Retrieval process. [10 marks]

Answer

i. Concept and roles

Information Retrieval (IR) finds and ranks documents or other information objects relevant to a user’s information need. A query is the user’s expression of that need. IR typically returns a ranked set of items; information extraction instead identifies structured facts or fields inside those items.

ApplicationRole of IRExample
Search engineMatches a submitted query against a large indexed collection and ranks candidate pagesA search for “symptoms of dehydration” returns relevant health pages
Recommender systemRetrieves candidate items using a user profile, recent activity or a content representation; a later model may personalize rankingRetrieve articles similar to topics the user reads

Retrieval is a component of many recommenders; recommendation also involves preferences, feedback, novelty and business constraints. A recommender need not wait for an explicit textual query.

ii. Retrieval process

Collect → Preprocess → Index → Process query → Retrieve candidates → Rank → Evaluate
  1. Collect and define documents. Crawl or ingest documents, record metadata and remove unwanted duplicates.
  2. Preprocess. Tokenize and normalize using rules suitable for both documents and queries. Preserve task-relevant information.
  3. Build representations and an index. An inverted index maps each term to the documents containing it; a dense retrieval system stores vector representations.
  4. Process the query. Apply compatible tokenization, spelling handling or expansion, then obtain query features.
  5. Retrieve candidates. Use the index to identify potentially relevant documents efficiently.
  6. Score and rank. A lexical method may use TF–IDF/BM25-style evidence; a semantic method compares representations. Rank scores estimate relevance rather than proving correctness.
  7. Present results. Return titles, snippets and links, possibly with filtering or diversification.
  8. Evaluate and update. Use relevance judgments and measures such as precision, recall or ranking metrics. Refresh the index as documents change.

Example: for “solar energy storage”, an inverted index retrieves documents containing those terms; a ranking function rewards useful matches. A semantic system may also retrieve relevant material using related terms such as batteries.

Yaad rakho: IR mein relevant document dhoondho aur rank karo; extraction mein document ke andar facts nikalo.

","m2":false,"units":"7","marks":10},{"id":"reexam-2026-q7b","title":"Q7B · Gutenberg corpus and normalization","html":"
Question. i. Discuss how the Gutenberg Corpus can be utilized for various text analysis tasks in Natural Language Processing?
ii. Explain the importance of text normalization in Natural Language Processing (NLP). Illustrate your answer with a suitable example of an application where text normalization plays a crucial role. [10 marks]

Answer

i. Using the Gutenberg corpus

The Gutenberg corpus provides literary texts, commonly used through tools such as NLTK’s Gutenberg collection. It supplies naturally occurring sentences and longer documents for exploring language patterns. The available subset and licensing should be checked before reuse in a new distribution.

TaskUse of the corpus
Frequency and vocabulary studyCount tokens and word types, inspect common words and compare lexical diversity across texts
Concordances and collocationsView a word in context and discover words that occur near each other
N-gram language modelingEstimate unigram or bigram counts and compare sequence probabilities
Stylistic or authorship analysisCompare sentence lengths, vocabulary and function-word patterns across authors
Text preprocessing experimentsStudy the effect of tokenization, case normalization and lemmatization on counts

A sound workflow selects files, inspects encoding and boundaries, tokenizes consistently, computes the relevant statistics and interprets them against corpus composition. Literary text is not automatically representative of chat, medical records or present-day search queries; domain mismatch matters.

ii. Text normalization

Normalization converts selected surface variants into a consistent form so that equivalent expressions do not unnecessarily produce different features. Possible steps include case folding, whitespace cleanup, Unicode normalization, contraction expansion and lemmatization. Choose these according to the task.

Example: product search

  1. Raw query: “ RUNNING shoes ”. Product description: “Comfortable run shoes”.
  2. Trim extra whitespace and apply consistent case: “running shoes”.
  3. If the application chooses lemmatization, running → run. The query and the indexed description now share the intended run and shoes features.
  4. Apply the same normalization to stored documents and future queries, otherwise matching remains inconsistent.

Normalization reduces avoidable vocabulary variation and can improve recall. It must preserve distinctions important to the task: lowercasing US to us may lose meaning, and removing “not” can reverse sentiment. Aggressive stemming or synonym replacement can create incorrect matches.

Yaad rakho: Same meaning ke surface variants ko consistent banao; useful meaning delete mat karo.

","m2":false,"units":"2,3","marks":10}]},{"id":"legacy-r25","title":"2025-02-21 · re-exam · recovered AI paper","unit":"PYQ","scope":"Previous examination · complete worked answers","sections":[{"id":"q1a","title":"Q1a · BoW and one-hot encoding","html":"
Question. Explain Bag of Words and one-hot encoding with appropriate examples. [individual allocation not printed; Q1 totals 20]

Answer

Bag of words (BoW) represents a document by the counts of vocabulary terms, discarding their positions. A binary BoW variant stores only presence/absence; a count BoW stores frequencies. State which is used.

\"x_{d,j}=C(v_j,d).\"

\"x_{d,j}\" is document \"d\"'s coordinate for vocabulary term \"v_j\". \"C(v_j,d)\" is its occurrence count.

A bag of n-grams uses short contiguous token sequences as features. Bigrams can distinguish “not good” from “good”, but increasing n enlarges the vocabulary and creates more rare features.

Question. Build count and binary BoW for D1=“cat cat dog” and D2=“dog fish”. Use vocabulary [cat,dog,fish] and no stopword removal.

Given

Two tokenized documents, fixed vocabulary order.

To find

Count vectors and binary vectors.

Formula and why it applies

Count BoW uses \"C(v_j,d)\". Binary BoW is 1 when that count is positive and 0 otherwise.

Step 1. D1 counts

cat=2, dog=1, fish=0 → [2,1,0].

Step 2. D2 counts

cat=0, dog=1, fish=1 → [0,1,1].

Step 3. Convert to presence

D1 [1,1,0]; D2 [0,1,1].

Answer. Count rows [2,1,0], [0,1,1]; binary rows [1,1,0], [0,1,1].

Question. After lowercasing and removing {is,a,means}, obtain BoW vectors for D1=“Continuous learning is learning lifetime”, D2=“Continuous learning means lifetime learning”, D3=“Learning is a good practice”. Use [continuous,good,learning,lifetime,practice].

Given

Three documents and exact stopword set from the supplied slides.

To find

The document–term count matrix.

Formula and why it applies

Remove only the listed stopwords, then count every remaining token in the fixed column order.

Step 1. Clean D1

[continuous,learning,learning,lifetime] → [1,0,2,1,0].

Step 2. Clean D2

[continuous,learning,lifetime,learning] → [1,0,2,1,0].

Step 3. Clean D3

[learning,good,practice] → [0,1,1,0,1].

Step 4. Compare D1 and D2

They have identical BoW vectors despite different original wording/order.

Answer. Rows [1,0,2,1,0], [1,0,2,1,0], [0,1,1,0,1].

Question. Build a bag of bigrams for “not good but not bad” with vocabulary [not good,good but,but not,not bad]. Do not add sentence boundary tokens.

Given

Five tokens; use adjacent pairs, stride 1.

To find

The bigram count vector.

Formula and why it applies

A sequence of \"L\" tokens has \"L-1\" adjacent bigrams when no boundaries are added.

Step 1. Enumerate pairs

(not,good), (good,but), (but,not), (not,bad).

Step 2. Count each vocabulary bigram

Each appears once.

Step 3. Write vector

[1,1,1,1].

Answer. Four bigram features, each with count 1.

BoW: order bhool, word-count rakh. Bag of bigrams: neighbouring word-pair count.

Label encoding assigns each category or word a unique integer ID. One-hot encoding assigns a length-\"V\" vector with one 1 at the word's vocabulary position and zeros elsewhere.

\"\\operatorname{onehot}(w)_j=\\begin{cases}1,&j=\\operatorname{index}(w)\\\\0,&\\text{otherwise.}\\end{cases}\"

The index is an ID, not a meaning score. One-hot represents one token/category; a multiword document generally needs multiple token vectors or an aggregation scheme.

Question. Encode “dog cat dog” using vocabulary [cat,dog,fish], IDs starting at 0. Give label IDs and a one-hot vector for every token.

Given

cat→0, dog→1, fish→2. Vector columns have the same order.

To find

The ID sequence and three one-hot rows.

Formula and why it applies

Replace each word by its ID for labels; set its vocabulary position to 1 for one-hot.

Step 1. Label IDs

dog→1, cat→0, dog→1 → [1,0,1].

Step 2. First dog

[0,1,0].

Step 3. Cat

[1,0,0].

Step 4. Second dog

[0,1,0]. Repeated tokens keep the same representation.

Answer. IDs [1,0,1]; one-hot rows [0,1,0], [1,0,0], [0,1,0].

Question. Encode the city sequence Mumbai, Delhi, Mumbai, Pune using vocabulary [Delhi,Mumbai,Pune] and label IDs starting at 1.

Given

Delhi→1, Mumbai→2, Pune→3.

To find

IDs and one-hot rows.

Formula and why it applies

One-hot uses three coordinates regardless of whether label IDs begin at 0 or 1.

Step 1. IDs

[2,1,2,3].

Step 2. Mumbai rows

Each Mumbai → [0,1,0].

Step 3. Delhi and Pune

Delhi→[1,0,0]; Pune→[0,0,1].

Answer. Rows [0,1,0], [1,0,0], [0,1,0], [0,0,1].

Question. One-hot encode categories [red,green,blue] for a linear regression with an intercept. Then give dummy coding after dropping blue.

Given

Full column order [red,green,blue]. Blue is the baseline.

To find

Full and reduced codes.

Formula and why it applies

Full one-hot rows sum to 1. With an intercept this creates a linear dependency; dropping one category column gives k−1 independent dummy indicators.

Step 1. Full encoding

Categoryredgreenblue
red100
green010
blue001

Step 2. Drop blue column

Categoryredgreen
red10
green01
blue00

Step 3. Interpret zero row

[0,0] now identifies the baseline blue, not missing data.

Answer. Full width 3; reduced dummy width 2. Dropping a column is model-dependent, not a requirement for every neural embedding input.

Limitations: one-hot is large and sparse and does not express semantic similarity. Unseen words need a defined unknown-token or vocabulary-update policy. Label IDs are compact but should not be treated as an ordered scalar feature for nominal words.

Label = ID number. One-hot = us ID ke box mein 1, baaki 0.

","m2":false,"units":"2","marks":null,"sourcePages":[1],"historical":true},{"id":"q1b","title":"Q1b · Lesk WSD","html":"
Question. Discuss Lesk word-sense disambiguation with a suitable example. [individual allocation not printed; Q1 totals 20]

Answer

Lesk is a knowledge-based WSD method that chooses senses whose dictionary descriptions have the greatest overlap with contextual evidence. Original Lesk compares definitions of neighbouring words' senses. A common simplified version compares each target-sense gloss directly with the sentence context. State which version you use.

Simplified Lesk procedure

  1. Remove the target word from the context; tokenize and normalize the remaining content words.
  2. Obtain the gloss for each candidate sense and apply the same normalization.
  3. Count shared words between the context and each gloss.
  4. Select the sense with maximum overlap. If scores tie, report the tie or use a specified tie-break rule.
\"\\operatorname{score}(s)=|C\\cap

\"C\" is the set of context words, \"G_s\" the set of words in sense \"s\"'s gloss, \"\\cap\" means common elements, \"|\\cdot|\" means count, and \"\\arg\\max\" means “the sense giving the largest score”. This version counts unique matching words once.

Question. Use simplified Lesk to disambiguate “bank” in “The river water rose near the bank.” Use C = {river, water, rose, near}; financial gloss = {institution, money, deposit}; river gloss = {land, river, water, edge}.

Given

The normalized context and two gloss sets are given. Count exact shared words.

To find

The score of each sense and the selected meaning.

Formula and why it applies

Use \"\\operatorname{score}(s)=|C\\cap because the task specifies context–gloss overlap.

Step 1. Compare the financial sense

No word occurs in both {river, water, rose, near} and {institution, money, deposit}. Overlap = {}.

Step 2. Count the financial score

\"\\operatorname{score}(F)=0\".

Step 3. Compare the river sense

Shared words are {river, water}. “Rose” and “near” are not in the gloss.

Step 4. Count and select

\"\\operatorname{score}(R)=2\". Since \"2>0\", choose the river-edge sense.

Answer. Bank means land beside a river.

Question. Use simplified Lesk for “The dog barked loudly.” Treat barked as bark. Context C = {dog, loudly}; sound gloss = {sharp, cry, dog}; tree gloss = {outer, covering, tree}; ship gloss = {sailing, ship}.

Given

Three glosses and normalized context {dog, loudly}.

To find

Select the sense of bark.

Formula and why it applies

\"\\operatorname{score}(s)=|C\\cap. Lemmatization lets barked match the target lemma bark; it does not create additional overlap words.

Step 1. Sound sense

Intersection = {dog}; score = 1.

Step 2. Tree sense

Intersection = {}; score = 0.

Step 3. Ship sense

Intersection = {}; score = 0.

Step 4. Compare

The sound sense has the unique maximum: 1 > 0.

Answer. Bark is the dog's cry.

Question. Apply definition–definition Lesk to “pine cone”. Pine senses: P1 = {evergreen, tree, needle, leaf}; P2 = {waste, sorrow, illness}. Cone senses: C1 = {solid, shape, circular}; C2 = {fruit, evergreen, tree}.

Given

Two candidate senses for each word. Tokens are already normalized; tree and trees would be reduced to the same form if necessary.

To find

The most compatible pair of senses.

Formula and why it applies

Use \"|G_{P_i}\\cap for every pair because this is original-style gloss–gloss comparison.

Step 1. P1 with C1

No shared word → 0.

Step 2. P1 with C2

Shared {evergreen, tree} → 2.

Step 3. P2 with each cone sense

P2–C1 has no overlap → 0; P2–C2 has no overlap → 0.

Step 4. Choose the best pair

The four scores are 0, 2, 0, 0. Maximum 2 belongs to P1–C2.

Answer. Pine is the evergreen tree; cone is its fruit-like reproductive structure.

Limitations: exact overlap misses synonyms, short glosses may produce zeros, and different preprocessing choices change the score. Extended Lesk may include related synset definitions or longer phrase overlaps; do not silently substitute it for a simple question.

Lesk: context aur dictionary mein common words gino; sabse bada overlap jeeta.

","m2":false,"units":"5","marks":null,"sourcePages":[1],"historical":true},{"id":"q1c","title":"Q1c · Syntax parsing approaches","html":"
Question. Explain syntax parsing and compare top-down and bottom-up approaches. [individual allocation not printed; Q1 totals 20]

Answer

A context-free grammar (CFG) specifies a language using production rules whose left side is a single nonterminal. Formally, G=(N,Σ,R,S): N is the set of nonterminals, Σ the terminal vocabulary, R the production rules, and S the start symbol. A rule can expand its nonterminal regardless of the neighboring symbols; that is the meaning of “context-free”.

ConceptPlain explanationExample
TerminalA word at a leaf; it is not expanded furtherthe, girl, read
NonterminalA grammatical category that can be expandedS, NP, VP, Det, N, V
ProductionA permitted replacementS → NP VP
Start symbolCategory for a complete sentenceS
LexiconWord-to-category rulesN → girl | paper
DerivationSequence of rule applicationsS ⇒ NP VP ⇒ Det N VP ⇒ …
Parse treeHierarchical picture of those expansionsRoot S, internal phrases, word leaves
\"S\\to

Read S→NP VP as “a sentence can contain a noun phrase followed by a verb phrase”. It does not mean subtraction or multiplication. A vertical bar in a grammar rule means alternatives: Det→the | a allows either the or a. This is different from the probability bar meaning “given”.

S: the whole sentence
→
NP: the girl + VP: read the paper
→
NP→Det N; VP→V NP
→
Leaves in order: the / girl / read / the / paper

A constituent is a group of words that acts as a grammatical unit, such as “the girl”. Syntactic parsing assigns a grammatical structure to an input sentence using a grammar. It can support grammar checking, information extraction, translation and later semantic analysis. Failure under one restricted grammar means that grammar could not derive the sentence; it does not necessarily prove the sentence is invalid English.

A treebank is a corpus annotated with parse trees. It supplies training/evaluation structures for parsers and lets linguists study recurring syntactic patterns. A tagged corpus gives word tags; a treebank additionally records relationships among phrases.

RepresentationExample information retained
Bag of wordsWords occur, but word order/grammar is lost
POS tagsEach word's local grammatical category
ChunksShort phrase boundaries
Parse treeNested constituent structure of the whole sentence

Yaad rakho: Grammar rule ek expansion instruction hai: S ko NP aur VP mein kholo.

Top-down parsing starts at the grammar's start symbol S and expands nonterminals until the leaves match the input words in order. Recursive descent commonly implements each nonterminal as a procedure that tries its productions. It builds from the root toward the leaves.

Start with S
→
Choose an allowed S production
→
Expand its next nonterminal
→
Match the resulting terminal to the input
→
Continue, or backtrack if the choice fails

Algorithm. Start with S as the current goal. Choose a production for the leftmost unexpanded nonterminal. Replace that nonterminal by the production's right side. If a terminal appears, compare it with the next input token. If it matches, consume the token; if it does not, reject that branch and try another production. Accept only when the complete input is consumed and all grammar goals are satisfied.

A leftmost derivation always expands the leftmost remaining nonterminal first. Top-down parsing can make guesses that the actual words cannot support, so it may need backtracking. A naive recursive-descent parser can loop on left-recursive rules such as NP→NP PP unless the grammar or algorithm is changed. Predictive LL(1) parsing chooses productions using one lookahead token when the grammar permits; not every English CFG is LL(1).

ApproachBegins withDirectionCommon difficulty
Top-downStart symbol SRoot to word leavesPredictions that do not match input; left recursion
Bottom-upActual input tokensLeaves to root SReductions that cannot contribute to a complete sentence

Yaad rakho: Top-down: sentence ka structure pehle, words tak neeche aao.

Bottom-up parsing starts from the input tokens and combines them into larger constituents until the start symbol is reached. A shift-reduce parser maintains a stack of recognized material and a list of remaining input tokens.

ActionExact operationExample
ShiftMove the next input token onto the stackStack: the; remaining input: dog runs
ReduceReplace a suffix of the stack matching a rule's RHS with its LHSDet N becomes NP using NP→Det N
AcceptStack is S and input is exhaustedStack: S; input: empty
Fail/backtrackChosen reductions cannot complete a parseA noun reading of book blocks an imperative parse

Reduction uses a production in reverse: the grammar NP→Det N expands an NP top-down, but reduces Det N into NP bottom-up. The stack's rightmost item is its top in the traces below. Each row shows the state after the listed action, avoiding an off-by-one mismatch between actions and stack contents.

A lexical reduction such as N→dog still counts as a reduction. Do not skip the sequence from word to category to phrase when the question asks for a complete trace. Backtracking means returning to an earlier choice and trying another applicable rule, not changing the original input.

Yaad rakho: Shift = input se stack. Reduce = chhote pieces se bada grammatical unit.

","m2":false,"units":"4","marks":null,"sourcePages":[1],"historical":true},{"id":"q1d","title":"Q1d · NLU versus NLG and applications","html":"
Question. Compare Natural Language Understanding and Natural Language Generation. List real-world NLP applications that use these concepts. [individual allocation not printed; Q1 totals 20]

Answer

Natural language understanding (NLU) maps language into an interpretation useful for a task: intents, entities, semantic relations or another meaning representation. Natural language generation (NLG) produces human-language output from data, a representation or another source.

“Book a ticket to Pune tomorrow.”
→
NLU: intent=book; destination=Pune; date=tomorrow
→
Application: search available journeys
→
NLG: “Here are the available journeys to Pune.”
FeatureNLUNLG
DirectionLanguage → interpretationData/representation → language
Main concernMeaning, intent, ambiguityContent choice, organization, fluent wording
ExampleFind destination in a requestWrite the booking confirmation
Common failureWrong destination or senseUnclear, unsupported or ungrammatical output

NLG stages in a classical pipeline

  1. Content determination: choose facts to communicate.
  2. Data interpretation: identify relationships or trends.
  3. Document planning: organize facts in a useful order.
  4. Sentence aggregation: combine related facts without repetition.
  5. Lexical and grammatical realization: choose words and construct grammatical sentences.
  6. Presentation: format the generated message for its audience.

For a weather report, facts {city=Pune, temperature=28°C, rain=yes} can become “Pune will have rain with a temperature of 28°C.” NLU would instead extract facts or intent from an incoming language message.

NLU = language se meaning andar. NLG = data se language bahar.

A question-answering assistant interprets intent/entities through NLU and composes an answer through NLG. Translation interprets source meaning and generates a target-language sequence. A customer-support system classifies requests and generates a contextual reply.

","m2":false,"units":"1,7","marks":null,"sourcePages":[1],"historical":true},{"id":"q2a","title":"Q2a · TF–IDF cosine after a fixed stoplist","html":"
Question. For “The cat sat on the mat.”, “The dog barked at the cat.” and “I love my dog and my cat.” remove stopwords {I,the,on,my,at,and}; construct TF–IDF and calculate cosine similarity between the first two documents. [10 marks]

Answer

Given

DocumentOriginal text
1The cat sat on the mat.
2The dog barked at the cat.
3I love my dog and my cat.

Stopwords: at, my, i, on, the, and

To find

BoW, IDF and TF–IDF plus requested cosine

Formula: which one and why

Use lowercase alphabetic tokens and exactly the stated stoplist. If no stoplist is supplied, retain function words. Adopt raw-count TF and natural-log IDF; the paper does not prescribe another TF/log convention.

\"\\mathrm{TFIDF}(w,d)=c(w,d)\\ln\\frac{N}{\\mathrm{df}(w)}\"

A term shared by all N documents has IDF ln(1)=0. Counts and TF–IDF weights are different representations.

Step 1. Tokenize each document

DocumentTokens
1cat, sat, mat
2dog, barked, cat
3love, dog, cat

Step 2. Fix the vocabulary order

barked, cat, dog, love, mat, sat

Step 3. Count occurrences (Bag of Words)

Documentbarkedcatdoglovematsat
1010011
2111000
3011100

Step 4. Calculate document frequency and IDF

TermdfN / dfln(N/df)
barked131.098612289
cat310.000000000
dog23/20.405465108
love131.098612289
mat131.098612289
sat131.098612289

Step 5. Multiply each TF by its IDF

Documentbarkedcatdoglovematsat
10.0000000.0000000.0000000.0000001.0986121.098612
21.0986120.0000000.4054650.0000000.0000000.000000
30.0000000.0000000.4054651.0986120.0000000.000000

Step 6. Calculate cosine from these same TF–IDF rows

\"\\cos(u,v)=\\frac{\\sum_i

Coordinate products: 0.000000000, 0.000000000, 0.000000000, 0.000000000, 0.000000000, 0.000000000

Dot product=0.000000000; squared norms=2.413897922, 1.371350915. Cosine=0.000000000

Final answer

The final table is the TF–IDF matrix in the declared vocabulary order. A different explicitly stated log base rescales all IDFs; length-normalized TF changes each document scale.

Cosine is 0.0. Shared words whose IDF is zero contribute nothing to the TF–IDF dot product.

Which terms are prominent?

D1: sat and mat have IDF ln3; D2: barked has ln3 and dog has ln(3/2); D3: love has ln3 and dog has ln(3/2). Cat occurs in all three documents and has zero unsmoothed IDF. Thus shared cat does not make D1/D2 TF–IDF cosine positive.

","m2":false,"units":"2","marks":10,"sourcePages":[1,2],"historical":true},{"id":"q2b","title":"Q2b · Weather Naive Bayes","html":"
Question. Use the supplied class/conditional probabilities to predict Play Tennis for Outlook=Sunny, Temperature=Cool, Humidity=High, Windy=True. [10 marks]

Answer

Given

ClassPriorSunnyCoolHighTrue
Yes9/142/93/93/93/9
No5/143/51/54/53/5

To find

Compare the Yes and No class scores.

Formula: which one and why

\"\\mathrm{score}(c)=P(c)P(Sunny\\mid

Step 1. Multiply the Yes factors

9/14 × 2/9 × 3/9 × 3/9 × 3/9 = 1/189 = 0.005291005

Step 2. Multiply the No factors

5/14 × 3/5 × 1/5 × 4/5 × 3/5 = 18/875 = 0.020571429

Step 3. Compare and normalize

Yes=0.204582651, No=0.795417349

Final answer

Predict No. Apply the supplied categorical probabilities directly; adding vocabulary smoothing would solve a different problem.

","m2":false,"units":"7","marks":10,"sourcePages":[2],"historical":true},{"id":"q3a","title":"Q3a · Coreference and anaphora","html":"
Question. Explain coreference and anaphoric reference with examples. [5 marks]

Answer

Pragmatic analysis interprets an utterance using its situation and the speaker’s intended meaning. Discourse analysis studies how utterances connect across a text or conversation. Syntax and literal word meanings alone cannot settle every interpretation.

TextLiteral contentContextual interpretation
“Can you close the window?”A question about ability.Usually a polite request.
“Meera bought a laptop. It was expensive.”Two sentences.“It” connects the second sentence to the laptop.
“Meet me here tomorrow.”An instruction.Me, here and tomorrow depend on speaker, place and date.

Reference, mention and antecedent

A referring expression points to an entity; a mention is its occurrence in the text. Its referent is the entity being discussed. An antecedent is a preceding expression used to interpret a later referring expression. Reference resolution identifies the intended referent. Coreference resolution groups mentions that refer to the same entity.

“Meera”name mention
→
Person entitysame referent
→
“she” / “the engineer”other mentions
ConceptDefinitionExample
AnaphoraReference back to something introduced earlier.“Riya arrived. She sat down.” She → Riya.
CataphoraReference forward to a later expression.“Before she spoke, Riya checked the microphone.” she → Riya.
ExophoraReference to the situation outside the text.“Put it there”, said while pointing.
CoreferenceTwo or more mentions refer to the same entity.“Dr Rao”, “the lecturer”, “she”.
Pronoun resolutionFind the referent of a pronoun.Resolve “it” in “The laptop stopped because it overheated.”

Coreference is broader than pronoun resolution: repeated names and descriptions also form a chain. Anaphora describes direction/dependence; coreference describes identity. In “I bought a car. The engine is noisy”, engine is related to the car but is not the same entity.

Yaad rakho: Reference = kiski baat? Coreference = alag mentions, same entity. Pronoun resolution = he/she/it kis se linked?

Cohesion, coherence and discourse markers

\n

Cohesion is the explicit linguistic linking of a text through reference, repeated or related words, conjunctions and other devices. In “Rita bought a book. She read it”, she and it connect the second sentence to Rita and the book.

\n

Coherence is the conceptual unity that makes a discourse meaningful. “The road flooded, so the match was cancelled” presents a plausible cause and consequence. A reader relates the propositions using context and knowledge.

\n

Discourse markers signal relationships or organize speech: however signals contrast; therefore signals a result; first and next signal sequence; for example introduces an illustration.

\n

These concepts differ: “I like apples. Therefore the moon is square” has a linking word but no sensible inference. A connector alone does not guarantee coherence.

\n

Yaad rakho: Cohesion = words se link; coherence = meaning se link; marker = relation ka signal.

A practical resolution procedure

First detect the pronoun and candidate noun phrases. Eliminate candidates inconsistent with person, number, grammatical binding or relevant semantic constraints. Rank the surviving candidates using syntactic prominence, recency and discourse context. Preserve ambiguity if the available evidence does not decide.

ApproachHow it worksStrengthLimitation
Rule based / syntacticUse parse trees, agreement and explicit search order; Hobbs is a classic example.Can explain each decision without labelled training data.Parsing errors and world knowledge gaps cause errors.
Salience basedScore candidates using recency, subjecthood and other discourse features; select the strongest compatible candidate.Captures the tendency to refer again to prominent entities.Feature weights and competing candidates can mislead.
Supervised statisticalLearn candidate/pronoun compatibility from labelled examples.Combines many features from data.Needs representative annotated examples.
Neural / contextualEncode the surrounding text and score mention pairs or antecedents.Can learn contextual patterns and long-range dependencies.Needs data/compute and can still make confident mistakes.

Worked analysis 1 · number and semantics

Resolve “they” in: “Riya put the books on the shelf because they were heavy.”

Given

Candidates: Riya (singular person), books (plural objects), shelf (singular object); pronoun they is plural here.

To find

The antecedent and a justified resolution procedure.

Formula and why it applies

Use number agreement first, then semantic compatibility. There is no multiplication formula in this linguistic task.

Solution

Step 1. List candidates

Riya, the books, the shelf.

Step 2. Apply number agreement

Riya and shelf are singular. Books is plural, matching they.

Step 3. Check meaning

Books can be heavy; the causal explanation fits carrying/placing books.

“They” refers to “the books”.

Worked analysis 2 · a complete coreference chain

Identify the coreference chain in: “Dr Sen entered. The surgeon greeted Ravi. She opened her notebook.”

Given

Mentions: Dr Sen, the surgeon, Ravi, she, her, her notebook. Assume the surgeon is Dr Sen and she/her refers to that surgeon.

To find

The person chain, and a mention that must not be placed in it.

Formula and why it applies

Group expressions by identity of referent; a possessive expression and the owned object are different entities.

Solution

Step 1. Link the name and description

Dr Sen and the surgeon denote the same person under the given context.

Step 2. Link the pronouns

She and her refer to the surgeon.

Step 3. Keep the notebook separate

“Her notebook” is an object owned by the person; it is not the person herself.

Person chain: {Dr Sen, the surgeon, She, her}. “Her notebook” is a separate object mention.

Worked analysis 3 · honest ambiguity

Discuss the pronoun in: “Asha called Nisha after she finished work.”

Given

Both named candidates are singular; the sentence gives no further context.

To find

Whether one referent is logically guaranteed.

Formula and why it applies

Agreement filters incompatible candidates but cannot always select between equally compatible candidates.

Solution

Step 1. Check agreement

Both Asha and Nisha are compatible with she.

Step 2. Check the event

Either person could have finished work.

Step 3. State the limitation

A heuristic may prefer a candidate, but grammar and this sentence alone do not prove the intended referent.

Ambiguous without further context. State any heuristic and parse assumption before giving its preferred candidate.
","m2":false,"units":"6","marks":5,"sourcePages":[2],"historical":true},{"id":"q3b","title":"Q3b · Cohesion, coherence and discourse markers","html":"
Question. Explain cohesion, coherence and discourse markers. [5 marks]

Answer

Pragmatic analysis interprets an utterance using its situation and the speaker’s intended meaning. Discourse analysis studies how utterances connect across a text or conversation. Syntax and literal word meanings alone cannot settle every interpretation.

TextLiteral contentContextual interpretation
“Can you close the window?”A question about ability.Usually a polite request.
“Meera bought a laptop. It was expensive.”Two sentences.“It” connects the second sentence to the laptop.
“Meet me here tomorrow.”An instruction.Me, here and tomorrow depend on speaker, place and date.

Reference, mention and antecedent

A referring expression points to an entity; a mention is its occurrence in the text. Its referent is the entity being discussed. An antecedent is a preceding expression used to interpret a later referring expression. Reference resolution identifies the intended referent. Coreference resolution groups mentions that refer to the same entity.

“Meera”name mention
→
Person entitysame referent
→
“she” / “the engineer”other mentions
ConceptDefinitionExample
AnaphoraReference back to something introduced earlier.“Riya arrived. She sat down.” She → Riya.
CataphoraReference forward to a later expression.“Before she spoke, Riya checked the microphone.” she → Riya.
ExophoraReference to the situation outside the text.“Put it there”, said while pointing.
CoreferenceTwo or more mentions refer to the same entity.“Dr Rao”, “the lecturer”, “she”.
Pronoun resolutionFind the referent of a pronoun.Resolve “it” in “The laptop stopped because it overheated.”

Coreference is broader than pronoun resolution: repeated names and descriptions also form a chain. Anaphora describes direction/dependence; coreference describes identity. In “I bought a car. The engine is noisy”, engine is related to the car but is not the same entity.

Yaad rakho: Reference = kiski baat? Coreference = alag mentions, same entity. Pronoun resolution = he/she/it kis se linked?

Cohesion, coherence and discourse markers

\n

Cohesion is the explicit linguistic linking of a text through reference, repeated or related words, conjunctions and other devices. In “Rita bought a book. She read it”, she and it connect the second sentence to Rita and the book.

\n

Coherence is the conceptual unity that makes a discourse meaningful. “The road flooded, so the match was cancelled” presents a plausible cause and consequence. A reader relates the propositions using context and knowledge.

\n

Discourse markers signal relationships or organize speech: however signals contrast; therefore signals a result; first and next signal sequence; for example introduces an illustration.

\n

These concepts differ: “I like apples. Therefore the moon is square” has a linking word but no sensible inference. A connector alone does not guarantee coherence.

\n

Yaad rakho: Cohesion = words se link; coherence = meaning se link; marker = relation ka signal.

Definitions and examples requested in this part

TermDefinitionExample
CohesionExplicit language links connecting text: pronouns, repetition, conjunctions, lexical tiesRita bought a book. She read it. She/it refer back to Rita/book.
CoherenceConceptual unity and plausible relations making the text meaningfulThe road flooded, so the match was cancelled: cause and consequence connect the propositions.
Discourse markerExpression signalling relation or organizing the discoursehowever=contrast; therefore=result; first/next=sequence; for example=illustration

“I like apples. Therefore the moon is square” contains a connector but lacks a sensible inference: surface cohesion does not guarantee coherence.

","m2":false,"units":"6","marks":5,"sourcePages":[2],"historical":true},{"id":"q3c","title":"Q3c · Question answering system","html":"
Question. Explain a question-answering system with its architecture and an example. [10 marks]

Answer

Question answering (QA) returns an answer to a natural-language question using available evidence or knowledge. Information retrieval usually returns ranked documents; QA must identify the requested information and deliver the answer. A relevant document is useful evidence but is not automatically the answer.

Question types and answer types

QuestionExpected answer typeRequired operation
Who founded the company?Person or organisationFind the founder relation
When did the museum open?Date/timeLocate the opening date
Where is the campus?LocationLocate the campus-location relation
How many books were issued?QuantityRead or compute the stated count
Why was the train cancelled?Reason/explanationFind the causal evidence
Is the office open on Sunday?Yes/no with evidenceCheck the relevant schedule statement

Closed-domain QA operates within a restricted area such as a university handbook. Open-domain QA addresses a wider range of subjects and usually needs broad retrieval. Extractive QA selects a span from a passage. Generative QA constructs answer text; it still must preserve the evidence and avoid unsupported details.

Question analysis
→
Evidence retrieval
→
Passage/record understanding
→
Answer selection or composition
→
Support check
→
Present answer

Explain each module

ModuleWhat happensWhy it matters
Question analysisIdentify topic, intent, constraints and answer type.When asks for a time, not a place.
RetrievalFind a relevant passage or structured fact.Reduces the search space.
Evidence understandingParse relations, resolve references and ambiguous words.The correct date must belong to the correct event.
Answer formationExtract a span or compose a short supported response.Turns a document into an actual answer.
VerificationCheck whether every claimed detail follows from evidence.Prevents filling gaps by guessing.
PresentationReturn a concise answer with suitable context.Avoids dumping unrelated text.

Worked application 1 · extract the correct fact

Question. Given “The City Museum opened in 1998 in Pune. It was renovated in 2016”, answer “When did the museum open?” Show the QA stages.

Given

A two-sentence passage. Dates 1998 and 2016 refer to different events.

To find

The opening date, not the renovation date.

Method and why it applies

Use extractive QA: identify the requested event and expected date type, then choose the span linked to that event.

Solution

Step 1. Analyse the question

When requires a date/time answer. The event is opening; the entity is the City Museum.

Step 2. Retrieve the relevant sentence

The first sentence mentions the museum and its opening. The second sentence discusses renovation.

Step 3. Identify event–date relations

EventDate
opened1998
renovated2016

Step 4. Select the answer span

1998 is attached to opened, so select the four-character year span 1998.

Step 5. Reject the distractor

2016 is also a date, but it answers a different question: when was the museum renovated?

Step 6. Present the answer

“The museum opened in 1998.”

Answer. 1998. Matching only the answer type date would be insufficient without matching the event.

Worked application 2 · resolve a pronoun before answering

Question. Given “Mira joined Orion Labs in 2022. She became its research lead in 2024”, answer “Who became Orion Labs' research lead, and when?”

Given

The passage contains the name Mira, pronoun She, organisation Orion Labs and possessive its. Assume the normal reading that She refers to Mira and its to Orion Labs.

To find

The person and the date of becoming research lead.

Method and why it applies

Combine reference resolution with relation extraction. The question asks for two fields, both tied to the promotion/role event.

Solution

Step 1. Identify requested fields

Who → person; when → date. The relation is became research lead of Orion Labs.

Step 2. Resolve mentions

MentionReferent
SheMira
itsOrion Labs

Step 3. Read the target event

After resolution, the second sentence states that Mira became Orion Labs' research lead in 2024.

Step 4. Distinguish the dates

2022 belongs to joining the organisation. 2024 belongs to becoming research lead.

Step 5. Compose a supported answer

“Mira became Orion Labs' research lead in 2024.”

Step 6. Verify completeness

The answer supplies both requested fields and does not add an unsupported department, qualification or reason.

Answer. Mira, in 2024. Reference resolution connects the second sentence to the named person and organisation.

Worked application 3 · detect insufficient evidence

Question. A handbook excerpt states “The library opens at 8 a.m. on weekdays.” A user asks “At what time does it close on Sunday?” Explain the correct QA response.

Given

Only the opening time for weekdays is supplied. No closing time or Sunday schedule is present.

To find

Whether an answer is supported and how to handle the missing information.

Method and why it applies

Compare every question constraint against the evidence. Topic overlap with library is not sufficient: time type and day must also match.

Solution

Step 1. Extract the question constraints

ConstraintRequested
EntityLibrary
EventClosing
DaySunday
Answer typeTime

Step 2. Extract the supplied fact

Entity=library; event=opening; day=weekdays; time=8 a.m.

Step 3. Identify mismatches

Opening is not closing. Weekdays do not supply a Sunday schedule.

Step 4. Avoid the unsupported substitution

Do not answer 8 a.m. just because it is the only time in the passage.

Step 5. Return an evidence-limited response

“The excerpt does not state the library's Sunday closing time.” A system with further retrieval can search a relevant schedule.

Step 6. State the benefit

The system avoids converting a relevant-looking number into an incorrect answer.

Answer. No Sunday closing time is supported by the excerpt. Report insufficient evidence or retrieve the missing schedule.

Advantages, limits and evaluation

Design choiceBenefitLimitation
Extractive answerThe returned wording comes directly from evidence.The answer may require combining sentences or may not be present as one span.
Generative answerCan combine supported facts into a readable response.Can introduce unsupported information unless carefully checked.
Closed-domain collectionControlled terminology and relevant resources.Cannot answer facts outside its knowledge source.
Broad retrievalWider question coverage.Irrelevant, outdated or conflicting sources can be retrieved.

Evaluate exact match when a canonical answer string is expected, token overlap/F1 when wording may vary, and factual correctness and completeness when answers are composed. An answer can be grammatically fluent but wrong. Questions with no answer in the evidence should be included in evaluation so the system's restraint is also tested.

For a longer exam answer, define QA, distinguish it from retrieval, explain answer types and system variants, draw the pipeline, work through a passage including distractors, and discuss ambiguity, missing evidence and evaluation. A list of chatbot names is not a QA design.

Yaad rakho: Question ka entity + event + constraint match karo. Passage mein koi bhi number mil gaya matlab answer nahi.

","m2":false,"units":"7","marks":10,"sourcePages":[2],"historical":true},{"id":"q4a","title":"Q4a · NLP pipeline","html":"
Question. Explain the stages of an NLP pipeline. [5 marks]

Answer

An NLP pipeline is an organized sequence of operations that transforms raw language into a task output and checks whether that output is useful. Not every task needs every cleaning operation.

Data acquisition
→
Preprocessing
→
Feature representation
→
Model
→
Evaluation
→
Deployment and monitoring
  1. Data acquisition: collect relevant text and, when needed, labels.
  2. Cleaning and preprocessing: handle markup, tokenize, normalize and decide which information to retain.
  3. Feature engineering: represent text numerically using BoW, TF–IDF or embeddings.
  4. Modelling: apply rules or train a statistical/neural model.
  5. Evaluation: test on held-out data using task-appropriate measures.
  6. Deployment: make the system available, log suitable feedback and monitor errors or data changes.

Concrete spam-filter example

Collect labelled emails → separate train/test data → tokenize text while preserving meaningful terms → fit vocabulary and IDF on training data → train classifier → evaluate spam detection on the held-out set → deploy an email service filter → monitor legitimate emails wrongly blocked.

Intrinsic evaluation measures model/task quality, such as precision, recall or tagging accuracy. Extrinsic evaluation measures effect in the wider application, such as whether users find answers faster. A higher internal metric does not automatically guarantee a better user outcome.

Data leakage: learning vocabulary statistics, labels or preprocessing decisions from the test set can make reported performance too optimistic. Fit learned transformations on training data, then apply them to validation/test data.

Collect → clean → numbers → model → test → use. Test data ko training ki information mat banao.

Definition. NLP is the study of computational methods for analyzing, understanding and generating human language. The pipeline diagram above turns raw language into a task-specific prediction or generated response.

","m2":false,"units":"1","marks":5,"sourcePages":[2],"historical":true},{"id":"q4b","title":"Q4b · Morphology and syntax","html":"
Question. Explain the importance of morphology and syntax in NLP. [5 marks]

Answer

A phoneme is a sound unit that can distinguish meanings in a language, such as the /p/ and /b/ contrast in “pat” and “bat”. A phoneme is a sound, not necessarily one alphabet letter. A morpheme is the smallest meaning-bearing grammatical unit: cats = cat + plural -s; unhappy = un- + happy.

A lexeme is an abstract vocabulary item grouping its inflected forms: write, writes, wrote and writing belong to WRITE. A phrase is a group of words functioning together; a clause normally contains a predicate and its associated participants; a sentence expresses a complete grammatical unit.

LevelWhat it analysesConcrete example
Phonetics / phonologyPhysical sounds / sound patterns in a languageDistinguish bat from pat
Morphology / lexical analysisWord structure and lexical itemsun + help + ful → unhelpful
SyntaxGrammatical arrangement and relationshipsThe dog chased the cat: subject, verb, object
SemanticsMeaning of words and composed expressionsThe dog is the chaser; the cat is chased
DiscourseConnections across sentences“Riya arrived. She smiled.” → She refers to Riya
PragmaticsIntended meaning in the situation“It is cold here” may request closing the window
word forms
→
grammatical structure
→
sentence meaning
→
cross-sentence context
→
speaker intention

Three worked interpretations

“The cats are sleeping.” Morphology identifies cat + s as plural. Syntax connects “cats” to “are sleeping”. Semantics represents animals engaged in sleeping.

“Aman bought a laptop. It was expensive.” Sentence-level syntax alone does not identify “it”. Discourse connects “it” to the laptop across the sentence boundary.

“Wonderful, another power cut!” The word wonderful is positive literally, but situational context can indicate sarcasm and frustration. This is a pragmatic interpretation.

The analysis levels are conceptual descriptions. Modern systems may learn several jointly rather than executing a compulsory separate module for every level.

Morphology = word ke andar; syntax = sentence ka structure; semantics = meaning; discourse = sentences ka link; pragmatics = intention.

Morphology analyzes internal word structure: unreadable→un+read+able, cats→cat+plural. Syntax analyzes how words combine: subject–verb–object relations distinguish “dog bites man” from “man bites dog”. Morphological normalization supports vocabulary matching; syntax preserves grammatical relations needed for parsing, extraction and translation.

Translation needs morphological number/tense agreement and target-language word order. Sentiment needs syntax to distinguish “not good” from “good” and morphology to relate disappointed/disappointing to their base form without discarding the negative meaning.

","m2":false,"units":"1","marks":5,"sourcePages":[2],"historical":true},{"id":"q4c","title":"Q4c · Most probable bigram continuations","html":"
Question. Train on <s> I love programming </s>; <s> Programming is fun </s>; <s> I love coding </s>; <s> Coding is great </s>. Predict next tokens for <s> I love ..., <s> programming ..., <s> coding ... using a bigram model. [10 marks]

Answer

Given

<s> I love programming </s>
<s> Programming is fun </s>
<s> I love coding </s>
<s> Coding is great </s>

To find

Most probable next token for each prefix.

Formula: which one and why

\"P(w\\mid

Lowercase lexical tokens; include sentence-end as an observed next token. Bigram prediction uses only the last token of the prefix.

Step 1. Count each adjacent pair separately within sentences

BigramCount
<s> → coding1
<s> → i2
<s> → programming1
coding → </s>1
coding → is1
fun → </s>1
great → </s>1
i → love2
is → fun1
is → great1
love → coding1
love → programming1
programming → </s>1
programming → is1

Step 2. Continue <s> I love

CandidateCount / history countProbability
coding1/20.500000
programming1/20.500000

Highest: programming, coding. This is a tie; no tie-break rule is supplied.

Step 3. Continue <s> programming

CandidateCount / history countProbability
</s>1/20.500000
is1/20.500000

Highest: </s>, is. This is a tie; no tie-break rule is supplied.

Step 4. Continue <s> coding

CandidateCount / history countProbability
</s>1/20.500000
is1/20.500000

Highest: </s>, is. This is a tie; no tie-break rule is supplied.

Final answer

<s> I love → programming or coding
<s> programming → </s> or is
<s> coding → </s> or is

If an examiner explicitly restricts the answer to a lexical word, exclude END and state that restricted candidate set. Do not silently remove an END tie.

","m2":false,"units":"3","marks":10,"sourcePages":[2,3],"historical":true},{"id":"q5a","title":"Q5a · Preprocess the NLP passage","html":"
Question. Perform the requested text preprocessing on: “Natural Language Processing (NLP) is a field of artificial intelligence that focuses on the interaction between computers and humans through natural language. The ultimate objective of NLP is to read, decipher, understand, and make sense of human languages in a valuable way.” Remove the supplied stopwords {is,a,of,that,on,I,the,and,to,in}. [6 marks]

Answer

StageResult
Sentence segmentationNatural Language Processing (NLP) is a field of artificial intelligence that focuses on the interaction between computers and humans through natural language.
The ultimate objective of NLP is to read, decipher, understand, and make sense of human languages in a valuable way.
Word tokens (punctuation omitted)Natural, Language, Processing, NLP, is, a, field, of, artificial, intelligence, that, focuses, on, the, interaction, between, computers, and, humans, through, natural, language, The, ultimate, objective, of, NLP, is, to, read, decipher, understand, and, make, sense, of, human, languages, in, a, valuable, way
Lowercasenatural, language, processing, nlp, is, a, field, of, artificial, intelligence, that, focuses, on, the, interaction, between, computers, and, humans, through, natural, language, the, ultimate, objective, of, nlp, is, to, read, decipher, understand, and, make, sense, of, human, languages, in, a, valuable, way
Declared stoplista, and, i, in, is, of, on, that, the, to
After exact stop removalnatural, language, processing, nlp, field, artificial, intelligence, focuses, interaction, between, computers, humans, through, natural, language, ultimate, objective, nlp, read, decipher, understand, make, sense, human, languages, valuable, way
OperationWhy it is useful
Sentence segmentationDefines separate units for parsing and prevents counts from crossing sentence boundaries.
LowercasingMerges case variants when capitalization is not needed; retain original case for NER when useful.
Stop removalCan remove frequent low-information words for a bag-of-words retrieval task; it is task-dependent and can damage negation/grammar.
TokenizationDefines the units whose counts/features are computed; punctuation and abbreviation conventions must be consistent.
","m2":false,"units":"2","marks":6,"sourcePages":[3],"historical":true},{"id":"q5b","title":"Q5b · Stemming and lemmatization tools","html":"
Question. Differentiate stemming and lemmatization, naming two approaches/tools for each. [4 marks]

Answer

Stemming applies string-based rules, often stripping suffixes, to reduce related forms to a stem. The result need not be a valid dictionary word. Lemmatization uses vocabulary and morphological analysis, often with POS information, to return a dictionary lemma.

CriterionStemmingLemmatization
BasisSurface form/suffix rulesLexicon and morphological/POS analysis
OutputMay be nonword, e.g. studiDictionary form, e.g. study
Context/POSUsually little or noneOften important
Irregular formsMay miss went→goCan map went→go with verb analysis
CostGenerally simplerUsually more linguistic processing

Porter applies a sequence of suffix-removal rules subject to conditions; Snowball provides related language-specific stemming algorithms; Lancaster is often more aggressive; regex stemming uses explicit patterns; Lovins is another established stemming approach. Exact output depends on the algorithm and implementation.

Three worked reductions

  1. studies, studying: an English stemmer may yield studi/study depending on the rule sequence; verb lemmatization aims at study. State the exact algorithm if exact outputs are asked.
  2. better: adjective-aware lemmatization can map it to good; naïve suffix removal cannot infer that irregular relation.
  3. saw: as a past-tense verb it can become see; as a cutting-tool noun it remains saw. POS and context matter.

Over-stemming merges forms that should remain distinct; under-stemming fails to merge related forms. Normalization can increase recall in search while losing useful distinctions, so it is a task-dependent choice.

Stem = rules se cut; lemma = dictionary form, meaning/POS ka dhyan.

Examples of stemmers: Porter and Lancaster (Snowball is another). Examples of lemmatization tools: WordNet lemmatizer with POS information and spaCy’s language-specific lemmatizer. Outputs depend on language and supplied POS; better→good requires lexical knowledge, unlike simple suffix cutting.

","m2":false,"units":"2","marks":4,"sourcePages":[3],"historical":true},{"id":"q5c","title":"Q5c · Hobbs on John and Mary","html":"
Question. Explain Hobbs’ algorithm and resolve the pronouns in “John gave a book to Mary after he finished reading it.” [10 marks]

Answer

Hobbs’ algorithm is a syntax-based procedure that searches constituent parse trees in a fixed order for a compatible noun-phrase antecedent of a pronoun. The tree determines which candidates are proposed first. Agreement and other relevant constraints determine whether a proposed candidate can be accepted.

It is not simply the rule “choose the closest noun”. A nearby noun may have the wrong number, may be syntactically ineligible, or may be visited after a structurally more prominent noun. The algorithm produces a preference under the supplied parse; ambiguous language can still require context.

The tree vocabulary you need

TermMeaningExample
NPNoun phrase: a phrase referring to an entitythe student; Anita; they
SA sentence/clause nodethe node containing a subject and predicate
VPVerb phrasetold Ravi that she should revise
XThe NP or S node currently being searchedAn algorithm label, not a word
Path pThe branch travelled upward from the pronoun/previous XUsed to distinguish left and right subtrees
N-bar (N′)The noun-centred part of an NP in a grammar using that labelThe nominal-child test depends on the given tree
Breadth-firstVisit shallower nodes before deeper onesLevel 1 before level 2
Left-to-rightVisit equally deep nodes in their written/tree orderEarlier left branch before right branch
Locate pronoun NP
→
Climb to NP/S
→
Propose candidates in tree order
→
Check compatibility
→
Accept or continue searching

The complete search procedure

Step 1. Start at the pronoun

Locate the NP immediately dominating the pronoun. “Dominating” means containing it as a descendant in the tree.

Step 2. Climb to the first NP or S

Move upward to the first NP or S above that starting NP. Call this node X. Keep the upward path p.

Step 3. Search the left branches initially

Below X, traverse branches to the left of p breadth-first and left-to-right. At this initial search, propose an NP only if another NP or S lies between that candidate and X. Test proposed candidates for the required agreement and constraints.

Step 4. If X is the highest sentence node, inspect earlier sentences

Search previous sentence trees from most recent to oldest. Within each, use breadth-first, left-to-right order to propose NPs. Accept the first eligible compatible candidate, or report that no candidate was found in the available context.

Step 5. Otherwise climb again

Move from X to the next NP or S above it. Record this new X and the path followed during the climb.

Step 6. Apply the higher-NP test

If the new X is an NP, propose X when the path did not pass through its nominal N-bar child. Use the given parse labels; do not invent an N-bar branch missing from the question's structure.

Step 7. Search its left branches

Traverse branches below the new X to the left of the path, breadth-first and left-to-right, proposing NPs. The special intervening-NP/S restriction belongs to the initial search in Step 3.

Step 8. If X is S, search the permitted right branches

Search branches to the right of the path breadth-first and left-to-right. Do not descend below an encountered NP or S. Propose eligible NP candidates there.

Step 9. Repeat or stop

Return to the highest-S check and continue the climb/search procedure until a compatible candidate is accepted or the available trees are exhausted. Record the candidate order rather than only writing an unexplained name.

Why the first left search has an extra condition

In a simple clause such as “John saw him”, the subject NP John is directly under the clause S. There is no extra NP or S between John and that S. The initial-search restriction therefore prevents casually selecting John as the ordinary local antecedent of him. Compare the reflexive “John saw himself”, which is a different grammatical case; do not treat every pronoun type as interchangeable.

Worked tree 1 · move to the previous sentence

Question. Apply Hobbs’ search to “The student arrived. She smiled.” Assume the student is compatible with she.

Given

Previous tree: [S [NP The student] [VP arrived]]. Current tree: [S [NP She] [VP smiled]].

To find

The selected antecedent and the search path.

Method and why it applies

Use the supplied constituent trees. The current sentence has no earlier candidate, so the highest-S rule activates previous-sentence search.

Solution

Step 1. Draw both sentence structures

Previous sentence             Current sentence\nS                             S\n├── NP: The student            ├── NP: She ← start\n└── VP: arrived                └── VP: smiled

Step 2. Locate and climb

Start at NP She. Its first higher NP/S is the current S. Set X to that S.

Step 3. Search left of the path

She is the leftmost subject. No NP lies to its left in the current sentence.

Step 4. Use the highest-S rule

This X is the highest S. Inspect the immediately preceding sentence tree.

Step 5. Propose its first NP

Breadth-first, left-to-right finds NP The student.

Step 6. Check compatibility and accept

The question stipulates that the student agrees with she. Accept this candidate.

Answer. She refers to the student under the supplied assumptions.

Worked tree 2 · reject an incompatible subject

Question. Resolve they in “Riya spoke to the students. They laughed.” Use a previous-sentence search and interpret they as plural.

Given

Previous tree has subject NP Riya, then VP spoke [PP to [NP the students]]. Current subject is NP They.

To find

The candidate order and accepted antecedent.

Method and why it applies

Breadth-first search favours a shallow subject, but agreement must still be applied; a first candidate is not automatically a valid answer.

Solution

Step 1. Draw the useful part of the previous tree

S\n├── NP: Riya                 ← candidate 1\n└── VP\n    ├── V: spoke\n    └── PP\n        ├── P: to\n        └── NP: the students ← candidate 2

Step 2. Search the current sentence

From NP They climb to its S; there is no earlier NP in “They laughed”. This is the highest S, so move to the previous tree.

Step 3. Visit Riya first

Riya is the shallower leftmost NP. Under the stated plural interpretation, this singular candidate fails number agreement.

Step 4. Visit the students

Continue breadth-first. The next relevant NP is the students.

Step 5. Apply number agreement

Students is plural and is an eligible candidate. It matches plural they.

Step 6. State the resolution

Accept the students. The method did not simply accept the first noun it saw.

Answer. They → the students. Candidate order: Riya rejected; the students accepted.

Worked tree 3 · climb out of an embedded clause

Question. Resolve she in “Anita told Ravi that she should revise.” Assume Anita is compatible with she and Ravi is not. Use the displayed parse.

Given

The pronoun is the subject of an embedded S inside the main VP.

To find

The preferred antecedent found by the climb and left-branch search.

Method and why it applies

The embedded clause has no left candidate. Climbing to the main S exposes its subject Anita as the first suitable higher-level candidate.

Solution

Step 1. Show the clause structure

S (main)\n├── NP: Anita                  ← accepted candidate\n└── VP\n    ├── V: told\n    ├── NP: Ravi\n    └── SBAR: that\n        └── S (embedded)       ← first X\n            ├── NP: she        ← start\n            └── VP: should revise

Step 2. Start inside the embedded clause

From NP she climb to the embedded S. There is no candidate to the left of she within this embedded S.

Step 3. Climb to the next NP/S

The embedded S is not the highest S. Move upward through SBAR and VP to the main S. These intermediate nodes are not NP/S search nodes.

Step 4. Search left of the upward path

At the main S, the subject NP Anita is a shallow left candidate. The object Ravi is deeper within the VP. Breadth-first order proposes Anita first.

Step 5. Check the stated compatibility

Anita is compatible with she by the question's assumption. Accept it; no need to choose a later incompatible candidate.

Step 6. Give the result and its limit

This is the algorithm's result under the displayed parse and agreement assumptions. It does not prove the intention of every speaker using an otherwise ambiguous sentence.

Answer. She → Anita under the stated tree and compatibility information.

Advantages, limitations and a complete exam answer

AdvantagesLimitations
Explicit candidate order makes the decision traceable.Wrong parse structure can produce the wrong search order.
Requires no labelled coreference-training corpus for its basic rules.World knowledge and discourse intention may override syntactic preference.
Uses structure rather than only surface distance.Some ambiguous, long-distance or unusual constructions need additional treatment.

For a longer answer, write the definition, inputs, tree notation, ordered algorithm, one labelled tree with a candidate table, final antecedent, and limitations. In a worked question show separately: candidate → why proposed → agreement result → accept/reject. Do not equate the selected candidate with a logically guaranteed interpretation when the sentence is ambiguous.

Yaad rakho: Tree order candidate deta hai; agreement usko allow/reject karta hai. Nearest noun automatically answer nahi.

Apply to this sentence

  1. Build the main clause with subject NP John; VP gave contains object NP a book and PP to Mary; the after-clause has subject he and object it under reading.
  2. For he, ascend from its NP to the governing S/NP and traverse candidate NPs to the left using the Hobbs breadth-first search. Test number/person/gender and binding compatibility.
  3. John is an earlier singular human candidate. Under the intended conventional name-gender reading, he→John. Mary is not chosen merely because it is nearer. Names alone do not establish real-world gender; this is the ordinary exercise reading.
  4. For it, the inanimate singular a book is compatible with reading; John and Mary are human mentions. Resolve it→a book.
  5. Final interpretation: John gave the book after John finished reading the book. The trees/candidate order guide the search; semantic compatibility is also needed.

The four requested properties

PropertyWhat it meansApply here
BindingSyntactic restrictions on reference; ordinary personal pronouns and reflexives have different local-antecedent requirementsDo not mechanically link every nearby NP; he in an after/because clause can refer outside that local clause.
Gender agreementCandidate must match the pronoun under available linguistic/contextual knowledgeJohn/he is the exercise’s conventional reading; professor/student do not provide gender by themselves.
DistanceToken, syntactic or sentence separation between pronoun and candidateCloser candidates may be preferred, but cannot override incompatibility or Hobbs search order.
RecencyRecently mentioned discourse entities are often more salientMary/student is later than John/professor, but recency is a preference, not a guaranteed antecedent rule.

Reference resolution preserves who performed each action; without it an extraction or QA system could wrongly attribute the reading to Mary. John is the first suitable main-clause subject under the declared parse; the inanimate book fails human he compatibility.

","m2":false,"units":"6","marks":10,"sourcePages":[3],"historical":true},{"id":"q6a","title":"Q6a · Corpora and acquisition","html":"
Question. Explain types, characteristics and acquisition of text corpora. [10 marks]

Answer

A corpus is a collection of language data assembled for study or computational use. Its plural is corpora. A corpus can contain books, news, reviews or conversations. It need not be labelled. Annotation adds linguistic information such as POS tags, entities, parse structures or sense labels.

StructureMeaningUse
PlainCollection without category labelsWord counts, language modelling
CategorizedDocuments grouped by genre/topicClassification
Overlapping categoriesOne document has several categoriesMulti-topic news
TemporalDocuments indexed by timeLanguage change
AnnotatedLinguistic labels attachedTagging, parsing, WSD
ParallelAligned texts in different languagesTranslation

A tagset is the allowed label inventory. A treebank is a corpus annotated with syntax trees. Metadata records source, date, genre, language, annotation method and limitations. A common benchmark supports comparison under shared data and evaluation conditions.

Raw example: “Riya works in Pune.” POS annotation: Riya/PROPN works/VERB in/ADP Pune/PROPN. NER annotation: Riya/PERSON, Pune/LOCATION. Each annotation answers a different task.

A good corpus is relevant to its domain, sufficiently representative, consistently annotated and appropriately split. More data alone do not repair duplicated examples, incorrect labels or a mismatch between training and real use.

Corpus=text collection. Annotation=text ke saath labels. Metadata=collection ki information.

ResourceContentsTypical use
Gutenberg selectionLiterary/book textsLong-form language and vocabulary
Web textInformal domain-specific textStudy usage beyond edited books
Chat corpusConversational postsDialogue, informal forms
BrownEnglish texts across genresGenre comparison
ReutersNews with topic labelsText classification; overlapping topics
InauguralSpeeches indexed by timeVocabulary change across years
WordlistWords/names/stopwordsLexical filtering
Pronouncing dictionaryWords and phonetic formsSpeech, stress, rhyme

Gutenberg versus web/chat: book language does not represent every spelling or conversational pattern in messages. Brown versus Reuters: Brown emphasizes genres; Reuters news can have overlapping topic labels. Inaugural addresses: dates provide a time dimension.

Corpus sizes depend on version, subset and tokenization. Explain their purpose and structure rather than assuming a historical count is a permanent property.

Gutenberg=books; web/chat=informal; Brown=genres; Reuters=news topics; Inaugural=time.

Data acquisition obtains language examples relevant to the task from existing data, public resources or appropriate new collection. Product instrumentation can gather useful feedback. Supervised learning additionally needs an annotation process when labels are absent.

Data augmentation creates training variants intended to preserve the correct label. It is not simply changing words arbitrarily.

MethodProcedureCaution
Synonym replacementUse suitable synonyms for selected wordsChoose the right sense
Back translationTranslate to another language and backMay change meanings/entities
TF–IDF-informed replacementProtect informative words while varying othersDo not remove label-bearing clues
Bigram flippingSwap adjacent wordsCan break grammar or meaning
Entity replacementReplace by same-type entityMust preserve the task label
Noise injectionSimulate typos/keyboard errorsMatch expected noise
Active learningSelect informative cases for human labelsThis acquires labels; it is not synthetic text

Three label decisions

“The film is good”→“The film is excellent” may preserve a positive sentiment label. “The film is not good”→“The film is good” reverses the label and is not valid label-preserving augmentation. “Flights to Pune”→“Flights to Mumbai” may preserve a travel-topic label but changes the destination label.

Split train/test data before creating related augmented variants. Otherwise near-duplicates can leak between sets and produce misleading evaluation. Keep a representative evaluation set of genuine examples.

Sentence badal sakta hai; required correct label nahi. Label badla toh augmentation invalid.

Corpus structure diagram

Corpus\n├─ Documents (plain collection)\n│  └─ Sentences → tokens\n├─ Categories → document IDs (one or overlapping labels)\n├─ Dates → document IDs (temporal collection)\n├─ Annotations → token spans / tags / parse trees\n└─ Parallel pairs → aligned source and target sentences

An annotated corpus attaches linguistic labels to the raw text: e.g. Rita/PROPN runs/VERB. A wordlist is a lexical inventory without full discourse context; a Gutenberg corpus contains books; web/chat corpora contain informal online language. Their structures determine which evidence can be queried.

","m2":false,"units":"3","marks":10,"sourcePages":[4],"historical":true},{"id":"q6b","title":"Q6b · HMM components and estimates","html":"
Question. Explain HMM components, the number of possible tag sequences and limitations. Construct transition and emission tables for the supplied Alice/Bob/They/The-cat tagged corpus. [10 marks]

Answer

A Hidden Markov Model (HMM) is a probabilistic sequence model in which the underlying state sequence is unobserved, while each state emits an observed item. For POS tagging, words are observed and their grammatical tags are hidden. The task is to infer a suitable tag sequence from the words.

Hidden: Noun
→
Hidden: Verb
→
Hidden: Noun
Observed: birds
→
Observed: eat
→
Observed: seeds

Horizontal state-to-state links are transitions. The relation from each hidden state to its observed word is an emission. A transition P(V|N) asks how often a verb tag follows a noun tag. An emission P(eat|V) asks how often the observed word eat appears among verb-tagged tokens.

ComponentNotationMeaning
State setS={N,V,…}Possible hidden tags.
Observation vocabularyO={birds,eat,seeds,…}Visible word tokens.
Initial distributionπⱼ=P(t₁=j)Probability of starting in tag j.
Transition matrixaᵢⱼ=P(tₖ=j|tₖ₋₁=i)Probability of next tag j after i.
Emission matrixbⱼ(w)=P(w|tag j)Probability that state j emits word w.

Two basic assumptions are used: the current hidden state depends only on the previous hidden state, and the current observation depends only on the current hidden state. A standard first-order HMM therefore does not explicitly condition an emitted word on the whole sentence.

\"P(t_{1:T},w_{1:T})=\\pi_{t_1}b_{t_1}(w_1)\\prod_{k=2}^{T}a_{t_{k-1},t_k}b_{t_k}(w_k)\"

Read the formula as start × first emission × transition × emission × transition × emission…. There are T emission factors and T−1 transitions, plus one initial factor. If the question supplies an explicit end-state probability, multiply that too. Do not invent an end factor when the model has none.

Three different questions. (1) Scoring a specified tag path uses the joint product above. (2) Finding the probability of the observed words sums scores over all possible tag paths. (3) Finding the best tag path maximizes the path score, typically using Viterbi. A joint path score is not a normalized posterior P(tags|words).

Yaad rakho: Transition = tag se agla tag. Emission = tag se visible word.

If each of T observed words can receive any of N tags, there are N choices at each position. Thus the total number of complete tag sequences is N multiplied by itself T times. This counts possible assignments before grammar rules or zero probabilities eliminate any.

\"\\text{Number

Question. A four-word sentence is tagged using three possible POS tags per word. How many complete tag sequences are possible?

Given

N=3 possible tags per position; T=4 word positions.

To find

The number of tag sequences.

Formula and why it applies

\"M=N^T\"

Every independent position contributes a factor of three choices.

Solution

Step 1. Write one factor per position
\"M=3\\times3\\times3\\times3\"
Step 2. Multiply
\"3\\times3=9,\\quad9\\times3=27,\\quad27\\times3=81\"
Final answer. 81 possible tag sequences.

Question. A five-word sentence has two candidate tags per word. Count its tag sequences.

Given

N=2; T=5.

To find

M.

Formula and why it applies

\"M=N^T\"

The length is the exponent, not a multiplier.

Solution

Step 1. Substitute
\"M=2^5\"
Step 2. Evaluate
\"2\\times2\\times2\\times2\\times2=32\"
Final answer. 32 possible sequences.

Question. Using 45 possible tags, compare the number of tag sequences for sentences of length 2 and 3.

Given

N=45. First T=2, then T=3.

To find

Both counts and the growth factor.

Formula and why it applies

\"M_2=45^2,\\quad

Adding one position multiplies the possibilities by the number of tag choices.

Solution

Step 1. Two words
\"45^2=2025\"
Step 2. Three words
\"45^3=2025\\times45=91125\"
Step 3. Compare
\"91125/2025=45\"
Final answer. 2 words: 2025 paths. 3 words: 91125 paths. One additional word multiplies the count by 45.

The slides introduce Viterbi to avoid enumerating every path. It reuses the best prefix score for each ending state. For a dense first-order HMM, it needs O(TN²) recurrence work instead of enumerating NT complete paths.

Yaad rakho: Choices per word ki power = sentence length.

Exact corpus calculation

Given

SentenceGiven tags
Alice will eat cookiesN M V N
Bob can drink teaN M V N
They will watch moviesPronoun M V N
The cat is sleepingDet N V V

To find

Emission and transition tables

Formula: which one and why

\"P(w\\mid
\"P(t_j\\mid

The second formula is the declared no-END convention. Reset at sentence boundaries; never create a transition from the last word of one training sentence to the first of the next.

Step 1. Count emissions

TagTotal occurrencesAlicewilleatcookiesBobcandrinkteaTheywatchmoviesThecatissleeping
N6100110010010100
M3020001000000000
V5001000100100011
Pronoun1000000001000000
Det1000000000001000

Step 2. Divide each emission by its tag count

TagAlicewilleatcookiesBobcandrinkteaTheywatchmoviesThecatissleeping
N1/6001/61/6001/6001/601/600
M02/30001/3000000000
V001/50001/5001/50001/51/5
Pronoun000000001000000
Det000000000001000

Step 3. Count transitions inside each sentence

FromOutgoing transitionsSentence-final countNMVPronounDet
N3302100
M3000300
V4130100
Pronoun1001000
Det1010000

Step 4. Convert to a transition table

From / toNMVPronounDet
N02/31/300
M00100
V3/401/400
Pronoun01000
Det10000

Here sentence-final tags are excluded from the no-END outgoing denominator. If END is explicitly modeled, divide all tag→tag and tag→END counts by the total tag count instead; the row then includes END. Do not mix denominators.

Final answer

The two displayed probability tables are the estimated HMM. Zero means unobserved without smoothing, not logically impossible.

For T tokens and K possible tags per token, exhaustive assignments number K^T before constraints; the paper does not supply a single test length for a specific numerical total. HMM assumptions limit long-range context, first-order transitions and emission independence; sparsity and unknown words require smoothing or other handling. The supplied tag is for is and sleeping is V, so both count as V even though finer linguistic tagsets distinguish auxiliaries/participles.

","m2":false,"units":"4","marks":10,"sourcePages":[4],"historical":true},{"id":"q7a","title":"Q7a · Word classes, chunking and chinking","html":"
Question. Explain open/closed word classes, chunking and chinking with examples. [5 marks]

Answer

Part-of-speech (POS) tagging assigns a grammatical category to every word token in context, such as noun, verb, adjective or determiner. It is a sequence-labeling task: the input is a sequence of words, and the output is a tag for each word. A tag describes the word's role in this sentence, not its meaning in every possible sentence.

TagRoleExample in context
NN / NNSSingular / plural common nouna book / two books
NNPProper nounMary, Mumbai
VB / VBD / VBZBase / past / third-person singular verbread / read / reads
JJAdjectivethe red book
RBAdverbruns quickly
DTDeterminerthe, a, this
INPreposition or subordinating conjunctionon the table
PRPPersonal pronounshe, they
MDModal auxiliarycan, will, should

Open classes readily accept new content words: nouns, verbs, adjectives and adverbs. Examples include chatbot and livestream. Closed classes contain comparatively stable grammatical/function words: determiners, pronouns, prepositions and conjunctions. These are tendencies, not a claim that language never changes.

Input: Book that flight
→
Context: an imperative request
→
Output: Book/VB that/DT flight/NN

Compare “hand me that book”: book is a noun here. Similarly, “I can swim” has can/MD, while “the can is empty” has can/NN. The spelling stays the same; neighboring words and sentence structure change the correct tag. This is POS ambiguity.

SentenceTagged interpretationReason
They watch birds.They/PRP watch/VBP birds/NNSwatch is the action.
The watch stopped.The/DT watch/NN stopped/VBDwatch is the object that stopped.
She has a light bag.She/PRP has/VBZ a/DT light/JJ bag/NNlight describes bag.
Light the lamp.Light/VB the/DT lamp/NNLight is an imperative verb.

POS tags support parsing, named entity recognition, question answering, sentiment analysis and word-sense disambiguation. They do not by themselves identify every entity or fully resolve meaning.

Yaad rakho: Word ka naam same ho sakta hai; sentence mein uska kaam tag decide karta hai.

Chunking, also called shallow parsing, groups adjacent tagged words into meaningful non-overlapping units, commonly noun phrases. It usually identifies local phrase boundaries without producing the complete hierarchical structure of a sentence. Chinking removes specified tokens from an existing candidate chunk, splitting or trimming it.

The/DT small/JJ cat/NN sat/VBD on/IN the/DT mat/NN
→
NP: [The small cat]
→
Outside: sat on
→
NP: [the mat]

A POS tagger labels individual words. A chunker groups words into phrases. A full parser shows how phrases relate recursively to form the entire sentence. These are related tasks with different outputs.

OperationInputOutput
POS taggingThe small catThe/DT small/JJ cat/NN
NP chunkingTagged tokens[NP The small cat]
ChinkingA broad candidate chunkRemove verbs/prepositions, keep noun-phrase segments
Full parsingSentence + grammarA nested tree with sentence, noun phrase and verb phrase nodes

Rule-based chunking uses tag patterns. Statistical or learned chunkers infer boundaries from labeled examples. A useful NP rule is an optional determiner, followed by zero or more adjectives, followed by a noun. In NLTK-style notation: NP: {<DT>?<JJ>*<NN>}. Here ? means zero or one occurrence; * means zero or more; + means one or more when used. The braces mean “form a chunk around the matched sequence”.

Question. Apply the NP rule optional DT, zero or more JJ, then one NN to “The/DT quick/JJ brown/JJ fox/NN jumps/VBZ over/IN the/DT lazy/JJ dog/NN”.

Given

Tagged input is supplied. Rule: DT? JJ* NN.

To find

All noun-phrase chunks.

Formula and why it applies

NP → DT? JJ* NN

The rule groups a noun with its preceding determiner and adjectives; a verb or preposition does not match this NP pattern.

Solution

Step 1. Start at The

The matches optional DT; quick and brown both match JJ*; fox matches NN.

Step 2. Form the first NP

[NP The quick brown fox]

Step 3. Skip nonmatching tokens

jumps/VBZ and over/IN do not satisfy the NP pattern.

Step 4. Match the second NP

the is DT, lazy is JJ, dog is NN: [NP the lazy dog].

Final answer. [NP The quick brown fox] jumps over [NP the lazy dog].

Question. Using NP → DT? JJ* NN, identify the chunks in “bright/JJ stars/NN shine/VBP and/CC a/DT moon/NN glows/VBZ”. Use the supplied tags exactly.

Given

Input tags label stars as NN for this exercise. Rule requires optional DT, zero or more JJ, then NN.

To find

The matched NP spans.

Formula and why it applies

NP → DT? JJ* NN

The optional determiner may be absent and the adjective repetition may be empty.

Solution

Step 1. Match bright stars

No determiner is present, which DT? permits. bright matches JJ; stars matches NN.

Step 2. Inspect shine and and

VBP and CC fall outside the NP rule.

Step 3. Match a moon

a matches DT; no adjective is present; moon matches NN.

Final answer. Chunks are [NP bright stars] and [NP a moon]. With ordinary Penn tagging, stars is usually NNS; broaden the noun pattern to include NNS when that tag is used.

Question. Begin with one candidate chunk containing “The/DT cat/NN sat/VBD on/IN the/DT mat/NN”. Chink out verbs and prepositions. Show the remaining chunks.

Given

Initial candidate: [The cat sat on the mat]. Remove tags VBD and IN.

To find

The chunk segments after exclusion.

Formula and why it applies

Chinking removes matched tokens from the existing chunk.

Removing the middle verb/preposition region divides the original candidate into two remaining contiguous spans.

Solution

Step 1. Locate the tokens to remove

sat is VBD and on is IN.

Step 2. Remove those tokens from chunk membership

The remaining token groups are “The cat” and “the mat”. sat and on still exist in the sentence; they are simply outside these chunks.

Step 3. Write the final chunked sentence

[The cat] sat on [the mat].

Final answer. Two chunks remain: The cat; the mat.

Yaad rakho: Chunking = andar group karo; chinking = unwanted tokens ko group se bahar karo.

","m2":false,"units":"4","marks":5,"sourcePages":[4],"historical":true},{"id":"q7b","title":"Q7b · Named entity recognition","html":"
Question. Explain named-entity recognition and its approaches with examples. [5 marks]

Answer

Named Entity Recognition (NER) identifies text spans referring to entities and assigns their types, such as PERSON, ORGANIZATION and LOCATION. Many practical schemas also include DATE, TIME, MONEY or domain-specific types. The exact allowed labels depend on the application's annotation scheme.

NER has two parts: boundary detection finds where the entity starts and ends; classification assigns the appropriate type. A multiword name should normally be one entity span: “New York” is a location span, not two unrelated location decisions.

Text: Mary joined Google in New York on Monday.
→
Mary → PERSON
→
Google → ORGANIZATION
→
New York → LOCATION
→
Monday → DATE

Not every noun is a named entity. In “a developer works at Google”, developer is an ordinary job description while Google is a named organization. A specialized HR system may define a JOB_TITLE category, but it must state that schema explicitly. NER identifies mentions; entity linking is the separate task of connecting a mention to a unique database identity.

TaskQuestion it answersExample
POSWhat grammatical role does this token have?Google/NNP
ChunkingWhich adjacent words form a phrase?[NP a skilled developer]
NERWhich span names an entity, and of what type?[Google] ORGANIZATION
Entity linkingWhich real entity does this mention denote?Connect Paris to the intended city or person record

Applications. NER structures search results and documents, extracts organizations/people from news, routes support tickets by product or organization, supports recommendation, and extracts specialized entities from healthcare or legal text. Context is essential: “Apple released a laptop” names an organization, while “she ate an apple” describes fruit.

Yaad rakho: NER mein do kaam: naam kahan se kahan tak, aur kis category ka.

ApproachHow it worksExampleMain limitation
Dictionary / gazetteerMatch text against lists of known namesA city list includes Mumbai and PuneNew names, spelling variants and ambiguous names are missed
Rule-basedUse word shapes, regular expressions and contextA title such as Dr. followed by a name-like span suggests PERSONManual maintenance; exceptions and domain/language dependence
Machine learningLearn a labeling rule from annotated examples using featuresCapitalization, suffixes, POS and surrounding words help a classifier/CRFRequires labeled data and useful features
Deep learningLearn contextual word/character representations and predict labelsA contextual sequence encoder labels multiword entity spansTraining/domain dependence and resource demands

A pattern-based rule examines the form of text, such as an email pattern or a date format. A context-based rule examines neighboring words: “Dr.” can support a following PERSON span, while “headquartered in” can support a following LOCATION span. Capitalization alone is insufficient because sentence-initial common nouns are also capitalized.

A dictionary may find “May” as a person's name, but “in May” may refer to a month. A learned model uses surrounding evidence to disambiguate. Hybrid systems can combine dictionaries, rules and learned models; these categories are practical tools rather than mutually exclusive restrictions.

Preprocessing for NER. Tokenize while preserving token offsets and useful capitalization. Blindly removing capital letters or stopwords before recognition can damage names such as “Bank of India”. NER output should preserve the original span so the reader can locate it in the input.

Exam-ready 3-mark answer. NER locates named-entity spans and classifies them into categories such as person, organization and location. A dictionary-based approach matches known name lists; a rule-based approach uses patterns or context; machine-learning and deep-learning approaches learn entity decisions from labeled data and representations. For example, in “Mary works at Google in Mumbai”, Mary is PERSON, Google is ORGANIZATION and Mumbai is LOCATION.

","m2":false,"units":"4","marks":5,"sourcePages":[4],"historical":true},{"id":"q7c","title":"Q7c · Co-occurrence with inconsistent requested comparisons","html":"
Question. Construct a whole-sentence co-occurrence matrix for: The bird flew across the sky. The cat watched the bird. The flowers grew in the garden. The bird rested on the tree. The cat slept near the tree. Targets={bird,cat,garden,flowers}; contexts={sky,tree,garden,slept}. The printed follow-up asks dot products boy–dog and boy–sun and a plot on kicked/field dimensions. [10 marks]

Answer

Given

Use the exact target/context lists above.

To find

Construct the supported matrix and identify which later requests are undefined.

Formula: which one and why

Count ordered target-token/context-token pairs in the same sentence, excluding the target token itself. A single garden occurrence cannot co-occur with itself. If a classroom convention uses sentence-level presence including diagonal, only garden→garden changes from0to1; state that choice.

Step 1. Enumerate contributions and sum

SentenceTarget occurrencesContext occurrences
1birdsky
2cat, bird
3flowers, gardengarden
4birdtree
5catslept, tree
Target / contextskytreegardenslept
bird1100
cat0101
garden0000
flowers0010

Step 2. Check requested names before multiplying

boy,dog,sun are not target rows; kicked,field are not context columns. Therefore the requested dot products and plot cannot be calculated from the supplied matrix. Zero vectors for invented words would not repair the missing givens.

Step 3. Clearly labelled corrected illustration

If the follow-up were changed to bird–cat and bird–flowers, using the displayed rows: bird·cat=1×0+1×1+0×0+0×1=1; bird·flowers=1×0+1×0+0×1+0×0=0. A sky/tree plot would use bird(1,1),cat(0,1),garden(0,0),flowers(0,0). These substitutions are teaching examples, not printed instructions.

Final answer

The original matrix is solved; its requested boy/dog/sun comparisons and kicked/field plot are not well-defined. Ask for corrected target/context names rather than inventing a numerical answer.

Source / convention note. Source defect: follow-up names belong to another corpus. Exact supported matrix is complete; unsupported parts are explicitly unresolved from the original data.

","m2":false,"units":"5","marks":10,"sourcePages":[5],"historical":true}]},{"id":"legacy-f24","title":"2024-12-03 · final · recovered AI paper","unit":"PYQ","scope":"Previous examination · complete worked answers","sections":[{"id":"q1a","title":"Q1a · TF–IDF in retrieval","html":"
Question. Write TF–IDF expressions and explain their role in information retrieval. [individual allocation not printed; Q1 totals 20]

Answer

TF–IDF weights a term highly when it is frequent in a particular document but uncommon across the corpus. The supplied Unit 2 slides use normalized term frequency and base-10 inverse document frequency:

\"\\operatorname{TF}(t,d)=\\frac{C(t,d)}{N_d}\"
\"\\operatorname{IDF}(t)=\\log_{10}\\frac{D}{\\operatorname{df}(t)}\"
\"\\operatorname{TFIDF}(t,d)=\\operatorname{TF}(t,d)\\operatorname{IDF}(t).\"

\"N_d\" is the document's token count after the specified preprocessing; \"D\" is number of documents; \"\\operatorname{df}(t)\" is the number of documents containing the term at least once. A word repeated ten times in one document contributes one to document frequency.

Why the log? It compresses the ratio's scale. If a term occurs in all documents, \"D/df=1\", so IDF=0 under this unsmoothed convention. Other libraries use smoothed IDF, different TF or normalization. Use the question's formula; do not silently substitute a library default.

Question. A corpus contains 10 documents. The word model appears in 2 documents and occurs 3 times in a 12-token document. Compute TF, IDF and TF–IDF using the slide convention.

Given

D=10, df(model)=2, C(model,d)=3, Nd=12; log base 10.

To find

The TF–IDF weight.

Formula and why it applies

\"TF=C/N_d\", \"IDF=\\log_{10}(D/df)\", then multiply.

Step 1. TF

\"3/12=0.25\".

Step 2. Document ratio

\"10/2=5\".

Step 3. IDF

\"\\log_{10}5\\approx0.698970\".

Step 4. Weight

\"0.25\\times0.698970\\approx0.174743\".

Answer. TF=0.25, IDF≈0.698970, TF–IDF≈0.174743.

Question. A term occurs 4 times in a 20-token document and is present in all 8 corpus documents. Calculate its TF–IDF.

Given

C=4, Nd=20, D=8, df=8.

To find

Its weight under unsmoothed base-10 IDF.

Formula and why it applies

\"TF=C/N_d\", \"IDF=\\log_{10}(D/df)\".

Step 1. TF

\"4/20=0.2\".

Step 2. IDF

\"\\log_{10}(8/8)=\\log_{10}1=0\".

Step 3. Multiply

\"0.2\\times0=0\".

Answer. Weight 0: this term does not distinguish documents under this convention.

Question. Find TF–IDF vectors for D1=“cat cat dog”, D2=“dog fish”, D3=“cat fish fish”, using [cat,dog,fish], normalized TF and log10 IDF.

Given

No further preprocessing. Each term occurs in exactly two of the three documents.

To find

All three vectors.

Formula and why it applies

First count df by documents, then compute each document's TF, then multiply by its term's IDF.

Step 1. Common IDF

\"D=3\", \"df(cat)=df(dog)=df(fish)=2\". Each IDF \"=\\log_{10}(3/2)\\approx0.176091\".

Step 2. D1 term frequencies

Length 3 → \"[2/3,1/3,0]\".

Step 3. D1 weights

\"[(2/3)0.176091,(1/3)0.176091,0]\\approx[0.117394,0.058697,0]\".

Step 4. D2 weights

Length 2; TF \"[0,1/2,1/2]\". Vector \"\\approx[0,0.088046,0.088046]\".

Step 5. D3 weights

Length 3; TF \"[1/3,0,2/3]\". Vector \"\\approx[0.058697,0,0.117394]\".

Answer. D1≈[0.117394,0.058697,0]; D2≈[0,0.088046,0.088046]; D3≈[0.058697,0,0.117394].

TF: iss document mein kitna. IDF: poore corpus mein kitna rare. Dono multiply.

Information retrieval (IR) locates and ranks items relevant to an information need, usually expressed as a query. The collection may contain documents, product descriptions or scientific papers. Retrieval estimates relevance; it does not itself establish that every retrieved claim is true.

Documents → cleaning
→
Build index
→
Query → same representation
→
Match & rank
→
Return relevant results

The inverted index

An inverted index maps a term to the documents containing it, often with positions or counts. For D1 “cats drink milk” and D2 “dogs drink water”, the posting list for drink is {D1,D2}, while milk maps only to D1. This avoids rescanning every document for every query.

TermPosting list
catsD1
drinkD1, D2
milkD1
dogsD2
waterD2

Matching and ranking

Boolean retrieval combines term membership using AND, OR and NOT. Vector-space retrieval represents documents and queries as weighted vectors, for example with TF-IDF, and compares them with cosine similarity. Learned retrieval can use dense embeddings for meaning-based matching. The representation used at query time must be compatible with the document index.

Search engines and recommendation

A search engine receives an explicit query and ranks matching indexed pages. A recommender may derive an implicit query from a user’s liked items or profile and retrieve similar candidates. Both require candidate retrieval and ranking, but the source of the information need differs.

Evaluate the results

For retrieval, TP is the number of retrieved relevant documents, FP retrieved irrelevant documents, and FN relevant documents not retrieved. Precision and recall use the same formulas as classification. Ranking measures also consider the positions of relevant items; a relevant result at rank 1 is more useful than one far below.

Retrieval evaluation

A search system retrieves 8 documents; 6 are relevant. There are 10 relevant documents in the collection. Find precision, recall and F1.

Given

Retrieved=8; relevant retrieved=6; all relevant=10.

To find

P, R and F1.

Formula and why it applies

\"P=\\frac{\\text{relevant
\"F_1=\\frac{2PR}{P+R}\"

The denominator is 8 for precision and 10 for recall because they ask different questions.

Solution

Step 1. Identify the confusion counts

TP=6; FP=8−6=2; FN=10−6=4.

Step 2. Precision

\"P=\\frac68=0.75\"

Step 3. Recall

\"R=\\frac6{10}=0.6\"

Step 4. F1

\"F_1=\\frac{2(0.75)(0.6)}{0.75+0.6}=\\frac{0.9}{1.35}=\\frac23\\approx0.666667\"
Precision=0.75, recall=0.60, F1≈0.666667.

Yaad rakho: Precision: jo laaye, unmein kitne sahi? Recall: jo sahi the, unmein kitne laaye?

","m2":false,"units":"2,7","marks":null,"sourcePages":[1],"historical":true},{"id":"q1b","title":"Q1b · NER and approaches","html":"
Question. Explain NER with examples and approaches. [individual allocation not printed; Q1 totals 20]

Answer

Named Entity Recognition (NER) identifies text spans referring to entities and assigns their types, such as PERSON, ORGANIZATION and LOCATION. Many practical schemas also include DATE, TIME, MONEY or domain-specific types. The exact allowed labels depend on the application's annotation scheme.

NER has two parts: boundary detection finds where the entity starts and ends; classification assigns the appropriate type. A multiword name should normally be one entity span: “New York” is a location span, not two unrelated location decisions.

Text: Mary joined Google in New York on Monday.
→
Mary → PERSON
→
Google → ORGANIZATION
→
New York → LOCATION
→
Monday → DATE

Not every noun is a named entity. In “a developer works at Google”, developer is an ordinary job description while Google is a named organization. A specialized HR system may define a JOB_TITLE category, but it must state that schema explicitly. NER identifies mentions; entity linking is the separate task of connecting a mention to a unique database identity.

TaskQuestion it answersExample
POSWhat grammatical role does this token have?Google/NNP
ChunkingWhich adjacent words form a phrase?[NP a skilled developer]
NERWhich span names an entity, and of what type?[Google] ORGANIZATION
Entity linkingWhich real entity does this mention denote?Connect Paris to the intended city or person record

Applications. NER structures search results and documents, extracts organizations/people from news, routes support tickets by product or organization, supports recommendation, and extracts specialized entities from healthcare or legal text. Context is essential: “Apple released a laptop” names an organization, while “she ate an apple” describes fruit.

Yaad rakho: NER mein do kaam: naam kahan se kahan tak, aur kis category ka.

ApproachHow it worksExampleMain limitation
Dictionary / gazetteerMatch text against lists of known namesA city list includes Mumbai and PuneNew names, spelling variants and ambiguous names are missed
Rule-basedUse word shapes, regular expressions and contextA title such as Dr. followed by a name-like span suggests PERSONManual maintenance; exceptions and domain/language dependence
Machine learningLearn a labeling rule from annotated examples using featuresCapitalization, suffixes, POS and surrounding words help a classifier/CRFRequires labeled data and useful features
Deep learningLearn contextual word/character representations and predict labelsA contextual sequence encoder labels multiword entity spansTraining/domain dependence and resource demands

A pattern-based rule examines the form of text, such as an email pattern or a date format. A context-based rule examines neighboring words: “Dr.” can support a following PERSON span, while “headquartered in” can support a following LOCATION span. Capitalization alone is insufficient because sentence-initial common nouns are also capitalized.

A dictionary may find “May” as a person's name, but “in May” may refer to a month. A learned model uses surrounding evidence to disambiguate. Hybrid systems can combine dictionaries, rules and learned models; these categories are practical tools rather than mutually exclusive restrictions.

Preprocessing for NER. Tokenize while preserving token offsets and useful capitalization. Blindly removing capital letters or stopwords before recognition can damage names such as “Bank of India”. NER output should preserve the original span so the reader can locate it in the input.

Exam-ready 3-mark answer. NER locates named-entity spans and classifies them into categories such as person, organization and location. A dictionary-based approach matches known name lists; a rule-based approach uses patterns or context; machine-learning and deep-learning approaches learn entity decisions from labeled data and representations. For example, in “Mary works at Google in Mumbai”, Mary is PERSON, Google is ORGANIZATION and Mumbai is LOCATION.

","m2":false,"units":"4","marks":null,"sourcePages":[1],"historical":true},{"id":"q1c","title":"Q1c · Skip-gram and CBOW","html":"
Question. Explain Skip-gram and CBOW. [individual allocation not printed; Q1 totals 20]

Answer

Word2Vec learns dense word embeddings by predicting words from nearby context. Its two common architectures are CBOW and skip-gram. A vocabulary ID or one-hot vector is the input representation; the learned embedding table converts that identity into a short trainable vector.

ArchitectureInputPredictionMemory cue
CBOWSurrounding context wordsCentre/target wordContext → centre
Skip-gramCentre/target wordSurrounding wordsCentre → context
CBOW: the + sat
→
Average context embeddings
→
Predict: cat
Skip-gram: cat
→
Embedding of cat
→
Predict: the and sat

For vocabulary size \"V\" and embedding dimension \"d\", an input one-hot row is \"1\\times; the input embedding matrix is \"V\\times; the hidden row is \"1\\times; the output matrix is \"d\\times; output scores are \"1\\times. Multiplying a one-hot row by the embedding matrix simply selects one row.

\"P(w_o\\mid

\"v_{w_i}\" is the input word embedding, \"u_w\" an output-context embedding, and the dot product is a compatibility score. Softmax turns all scores into probabilities that sum to one. Training increases the probability of observed context words. Negative sampling or hierarchical softmax can reduce computation in larger vocabularies; use the algorithm specified by the question.

Question. For the sentence “the cat sat on mat”, generate CBOW and skip-gram training data for centre word sat with a window of one token on each side.

Given

Tokens indexed 1 to 5: the, cat, sat, on, mat. Centre = sat.

To find

The CBOW example and the skip-gram pairs.

Formula and why it applies

CBOW predicts the centre from its neighbours; skip-gram predicts each neighbour from the centre.

Step 1. Identify neighbours

Left neighbour = cat; right neighbour = on. “the” and “mat” are outside this window.

Step 2. CBOW example

Input [cat,on] → target sat.

Step 3. Skip-gram examples

(sat,cat) and (sat,on), with first item as input and second as predicted context.

Answer. CBOW: [cat,on]→sat. Skip-gram: sat→cat and sat→on.

Question. The embeddings of context words cat and on are [1,2] and [3,0]. Compute the CBOW hidden vector using mean aggregation. Output vectors for [sat,ran] are [1,0] and [0,1]. Compute softmax probabilities.

Given

Context embeddings [1,2], [3,0]; mean aggregation; two candidate output vectors.

To find

The hidden vector, logits and probabilities.

Formula and why it applies

Mean context vector \"h=(v_1+v_2)/2\". Logit \"z_w=u_w^Th\". Softmax \"P(w)=e^{z_w}/\\sum_j.

Step 1. Add coordinates

\"[1+3,\\,2+0]=[4,2]\".

Step 2. Divide by number of words

\"h=[4/2,2/2]=[2,1]\".

Step 3. Compute logits

\"z_{sat}=1(2)+0(1)=2\". \"z_{ran}=0(2)+1(1)=1\".

Step 4. Exponentials

\"e^2\\approx7.389056\", \"e^1\\approx2.718282\"; sum \"\\approx10.107338\".

Step 5. Divide

\"P(sat)\\approx0.731059\", \"P(ran)\\approx0.268941\".

Answer. h=[2,1]; sat is more probable (0.731059).

Question. A skip-gram centre embedding is v=[1,−1]. Output embeddings for [eat,sleep,drive] are [1,1], [0,1], [1,0]. Calculate the predicted context probabilities.

Given

One centre vector and three output vectors; full softmax over these three words.

To find

Three logits and three probabilities.

Formula and why it applies

\"z_j=u_j^Tv\", then \"P(j)=e^{z_j}/\\sum_k.

Step 1. Eat dot product

\"1(1)+1(-1)=0\".

Step 2. Sleep and drive dot products

\"z_{sleep}=0(1)+1(-1)=-1\". \"z_{drive}=1(1)+0(-1)=1\".

Step 3. Exponentiate and sum

\"e^0=1\", \"e^{-1}\\approx0.367879\", \"e^1\\approx2.718282\". Sum \"\\approx4.086161\".

Step 4. Normalize

\"P(eat)\\approx0.244728\"; \"P(sleep)\\approx0.090031\"; \"P(drive)\\approx0.665241\".

Answer. drive has the highest probability in this toy learned model.

Limitations: a standard static Word2Vec vector merges a word's senses; it has limited representation of subword morphology and out-of-vocabulary words; its local prediction objective does not directly fit a full global co-occurrence table. Similarity is learned from data and is not a guarantee of factual or causal understanding.

CBOW = neighbours se missing centre. Skip-gram = centre se neighbours.

","m2":false,"units":"5","marks":null,"sourcePages":[1],"historical":true},{"id":"q1d","title":"Q1d · Pragmatics and discourse","html":"
Question. Explain pragmatics and discourse in NLP. [individual allocation not printed; Q1 totals 20]

Answer

Pragmatic analysis interprets an utterance using its situation and the speaker’s intended meaning. Discourse analysis studies how utterances connect across a text or conversation. Syntax and literal word meanings alone cannot settle every interpretation.

TextLiteral contentContextual interpretation
“Can you close the window?”A question about ability.Usually a polite request.
“Meera bought a laptop. It was expensive.”Two sentences.“It” connects the second sentence to the laptop.
“Meet me here tomorrow.”An instruction.Me, here and tomorrow depend on speaker, place and date.

Reference, mention and antecedent

A referring expression points to an entity; a mention is its occurrence in the text. Its referent is the entity being discussed. An antecedent is a preceding expression used to interpret a later referring expression. Reference resolution identifies the intended referent. Coreference resolution groups mentions that refer to the same entity.

“Meera”name mention
→
Person entitysame referent
→
“she” / “the engineer”other mentions
ConceptDefinitionExample
AnaphoraReference back to something introduced earlier.“Riya arrived. She sat down.” She → Riya.
CataphoraReference forward to a later expression.“Before she spoke, Riya checked the microphone.” she → Riya.
ExophoraReference to the situation outside the text.“Put it there”, said while pointing.
CoreferenceTwo or more mentions refer to the same entity.“Dr Rao”, “the lecturer”, “she”.
Pronoun resolutionFind the referent of a pronoun.Resolve “it” in “The laptop stopped because it overheated.”

Coreference is broader than pronoun resolution: repeated names and descriptions also form a chain. Anaphora describes direction/dependence; coreference describes identity. In “I bought a car. The engine is noisy”, engine is related to the car but is not the same entity.

Yaad rakho: Reference = kiski baat? Coreference = alag mentions, same entity. Pronoun resolution = he/she/it kis se linked?

Cohesion, coherence and discourse markers

\n

Cohesion is the explicit linguistic linking of a text through reference, repeated or related words, conjunctions and other devices. In “Rita bought a book. She read it”, she and it connect the second sentence to Rita and the book.

\n

Coherence is the conceptual unity that makes a discourse meaningful. “The road flooded, so the match was cancelled” presents a plausible cause and consequence. A reader relates the propositions using context and knowledge.

\n

Discourse markers signal relationships or organize speech: however signals contrast; therefore signals a result; first and next signal sequence; for example introduces an illustration.

\n

These concepts differ: “I like apples. Therefore the moon is square” has a linking word but no sensible inference. A connector alone does not guarantee coherence.

\n

Yaad rakho: Cohesion = words se link; coherence = meaning se link; marker = relation ka signal.

","m2":false,"units":"6","marks":null,"sourcePages":[1],"historical":true},{"id":"q2a","title":"Q2a · Six preprocessing operations","html":"
Question. Explain lowercasing, tokenization, stopword removal, stemming, lemmatization and POS tagging with examples. [6 marks]

Answer

Tokenization divides text into units called tokens for further processing. A token may be a word, punctuation mark, character or subword, depending on the tokenizer. Sentence segmentation identifies sentence boundaries. Tokenization is not necessarily just splitting on spaces.

MethodExampleTrade-off
Word“cats sleep” → [cats,sleep]Easy to interpret; unfamiliar words can be unseen tokens
Charactercat → [c,a,t]Handles spelling units; creates longer sequences
Subwordunhappiness → pieces such as [un,happi,ness] in an illustrative vocabularyBalances vocabulary size and rare-word representation
Sentence“Birds fly. Fish swim.” → two sentencesAbbreviations and punctuation complicate boundaries

Actual subword splits depend on the learned vocabulary; the example is illustrative, not a universal tokenizer output. Word segmentation is especially important where spaces do not reliably separate words or in compound-rich languages.

Question. Tokenize “Dr. Rao can't come today.” into words and punctuation under this rule: preserve Dr. and can't as whole tokens, and split only the final full stop.

Given

The tokenization rule is explicitly supplied.

To find

The resulting token sequence.

Formula and why it applies

Apply the stated boundary rule; do not assume every period or apostrophe is a split point.

Step 1. Handle abbreviation

Dr. remains one token.

Step 2. Handle contraction

can't remains one token.

Step 3. Write sequence

[Dr., Rao, can't, come, today, .]

Answer. 6 tokens under the specified rule. Another tokenizer may legitimately produce a different sequence.

Three difficult cases

New York may be two tokens but one named entity. 3.14 should not normally end a sentence at the decimal point. don't may remain whole or split into do + n't, depending on the tokenizer. Always state the convention when calculating counts.

Token = chosen unit. Space split ko universal rule mat samajh.

Normalization reduces superficial variation so comparable forms can be processed consistently. Common operations include lowercasing, Unicode normalization, appropriate spelling correction, contraction handling, and optional stemming or lemmatization.

Stopwords are frequent words selected for removal for a particular task, such as the or a. Removing them may reduce feature size, but removing not, no or nor can destroy sentiment meaning. Punctuation can convey sentence boundaries, emotion or mathematical structure. Lowercasing can erase differences such as US versus us.

Question. Preprocess “The CATS are not sleeping!!!” by lowercasing, removing punctuation, removing only stopwords {the,are}, and lemmatizing plural cats→cat and sleeping→sleep.

Given

Operations and stopword set are fixed. Do not remove not.

To find

The final token list.

Formula and why it applies

Apply the requested operations in order, keeping a visible intermediate result after each transformation.

Step 1. Lowercase

the cats are not sleeping!!!

Step 2. Tokenize and remove punctuation

[the,cats,are,not,sleeping]

Step 3. Remove specified stopwords

Remove the and are → [cats,not,sleeping].

Step 4. Lemmatize

cats→cat; sleeping→sleep → [cat,not,sleep].

Answer. [cat,not,sleep]. The negative meaning is retained.

Task decides preprocessing

For a simple keyword-retrieval model, removing selected function words can help. For machine translation, question answering, grammar analysis or transformer input, aggressive removal may discard necessary syntax or violate the model's expected input format. Emoji can carry sentiment; convert or retain them when relevant instead of always deleting them.

Clean karna = useful signal bachake noise hatao. Har word/punctuation delete karna cleaning nahi.

Stemming applies string-based rules, often stripping suffixes, to reduce related forms to a stem. The result need not be a valid dictionary word. Lemmatization uses vocabulary and morphological analysis, often with POS information, to return a dictionary lemma.

CriterionStemmingLemmatization
BasisSurface form/suffix rulesLexicon and morphological/POS analysis
OutputMay be nonword, e.g. studiDictionary form, e.g. study
Context/POSUsually little or noneOften important
Irregular formsMay miss went→goCan map went→go with verb analysis
CostGenerally simplerUsually more linguistic processing

Porter applies a sequence of suffix-removal rules subject to conditions; Snowball provides related language-specific stemming algorithms; Lancaster is often more aggressive; regex stemming uses explicit patterns; Lovins is another established stemming approach. Exact output depends on the algorithm and implementation.

Three worked reductions

  1. studies, studying: an English stemmer may yield studi/study depending on the rule sequence; verb lemmatization aims at study. State the exact algorithm if exact outputs are asked.
  2. better: adjective-aware lemmatization can map it to good; naïve suffix removal cannot infer that irregular relation.
  3. saw: as a past-tense verb it can become see; as a cutting-tool noun it remains saw. POS and context matter.

Over-stemming merges forms that should remain distinct; under-stemming fails to merge related forms. Normalization can increase recall in search while losing useful distinctions, so it is a task-dependent choice.

Stem = rules se cut; lemma = dictionary form, meaning/POS ka dhyan.

Part-of-speech (POS) tagging assigns a grammatical category to every word token in context, such as noun, verb, adjective or determiner. It is a sequence-labeling task: the input is a sequence of words, and the output is a tag for each word. A tag describes the word's role in this sentence, not its meaning in every possible sentence.

TagRoleExample in context
NN / NNSSingular / plural common nouna book / two books
NNPProper nounMary, Mumbai
VB / VBD / VBZBase / past / third-person singular verbread / read / reads
JJAdjectivethe red book
RBAdverbruns quickly
DTDeterminerthe, a, this
INPreposition or subordinating conjunctionon the table
PRPPersonal pronounshe, they
MDModal auxiliarycan, will, should

Open classes readily accept new content words: nouns, verbs, adjectives and adverbs. Examples include chatbot and livestream. Closed classes contain comparatively stable grammatical/function words: determiners, pronouns, prepositions and conjunctions. These are tendencies, not a claim that language never changes.

Input: Book that flight
→
Context: an imperative request
→
Output: Book/VB that/DT flight/NN

Compare “hand me that book”: book is a noun here. Similarly, “I can swim” has can/MD, while “the can is empty” has can/NN. The spelling stays the same; neighboring words and sentence structure change the correct tag. This is POS ambiguity.

SentenceTagged interpretationReason
They watch birds.They/PRP watch/VBP birds/NNSwatch is the action.
The watch stopped.The/DT watch/NN stopped/VBDwatch is the object that stopped.
She has a light bag.She/PRP has/VBZ a/DT light/JJ bag/NNlight describes bag.
Light the lamp.Light/VB the/DT lamp/NNLight is an imperative verb.

POS tags support parsing, named entity recognition, question answering, sentiment analysis and word-sense disambiguation. They do not by themselves identify every entity or fully resolve meaning.

Yaad rakho: Word ka naam same ho sakta hai; sentence mein uska kaam tag decide karta hai.

","m2":false,"units":"2,4","marks":6,"sourcePages":[1],"historical":true},{"id":"q2b","title":"Q2b · Lexical similarity and cosine","html":"
Question. Explain the need for lexical similarity and illustrate cosine similarity. [4 marks]

Answer

Similarity supports retrieval, clustering, duplicate detection and semantic matching. Choose a representation first: character overlap measures spelling; context embeddings approximate semantic relatedness. An illustrative cosine computation follows; these vectors are chosen to explain the method, not given exam data.

Given

u=[1, 1, 0]; v=[1, 0, 1]

To find

Cosine similarity in the given coordinate order.

Formula: which one and why

\"\\cos(u,v)=\\frac{u\\cdot

Step 1. Multiply matching coordinates

1×1 + 1×0 + 0×1 = 1

Step 2. Calculate both squared lengths

u: 1² + 1² + 0² = 2; v: 1² + 0² + 1² = 2

Step 3. Take square roots and divide

1 / (√2 × √2) = 0.500000000

Final answer

Cosine=0.500000000. This measures direction, not a probability.

","m2":false,"units":"2,5","marks":4,"sourcePages":[1],"historical":true},{"id":"q2c","title":"Q2c · Animal categorical Naive Bayes","html":"
Question. Using the printed20-animal training table, classify an animal with Give Birth=Yes, Can Fly=No, Live in Water=Yes, Have Legs=No. Treat Name only as an identifier. [10 marks]

Answer

Rows below follow the source order: Human, Python, Salmon, Whale, Frog, Komodo, Bat, Pigeon, Cat, Leopard Shark, Turtle, Penguin, Porcupine, Eel, Salamander, Gila Monster, Platypus, Owl, Dolphin, Eagle. The source assigns Leopard Shark to Non-mammals and Platypus to Mammals; retain those labels.

Given

Give BirthCan FlyLive in WaterHave LegsClass
YesNoNoYesMammals
NoNoNoNoNon-mammals
NoNoYesNoNon-mammals
YesNoYesNoMammals
NoNoSometimesYesNon-mammals
NoNoNoYesNon-mammals
YesYesNoYesMammals
NoYesNoYesNon-mammals
YesNoNoYesMammals
YesNoYesNoNon-mammals
NoNoSometimesYesNon-mammals
NoNoSometimesYesNon-mammals
YesNoNoYesMammals
NoNoYesNoNon-mammals
NoNoSometimesYesNon-mammals
NoNoNoYesNon-mammals
NoNoNoYesMammals
NoYesNoYesNon-mammals
YesNoYesNoMammals
NoYesNoYesNon-mammals

Test: {'Give Birth': 'Yes', 'Can Fly': 'No', 'Live in Water': 'Yes', 'Have Legs': 'No'}

To find

Class under categorical Naive Bayes.

Formula: which one and why

\"\\mathrm{score}(c)=P(c)\\prod_jP(x_j\\mid

Use empirical categorical frequencies, not a multinomial word model. No likelihood is zero for the requested test, so unsmoothed estimates are sufficient. Each feature contributes once; ignore names/identifiers as predictors.

Step 1. Count examples in each class

ClassCountPrior
Mammals77/20
Non-mammals1313/20

Step 2. Count each requested feature within each class and multiply

ClassnPriorGive BirthCan FlyLive in WaterHave LegsExact productDecimal product
Mammals77/206/76/72/72/736/17150.020991253644
Non-mammals1313/201/1310/133/134/136/21970.002730996814

Step 3. Normalize for a posterior if wanted

ClassScore / score sum
Mammals0.884876150
Non-mammals0.115123850

Final answer

Predicted class: Mammals. The independence assumption is conditional on class, not a claim that these biological/weather features are truly independent.

","m2":false,"units":"7","marks":10,"sourcePages":[2],"historical":true},{"id":"q3a","title":"Q3a · Hobbs and ambiguous professor–student reference","html":"
Question. Explain Hobbs’ algorithm, including binding, gender, distance and recency. Resolve “The professor advised the student because she was struggling.” [10 marks]

Answer

Hobbs’ algorithm is a syntax-based procedure that searches constituent parse trees in a fixed order for a compatible noun-phrase antecedent of a pronoun. The tree determines which candidates are proposed first. Agreement and other relevant constraints determine whether a proposed candidate can be accepted.

It is not simply the rule “choose the closest noun”. A nearby noun may have the wrong number, may be syntactically ineligible, or may be visited after a structurally more prominent noun. The algorithm produces a preference under the supplied parse; ambiguous language can still require context.

The tree vocabulary you need

TermMeaningExample
NPNoun phrase: a phrase referring to an entitythe student; Anita; they
SA sentence/clause nodethe node containing a subject and predicate
VPVerb phrasetold Ravi that she should revise
XThe NP or S node currently being searchedAn algorithm label, not a word
Path pThe branch travelled upward from the pronoun/previous XUsed to distinguish left and right subtrees
N-bar (N′)The noun-centred part of an NP in a grammar using that labelThe nominal-child test depends on the given tree
Breadth-firstVisit shallower nodes before deeper onesLevel 1 before level 2
Left-to-rightVisit equally deep nodes in their written/tree orderEarlier left branch before right branch
Locate pronoun NP
→
Climb to NP/S
→
Propose candidates in tree order
→
Check compatibility
→
Accept or continue searching

The complete search procedure

Step 1. Start at the pronoun

Locate the NP immediately dominating the pronoun. “Dominating” means containing it as a descendant in the tree.

Step 2. Climb to the first NP or S

Move upward to the first NP or S above that starting NP. Call this node X. Keep the upward path p.

Step 3. Search the left branches initially

Below X, traverse branches to the left of p breadth-first and left-to-right. At this initial search, propose an NP only if another NP or S lies between that candidate and X. Test proposed candidates for the required agreement and constraints.

Step 4. If X is the highest sentence node, inspect earlier sentences

Search previous sentence trees from most recent to oldest. Within each, use breadth-first, left-to-right order to propose NPs. Accept the first eligible compatible candidate, or report that no candidate was found in the available context.

Step 5. Otherwise climb again

Move from X to the next NP or S above it. Record this new X and the path followed during the climb.

Step 6. Apply the higher-NP test

If the new X is an NP, propose X when the path did not pass through its nominal N-bar child. Use the given parse labels; do not invent an N-bar branch missing from the question's structure.

Step 7. Search its left branches

Traverse branches below the new X to the left of the path, breadth-first and left-to-right, proposing NPs. The special intervening-NP/S restriction belongs to the initial search in Step 3.

Step 8. If X is S, search the permitted right branches

Search branches to the right of the path breadth-first and left-to-right. Do not descend below an encountered NP or S. Propose eligible NP candidates there.

Step 9. Repeat or stop

Return to the highest-S check and continue the climb/search procedure until a compatible candidate is accepted or the available trees are exhausted. Record the candidate order rather than only writing an unexplained name.

Why the first left search has an extra condition

In a simple clause such as “John saw him”, the subject NP John is directly under the clause S. There is no extra NP or S between John and that S. The initial-search restriction therefore prevents casually selecting John as the ordinary local antecedent of him. Compare the reflexive “John saw himself”, which is a different grammatical case; do not treat every pronoun type as interchangeable.

Worked tree 1 · move to the previous sentence

Question. Apply Hobbs’ search to “The student arrived. She smiled.” Assume the student is compatible with she.

Given

Previous tree: [S [NP The student] [VP arrived]]. Current tree: [S [NP She] [VP smiled]].

To find

The selected antecedent and the search path.

Method and why it applies

Use the supplied constituent trees. The current sentence has no earlier candidate, so the highest-S rule activates previous-sentence search.

Solution

Step 1. Draw both sentence structures

Previous sentence             Current sentence\nS                             S\n├── NP: The student            ├── NP: She ← start\n└── VP: arrived                └── VP: smiled

Step 2. Locate and climb

Start at NP She. Its first higher NP/S is the current S. Set X to that S.

Step 3. Search left of the path

She is the leftmost subject. No NP lies to its left in the current sentence.

Step 4. Use the highest-S rule

This X is the highest S. Inspect the immediately preceding sentence tree.

Step 5. Propose its first NP

Breadth-first, left-to-right finds NP The student.

Step 6. Check compatibility and accept

The question stipulates that the student agrees with she. Accept this candidate.

Answer. She refers to the student under the supplied assumptions.

Worked tree 2 · reject an incompatible subject

Question. Resolve they in “Riya spoke to the students. They laughed.” Use a previous-sentence search and interpret they as plural.

Given

Previous tree has subject NP Riya, then VP spoke [PP to [NP the students]]. Current subject is NP They.

To find

The candidate order and accepted antecedent.

Method and why it applies

Breadth-first search favours a shallow subject, but agreement must still be applied; a first candidate is not automatically a valid answer.

Solution

Step 1. Draw the useful part of the previous tree

S\n├── NP: Riya                 ← candidate 1\n└── VP\n    ├── V: spoke\n    └── PP\n        ├── P: to\n        └── NP: the students ← candidate 2

Step 2. Search the current sentence

From NP They climb to its S; there is no earlier NP in “They laughed”. This is the highest S, so move to the previous tree.

Step 3. Visit Riya first

Riya is the shallower leftmost NP. Under the stated plural interpretation, this singular candidate fails number agreement.

Step 4. Visit the students

Continue breadth-first. The next relevant NP is the students.

Step 5. Apply number agreement

Students is plural and is an eligible candidate. It matches plural they.

Step 6. State the resolution

Accept the students. The method did not simply accept the first noun it saw.

Answer. They → the students. Candidate order: Riya rejected; the students accepted.

Worked tree 3 · climb out of an embedded clause

Question. Resolve she in “Anita told Ravi that she should revise.” Assume Anita is compatible with she and Ravi is not. Use the displayed parse.

Given

The pronoun is the subject of an embedded S inside the main VP.

To find

The preferred antecedent found by the climb and left-branch search.

Method and why it applies

The embedded clause has no left candidate. Climbing to the main S exposes its subject Anita as the first suitable higher-level candidate.

Solution

Step 1. Show the clause structure

S (main)\n├── NP: Anita                  ← accepted candidate\n└── VP\n    ├── V: told\n    ├── NP: Ravi\n    └── SBAR: that\n        └── S (embedded)       ← first X\n            ├── NP: she        ← start\n            └── VP: should revise

Step 2. Start inside the embedded clause

From NP she climb to the embedded S. There is no candidate to the left of she within this embedded S.

Step 3. Climb to the next NP/S

The embedded S is not the highest S. Move upward through SBAR and VP to the main S. These intermediate nodes are not NP/S search nodes.

Step 4. Search left of the upward path

At the main S, the subject NP Anita is a shallow left candidate. The object Ravi is deeper within the VP. Breadth-first order proposes Anita first.

Step 5. Check the stated compatibility

Anita is compatible with she by the question's assumption. Accept it; no need to choose a later incompatible candidate.

Step 6. Give the result and its limit

This is the algorithm's result under the displayed parse and agreement assumptions. It does not prove the intention of every speaker using an otherwise ambiguous sentence.

Answer. She → Anita under the stated tree and compatibility information.

Advantages, limitations and a complete exam answer

AdvantagesLimitations
Explicit candidate order makes the decision traceable.Wrong parse structure can produce the wrong search order.
Requires no labelled coreference-training corpus for its basic rules.World knowledge and discourse intention may override syntactic preference.
Uses structure rather than only surface distance.Some ambiguous, long-distance or unusual constructions need additional treatment.

For a longer answer, write the definition, inputs, tree notation, ordered algorithm, one labelled tree with a candidate table, final antecedent, and limitations. In a worked question show separately: candidate → why proposed → agreement result → accept/reject. Do not equate the selected candidate with a logically guaranteed interpretation when the sentence is ambiguous.

Yaad rakho: Tree order candidate deta hai; agreement usko allow/reject karta hai. Nearest noun automatically answer nahi.

The intended antecedent is not uniquely determined by the printed sentence. Both professor and student can be singular female people; neither noun supplies gender. Structural Hobbs search can propose the matrix subject professor early, while the causal meaning of advice because of struggling often supports student. Distance/recency favouring student is a preference, not proof.

  1. Parse subject NP the professor, object NP the student, subordinate-clause subject she.
  2. Generate structurally accessible earlier NPs through the Hobbs ascent and breadth-first leftward search.
  3. Check grammatical number/person and binding constraints. Both candidates can pass here; she is not a reflexive.
  4. Gender/world knowledge is unavailable, so do not invent either person’s gender.
  5. Use broader discourse to select. Without it, report professor/student ambiguity; explain that contextual interpretation often prefers student but the syntax-only algorithm does not guarantee that intent.

Source / convention note. Pronoun referent is ambiguous without extra gender/discourse information; structural algorithm preference is separated from semantic interpretation.

The four requested properties

PropertyWhat it meansApply here
BindingSyntactic restrictions on reference; ordinary personal pronouns and reflexives have different local-antecedent requirementsDo not mechanically link every nearby NP; he in an after/because clause can refer outside that local clause.
Gender agreementCandidate must match the pronoun under available linguistic/contextual knowledgeJohn/he is the exercise’s conventional reading; professor/student do not provide gender by themselves.
DistanceToken, syntactic or sentence separation between pronoun and candidateCloser candidates may be preferred, but cannot override incompatibility or Hobbs search order.
RecencyRecently mentioned discourse entities are often more salientMary/student is later than John/professor, but recency is a preference, not a guaranteed antecedent rule.
","m2":false,"units":"6","marks":10,"sourcePages":[2],"historical":true},{"id":"q3b","title":"Q3b · Classification and sentiment in industry","html":"
Question. Discuss real-world text classification and sentiment analysis, with examples showing decision-making and customer engagement. [10 marks]

Answer

Text classification assigns a document to one or more predefined categories. Spam detection usually uses two classes; news classification may use several topics. In single-label multiclass classification, choose one of several categories. In multilabel classification, several labels may apply to the same document.

Define labels
→
Prepare labelled examples
→
Train representation + model
→
Evaluate on unseen text
→
Predict new labels

An exam-ready pipeline

StageWhat you doWhy it matters
Task definitionDecide what counts as spam or which news topics exist.Unclear labels create inconsistent training data.
Data collection and annotationCollect representative examples and assign correct labels.The model learns from these examples.
Split dataSeparate training, validation and test data.Test examples must not influence training choices.
PreprocessTokenize and normalise where suitable; retain meaningful negation.Reduce irrelevant variation without deleting signal.
RepresentUse BoW, TF-IDF or learned embeddings.Models need numerical features.
TrainFit Naive Bayes, logistic regression or another appropriate model.Learn how features predict categories.
EvaluateMeasure relevant precision, recall and F1 on held-out examples.Accuracy alone may hide poor minority-class performance.
Deploy and monitorApply the same preprocessing and monitor changing language.Incoming data can differ from old examples.

Naive Bayes is especially useful for hand calculations. With a bag-of-words representation, its class score is the prior multiplied by the conditional probability of each token; repeated tokens repeat the factor. The “naive” assumption is conditional independence of features given the class, not that words are independent in real language.

\"s(c)=P(c)\\prod_{w\\in
\"P(w\\mid

C(w,d) counts word w in document d. C(w,c) counts it in training documents belonging to class c. Nc is the total token count for class c. Add-one smoothing adds one to every vocabulary count and therefore adds |V| to the denominator. This is the same arithmetic taught for WSD, with document classes instead of senses.

Worked classification · two-token message

A spam classifier has vocabulary {win, prize, meeting}. Priors are P(spam)=P(ham)=0.5. Spam token counts are [3,2,0]; ham counts are [0,0,5]. With add-one smoothing, classify “win prize”.

Given

|V|=3; N(spam)=5; N(ham)=5; message has win once and prize once.

To find

Class scores and predicted label.

Formula and why it applies

\"s(c)=P(c)P(\\text{win}\\mid
\"P(w\\mid

The model is multinomial Naive Bayes; multiply the two token likelihoods and the class prior.

Solution

Step 1. Compute the smoothed denominators

\"5+3=8\"

Step 2. Compute spam likelihoods

\"P(\\text{win}\\mid
\"P(\\text{prize}\\mid

Step 3. Compute ham likelihoods

\"P(\\text{win}\\mid

Step 4. Multiply the spam score

\"s(S)=\\frac12\\times\\frac12\\times\\frac38=\\frac{3}{32}=0.09375\"

Step 5. Multiply the ham score

\"s(H)=\\frac12\\times\\frac18\\times\\frac18=\\frac{1}{128}=0.0078125\"

Step 6. Compare

0.09375 > 0.0078125. These are unnormalised scores; the label is chosen by the larger score.

Predicted label: spam.

Yaad rakho: Class prior × har word ki class-likelihood; sabse bada score wins.

Sentiment analysis identifies expressed opinion or attitude, often as positive, negative or neutral. Document-level analysis labels a complete text; sentence-level analysis labels a sentence; aspect-level analysis attaches sentiment to a specific feature or entity.

ReviewUseful interpretation
“The display is excellent, but the battery is poor.”Display: positive; battery: negative. One overall label loses this distinction.
“The service is not good.”Negative: negation changes good. Removing not can reverse the intended meaning.
“Great, another two-hour delay.”Likely sarcasm: literal positive word alone is insufficient.

Approaches

Lexicon based: look up sentiment scores, combine them, and apply stated handling for negation/intensifiers. It is interpretable but fragile with context and sarcasm. Supervised: learn a mapping from labelled text features to sentiment labels. Neural/contextual: learn or use contextual representations to capture word interactions, then fine-tune a sentiment classifier. None guarantees correct interpretation of every review.

A specified lexicon calculation

Use good = +2, excellent = +3 and slow = −2. A “not” immediately before a sentiment word reverses its score. Score “The support was not good but the delivery was excellent and slow.”

Given

Three sentiment words; not modifies good only; sum word scores under the stated rule.

To find

Total sentiment score and label, taking positive totals as positive.

Formula and why it applies

\"S=\\sum_i

Apply the given negation rule before summing. This is a simplified specified model, not a universal linguistic rule.

Solution

Step 1. Identify sentiment words

good, excellent and slow.

Step 2. Apply negation

\"s(\\text{not

Step 3. Add the contributions

\"S=-2+3-2=-1\"

Step 4. Map score to label

The sum is negative.

Total score = −1; negative under this rule.
IndustryConcrete input and outputUseful decision / limitation
RetailReview→topic and sentimentRoute delivery complaints; sarcasm/negation may mislead
BankingCustomer message→card/fraud/loan intentRoute to the correct team; a model flag is not proof of fraud
HealthcareFeedback→service-quality topicsDetect recurring appointment issues; protect patient identifiers
MediaArticle→topic; public responses→sentimentOrganize feeds and monitor reaction; sampling bias prevents treating posts as a census
","m2":false,"units":"7","marks":10,"sourcePages":[3],"historical":true},{"id":"q4a","title":"Q4a · Top-down syntax tree","html":"
Question. Explain the need for syntax parsing; compare top-down and bottom-up parsing. Parse “the cat chased the dog” using S→NP VP, NP→Det N, VP→V NP, Det→the, N→cat|dog, V→chased|saw. [10 marks]

Answer

Top-down parsing starts at the grammar's start symbol S and expands nonterminals until the leaves match the input words in order. Recursive descent commonly implements each nonterminal as a procedure that tries its productions. It builds from the root toward the leaves.

Start with S
→
Choose an allowed S production
→
Expand its next nonterminal
→
Match the resulting terminal to the input
→
Continue, or backtrack if the choice fails

Algorithm. Start with S as the current goal. Choose a production for the leftmost unexpanded nonterminal. Replace that nonterminal by the production's right side. If a terminal appears, compare it with the next input token. If it matches, consume the token; if it does not, reject that branch and try another production. Accept only when the complete input is consumed and all grammar goals are satisfied.

A leftmost derivation always expands the leftmost remaining nonterminal first. Top-down parsing can make guesses that the actual words cannot support, so it may need backtracking. A naive recursive-descent parser can loop on left-recursive rules such as NP→NP PP unless the grammar or algorithm is changed. Predictive LL(1) parsing chooses productions using one lookahead token when the grammar permits; not every English CFG is LL(1).

ApproachBegins withDirectionCommon difficulty
Top-downStart symbol SRoot to word leavesPredictions that do not match input; left recursion
Bottom-upActual input tokensLeaves to root SReductions that cannot contribute to a complete sentence

Yaad rakho: Top-down: sentence ka structure pehle, words tak neeche aao.

Bottom-up parsing starts from the input tokens and combines them into larger constituents until the start symbol is reached. A shift-reduce parser maintains a stack of recognized material and a list of remaining input tokens.

ActionExact operationExample
ShiftMove the next input token onto the stackStack: the; remaining input: dog runs
ReduceReplace a suffix of the stack matching a rule's RHS with its LHSDet N becomes NP using NP→Det N
AcceptStack is S and input is exhaustedStack: S; input: empty
Fail/backtrackChosen reductions cannot complete a parseA noun reading of book blocks an imperative parse

Reduction uses a production in reverse: the grammar NP→Det N expands an NP top-down, but reduces Det N into NP bottom-up. The stack's rightmost item is its top in the traces below. Each row shows the state after the listed action, avoiding an off-by-one mismatch between actions and stack contents.

A lexical reduction such as N→dog still counts as a reduction. Do not skip the sequence from word to category to phrase when the question asks for a complete trace. Backtracking means returning to an earlier choice and trying another applicable rule, not changing the original input.

Yaad rakho: Shift = input se stack. Reduce = chhote pieces se bada grammatical unit.

Exact top-down derivation

  1. S
  2. NP VP
  3. Det N VP
  4. the N VP
  5. the cat VP
  6. the cat V NP
  7. the cat chased NP
  8. the cat chased Det N
  9. the cat chased the N
  10. the cat chased the dog
SNPDettheNcatVPVchasedNPDettheNdog
[S [NP [Det the] [N cat]] [VP [V chased] [NP [Det the] [N dog]]]]

Parsing assigns constituent structure and grammatical relationships for extraction, translation and ambiguity resolution. Top-down expands from S toward input; bottom-up combines observed tokens toward S. This derivation uses only supplied rules.

","m2":false,"units":"4","marks":10,"sourcePages":[3],"historical":true},{"id":"q4b","title":"Q4b · Data acquisition and four augmentation methods","html":"
Question. Explain NLP data acquisition, why augmentation is needed, and any four augmentation methods. [10 marks]

Answer

Data acquisition obtains language examples relevant to the task from existing data, public resources or appropriate new collection. Product instrumentation can gather useful feedback. Supervised learning additionally needs an annotation process when labels are absent.

Data augmentation creates training variants intended to preserve the correct label. It is not simply changing words arbitrarily.

MethodProcedureCaution
Synonym replacementUse suitable synonyms for selected wordsChoose the right sense
Back translationTranslate to another language and backMay change meanings/entities
TF–IDF-informed replacementProtect informative words while varying othersDo not remove label-bearing clues
Bigram flippingSwap adjacent wordsCan break grammar or meaning
Entity replacementReplace by same-type entityMust preserve the task label
Noise injectionSimulate typos/keyboard errorsMatch expected noise
Active learningSelect informative cases for human labelsThis acquires labels; it is not synthetic text

Three label decisions

“The film is good”→“The film is excellent” may preserve a positive sentiment label. “The film is not good”→“The film is good” reverses the label and is not valid label-preserving augmentation. “Flights to Pune”→“Flights to Mumbai” may preserve a travel-topic label but changes the destination label.

Split train/test data before creating related augmented variants. Otherwise near-duplicates can leak between sets and produce misleading evaluation. Keep a representative evaluation set of genuine examples.

Sentence badal sakta hai; required correct label nahi. Label badla toh augmentation invalid.

MethodExampleRequired label check
Synonym replacementgood film→excellent filmReplacement must preserve sense and sentiment
Back-translationEnglish→French→English paraphraseCheck negation, entity and factual preservation
Contextual paraphrasingThe delivery was late→The package arrived lateDo not change class or named facts
Controlled noise / perturbationSmall realistic spelling error or punctuation variationAppropriate only if target should remain unchanged

Acquire licensed/public datasets, commissioned annotation, surveys/user-consented logs or permitted scraping. Augment training data only after splitting to avoid near-duplicate leakage into validation/test. More synthetic examples do not guarantee better generalization.

","m2":false,"units":"3","marks":10,"sourcePages":[3],"historical":true},{"id":"q5a","title":"Q5a · Feature engineering, BoW and Euclidean","html":"
Question. Explain text feature engineering and list vectorization approaches. For the three sentences below remove stopwords {is,that,are,to,and}, create vocabulary and BoW, then compute Euclidean distance between sentences1and2: “Natural Language Processing is fascinating”; “I love Natural Language processing and like to study Natural Language”; “I like things that are fascinating”. [10 marks]

Answer

Feature engineering converts text into informative model inputs. Common approaches: one-hot, BoW, bag of n-grams, TF–IDF and dense embeddings. This question specifically requests count-based BoW, so do not use TF–IDF in its distance.

Given

Use lowercase lexical tokens and the exact printed stoplist; I remains because it is not a stopword here.

To find

Vocabulary, count matrix and distance D1–D2.

Formula: which one and why

\"d_E(u,v)=\\sqrt{\\sum_i(u_i-v_i)^2}\"

Step 1. Tokenize and remove the exact stoplist

DocumentRetained tokens
1natural, language, processing, fascinating
2i, love, natural, language, processing, like, study, natural, language
3i, like, things, fascinating

Step 2. Fix vocabulary and count repeats

Documentfascinatingilanguagelikelovenaturalprocessingstudythings
1101001100
2012112110
3110100001

Step 3. Subtract coordinates

[1, -1, -1, -1, -1, -1, 0, -1, 0]

Step 4. Square, sum and take the root

1 + 1 + 1 + 1 + 1 + 1 + 0 + 1 + 0 = 7; √7 = 2.645751311

Final answer

Euclidean distance=2.645751311. Natural and Language each occur twice in sentence2; preserve both counts.

","m2":false,"units":"2","marks":10,"sourcePages":[3],"historical":true},{"id":"q5b","title":"Q5b · WSD approaches and uses","html":"
Question. Explain WSD, its approaches and applications. [5 marks]

Answer

Word sense disambiguation (WSD) is the task of selecting the intended sense of an ambiguous word in its context from a set of possible senses. The output is a sense label, not a different replacement word.

Input: “He grilled the bass.”
→
Candidate senses: fish / low sound
→
Context: grilled
→
Output: fish sense

The target word is the ambiguous word. The context is the surrounding evidence. The sense inventory lists candidate meanings, often using WordNet. A feature is a usable clue: nearby word, position, part-of-speech tag or a phrase.

ApproachWhat it needsHow it decidesLimitation
Knowledge basedDictionary, thesaurus or semantic graphDefinition overlap, category evidence or graph scoresShort definitions and missing knowledge
SupervisedExamples already labelled with sensesLearns context → sense mappingLabelling cost, limited rare-sense examples
Semi-supervisedSmall labelled set plus unlabelled dataBootstraps from reliable seed examplesEarly errors can spread
Unsupervised sense inductionUnlabelled contextual occurrencesClusters similar contextsClusters are not automatically named dictionary senses
Neural/contextualLearned word/context representationsPredicts or compares a context representationNeeds suitable data/model and evaluation

Bag-of-words features record nearby words while ignoring exact positions. Collocational features preserve a positional phrase such as “river bank”, “bank account”, “chocolate bar” or “bar exam”.

Why WSD matters

Machine translation needs the correct target-language meaning; information retrieval should distinguish river-bank travel from banking services; text-to-speech may need different pronunciations for “bass”; information extraction should avoid connecting entities through the wrong relation.

WSD: same spelling ke options mein context dekh ke sahi meaning select.

","m2":false,"units":"5","marks":5,"sourcePages":[3],"historical":true},{"id":"q5c","title":"Q5c · Discourse analysis of a conversation","html":"
Question. Analyze coherence, cohesion, topic structure, information flow and discourse markers: Speaker1 “I went to the store to buy some milk.” Speaker2 “Was it crowded?” Speaker1 “Yes, I had to wait in line for10minutes.” Speaker3 “I was going to attend the meeting, but I got stuck in traffic.” [5 marks]

Answer

FeatureEvidence and explanation
CoherenceFirst3turns form a meaningful store/crowding/waiting exchange. The meeting/traffic turn shifts topic without an explicit bridge; local relevance is weaker.
Cohesionit refers to the store; Yes answers the preceding question; first-person I shifts referent with each speaker.
Topic structureStore visit→crowding→wait; then new topic of meeting attendance and traffic.
Information flowKnown store is followed by new question crowding, then new evidence10-minute wait. The last turn introduces separate plans and obstacle.
Discourse markersYes signals affirmation; but signals contrast between intention and obstructing traffic. “To buy” states purpose, not merely temporal order.

Cohesion is explicit linguistic linkage; coherence is meaningful conceptual connection. A conversation can contain grammatical connectors and still have a poorly connected topic shift.

","m2":false,"units":"6","marks":5,"sourcePages":[4],"historical":true},{"id":"q6a","title":"Q6a · Henry bigram continuations","html":"
Question. From the seven supplied training sentences below, predict the most probable next token for <s> Do ..., <s> I like ..., and <s> I ... using bigrams. [10 marks]

Answer

Given

<s> I am Henry </s>
<s> I like college </s>
<s> Do Henry like college </s>
<s> Henry I am </s>
<s> Do I like Henry </s>
<s> Do I like college </s>
<s> I do like Henry </s>

To find

Most probable next token for each prefix.

Formula: which one and why

\"P(w\\mid

Retain case exactly; include sentence-end as an observed next token. Bigram prediction uses only the last token of the prefix.

Step 1. Count each adjacent pair separately within sentences

BigramCount
<s> → Do3
<s> → Henry1
<s> → I3
Do → Henry1
Do → I2
Henry → </s>3
Henry → I1
Henry → like1
I → am2
I → do1
I → like3
am → </s>1
am → Henry1
college → </s>3
do → like1
like → Henry2
like → college3

Step 2. Continue <s> Do

CandidateCount / history countProbability
Henry1/30.333333
I2/30.666667

Highest: I.

Step 3. Continue <s> I like

CandidateCount / history countProbability
Henry2/50.400000
college3/50.600000

Highest: college.

Step 4. Continue <s> I

CandidateCount / history countProbability
am2/60.333333
do1/60.166667
like3/60.500000

Highest: like.

Final answer

<s> Do → I
<s> I like → college
<s> I → like

If an examiner explicitly restricts the answer to a lexical word, exclude END and state that restricted candidate set. Do not silently remove an END tie.

Source / convention note. Case is retained as printed: Do and do are distinct. The corpus is not silently grammatically corrected.

","m2":false,"units":"3","marks":10,"sourcePages":[4],"historical":true},{"id":"q6b","title":"Q6b · Five linguistic levels","html":"
Question. Explain the contribution of phonology, morphology, syntax, semantics and pragmatics to language understanding. [5 marks]

Answer

A phoneme is a sound unit that can distinguish meanings in a language, such as the /p/ and /b/ contrast in “pat” and “bat”. A phoneme is a sound, not necessarily one alphabet letter. A morpheme is the smallest meaning-bearing grammatical unit: cats = cat + plural -s; unhappy = un- + happy.

A lexeme is an abstract vocabulary item grouping its inflected forms: write, writes, wrote and writing belong to WRITE. A phrase is a group of words functioning together; a clause normally contains a predicate and its associated participants; a sentence expresses a complete grammatical unit.

LevelWhat it analysesConcrete example
Phonetics / phonologyPhysical sounds / sound patterns in a languageDistinguish bat from pat
Morphology / lexical analysisWord structure and lexical itemsun + help + ful → unhelpful
SyntaxGrammatical arrangement and relationshipsThe dog chased the cat: subject, verb, object
SemanticsMeaning of words and composed expressionsThe dog is the chaser; the cat is chased
DiscourseConnections across sentences“Riya arrived. She smiled.” → She refers to Riya
PragmaticsIntended meaning in the situation“It is cold here” may request closing the window
word forms
→
grammatical structure
→
sentence meaning
→
cross-sentence context
→
speaker intention

Three worked interpretations

“The cats are sleeping.” Morphology identifies cat + s as plural. Syntax connects “cats” to “are sleeping”. Semantics represents animals engaged in sleeping.

“Aman bought a laptop. It was expensive.” Sentence-level syntax alone does not identify “it”. Discourse connects “it” to the laptop across the sentence boundary.

“Wonderful, another power cut!” The word wonderful is positive literally, but situational context can indicate sarcasm and frustration. This is a pragmatic interpretation.

The analysis levels are conceptual descriptions. Modern systems may learn several jointly rather than executing a compulsory separate module for every level.

Morphology = word ke andar; syntax = sentence ka structure; semantics = meaning; discourse = sentences ka link; pragmatics = intention.

","m2":false,"units":"1","marks":5,"sourcePages":[4],"historical":true},{"id":"q6c","title":"Q6c · NLU and NLG working together","html":"
Question. Explain how NLU and NLG work together for meaningful human–machine communication. [5 marks]

Answer

Natural language understanding (NLU) maps language into an interpretation useful for a task: intents, entities, semantic relations or another meaning representation. Natural language generation (NLG) produces human-language output from data, a representation or another source.

“Book a ticket to Pune tomorrow.”
→
NLU: intent=book; destination=Pune; date=tomorrow
→
Application: search available journeys
→
NLG: “Here are the available journeys to Pune.”
FeatureNLUNLG
DirectionLanguage → interpretationData/representation → language
Main concernMeaning, intent, ambiguityContent choice, organization, fluent wording
ExampleFind destination in a requestWrite the booking confirmation
Common failureWrong destination or senseUnclear, unsupported or ungrammatical output

NLG stages in a classical pipeline

  1. Content determination: choose facts to communicate.
  2. Data interpretation: identify relationships or trends.
  3. Document planning: organize facts in a useful order.
  4. Sentence aggregation: combine related facts without repetition.
  5. Lexical and grammatical realization: choose words and construct grammatical sentences.
  6. Presentation: format the generated message for its audience.

For a weather report, facts {city=Pune, temperature=28°C, rain=yes} can become “Pune will have rain with a temperature of 28°C.” NLU would instead extract facts or intent from an incoming language message.

NLU = language se meaning andar. NLG = data se language bahar.

","m2":false,"units":"1","marks":5,"sourcePages":[4],"historical":true},{"id":"q7a","title":"Q7a · POS, chunks and boundary-aware HMM","html":"
Question. Define function/content words, chunking/chinking and POS applications. Construct transition and emission tables including start/end for: Book/Verb a/Det flight/Noun; Read/Verb the/Det book/Noun; The/Det book/Noun is/Modal-Verb green/Noun. [10 marks]

Answer

Part-of-speech (POS) tagging assigns a grammatical category to every word token in context, such as noun, verb, adjective or determiner. It is a sequence-labeling task: the input is a sequence of words, and the output is a tag for each word. A tag describes the word's role in this sentence, not its meaning in every possible sentence.

TagRoleExample in context
NN / NNSSingular / plural common nouna book / two books
NNPProper nounMary, Mumbai
VB / VBD / VBZBase / past / third-person singular verbread / read / reads
JJAdjectivethe red book
RBAdverbruns quickly
DTDeterminerthe, a, this
INPreposition or subordinating conjunctionon the table
PRPPersonal pronounshe, they
MDModal auxiliarycan, will, should

Open classes readily accept new content words: nouns, verbs, adjectives and adverbs. Examples include chatbot and livestream. Closed classes contain comparatively stable grammatical/function words: determiners, pronouns, prepositions and conjunctions. These are tendencies, not a claim that language never changes.

Input: Book that flight
→
Context: an imperative request
→
Output: Book/VB that/DT flight/NN

Compare “hand me that book”: book is a noun here. Similarly, “I can swim” has can/MD, while “the can is empty” has can/NN. The spelling stays the same; neighboring words and sentence structure change the correct tag. This is POS ambiguity.

SentenceTagged interpretationReason
They watch birds.They/PRP watch/VBP birds/NNSwatch is the action.
The watch stopped.The/DT watch/NN stopped/VBDwatch is the object that stopped.
She has a light bag.She/PRP has/VBZ a/DT light/JJ bag/NNlight describes bag.
Light the lamp.Light/VB the/DT lamp/NNLight is an imperative verb.

POS tags support parsing, named entity recognition, question answering, sentiment analysis and word-sense disambiguation. They do not by themselves identify every entity or fully resolve meaning.

Yaad rakho: Word ka naam same ho sakta hai; sentence mein uska kaam tag decide karta hai.

Chunking, also called shallow parsing, groups adjacent tagged words into meaningful non-overlapping units, commonly noun phrases. It usually identifies local phrase boundaries without producing the complete hierarchical structure of a sentence. Chinking removes specified tokens from an existing candidate chunk, splitting or trimming it.

The/DT small/JJ cat/NN sat/VBD on/IN the/DT mat/NN
→
NP: [The small cat]
→
Outside: sat on
→
NP: [the mat]

A POS tagger labels individual words. A chunker groups words into phrases. A full parser shows how phrases relate recursively to form the entire sentence. These are related tasks with different outputs.

OperationInputOutput
POS taggingThe small catThe/DT small/JJ cat/NN
NP chunkingTagged tokens[NP The small cat]
ChinkingA broad candidate chunkRemove verbs/prepositions, keep noun-phrase segments
Full parsingSentence + grammarA nested tree with sentence, noun phrase and verb phrase nodes

Rule-based chunking uses tag patterns. Statistical or learned chunkers infer boundaries from labeled examples. A useful NP rule is an optional determiner, followed by zero or more adjectives, followed by a noun. In NLTK-style notation: NP: {<DT>?<JJ>*<NN>}. Here ? means zero or one occurrence; * means zero or more; + means one or more when used. The braces mean “form a chunk around the matched sequence”.

Question. Apply the NP rule optional DT, zero or more JJ, then one NN to “The/DT quick/JJ brown/JJ fox/NN jumps/VBZ over/IN the/DT lazy/JJ dog/NN”.

Given

Tagged input is supplied. Rule: DT? JJ* NN.

To find

All noun-phrase chunks.

Formula and why it applies

NP → DT? JJ* NN

The rule groups a noun with its preceding determiner and adjectives; a verb or preposition does not match this NP pattern.

Solution

Step 1. Start at The

The matches optional DT; quick and brown both match JJ*; fox matches NN.

Step 2. Form the first NP

[NP The quick brown fox]

Step 3. Skip nonmatching tokens

jumps/VBZ and over/IN do not satisfy the NP pattern.

Step 4. Match the second NP

the is DT, lazy is JJ, dog is NN: [NP the lazy dog].

Final answer. [NP The quick brown fox] jumps over [NP the lazy dog].

Question. Using NP → DT? JJ* NN, identify the chunks in “bright/JJ stars/NN shine/VBP and/CC a/DT moon/NN glows/VBZ”. Use the supplied tags exactly.

Given

Input tags label stars as NN for this exercise. Rule requires optional DT, zero or more JJ, then NN.

To find

The matched NP spans.

Formula and why it applies

NP → DT? JJ* NN

The optional determiner may be absent and the adjective repetition may be empty.

Solution

Step 1. Match bright stars

No determiner is present, which DT? permits. bright matches JJ; stars matches NN.

Step 2. Inspect shine and and

VBP and CC fall outside the NP rule.

Step 3. Match a moon

a matches DT; no adjective is present; moon matches NN.

Final answer. Chunks are [NP bright stars] and [NP a moon]. With ordinary Penn tagging, stars is usually NNS; broaden the noun pattern to include NNS when that tag is used.

Question. Begin with one candidate chunk containing “The/DT cat/NN sat/VBD on/IN the/DT mat/NN”. Chink out verbs and prepositions. Show the remaining chunks.

Given

Initial candidate: [The cat sat on the mat]. Remove tags VBD and IN.

To find

The chunk segments after exclusion.

Formula and why it applies

Chinking removes matched tokens from the existing chunk.

Removing the middle verb/preposition region divides the original candidate into two remaining contiguous spans.

Solution

Step 1. Locate the tokens to remove

sat is VBD and on is IN.

Step 2. Remove those tokens from chunk membership

The remaining token groups are “The cat” and “the mat”. sat and on still exist in the sentence; they are simply outside these chunks.

Step 3. Write the final chunked sentence

[The cat] sat on [the mat].

Final answer. Two chunks remain: The cat; the mat.

Yaad rakho: Chunking = andar group karo; chinking = unwanted tokens ko group se bahar karo.

Exact HMM table

Given

SentenceSupplied tags
Book a flightVerb Det Noun
Read the bookVerb Det Noun
The book is greenDet Noun Modal-Verb Noun

To find

Transition and emission probability tables including START and END.

Formula: which one and why

\"a_{ij}=\\frac{c(t_i,t_j)}{c(t_i)},\\qquad

For START divide by sentence count. END has no outgoing or emission distribution. Keep the supplied tags even when their linguistic choice is unusual, and retain case-sensitive Book/book and The/the as printed.

Step 1. Add boundary tags to every sentence

START Verb Det Noun END
START Verb Det Noun END
START Det Noun Modal-Verb Noun END

Step 2. Count transitions

From → toCount
START → Verb2
Verb → Det2
Det → Noun3
Noun → END3
START → Det1
Noun → Modal-Verb1
Modal-Verb → Noun1

Step 3. Divide by each source tag count

FromDenominatorVerbDetNounModal-VerbEND
START32/31/3000
Verb201000
Det300100
Noun40001/43/4
Modal-Verb100100

Step 4. Count and normalize emissions

TagCountBookaflightReadthebookTheisgreen
Verb21/2001/200000
Det301/3001/301/300
Noun4001/4001/2001/4
Modal-Verb1000000010

Final answer

Every nonterminal transition row sums to1 when END is included; every emission row sums to1. Boundary symbols do not emit lexical words.

POS applications include syntax parsing, disambiguation, lemmatization, chunking and extraction. For this numerical retain printed green=Noun and is=Modal-Verb even though ordinary linguistic tagging would differ; the task is estimation from supplied labels.

Source / convention note. Printed tags are used as training data; START and END are explicitly required and included.

","m2":false,"units":"4","marks":10,"sourcePages":[5],"historical":true},{"id":"q7b","title":"Q7b · Co-occurrence, dot products and plot","html":"
Question. Define target words, context window, context words and co-occurrence matrix. For the printed boy/ball/dog/sun corpus below, use one whole sentence as the window; contexts={kicked,field,tree,ran}. Find boy·dog and boy·sun, plot on kicked/field dimensions and comment. [10 marks]

Answer

Target is the word whose row we represent. Context is an observed neighbouring word whose column we count. A window defines which tokens count as neighbours; here all other tokens in the same sentence. A co-occurrence matrix records those counts.

Given

The boy kicked the ball into the field. The dog barked at the boy. The sun was shining over the field. The boy ran after the dog. The ball rolled to the tree.

To find

Matrix, two dot products and the requested 2D coordinates.

Formula: which one and why

\"u\\cdot

Use lowercase token-pair counts within each sentence; do not cross sentence boundaries.

Step 1. List contributions and build the matrix

SentenceTarget occurrencesContext occurrences
1boy, ballkicked, field
2dog, boy
3sunfield
4boy, dogran
5balltree
Target / contextkickedfieldtreeran
boy1101
ball1110
dog0001
sun0100

Step 2. Dot boy and dog

[1, 1, 0, 1] · [0, 0, 0, 1] = 1×0 + 1×0 + 0×0 + 1×1 = 1

Step 3. Dot boy and sun

[1, 1, 0, 1] · [0, 1, 0, 0] = 1×0 + 1×1 + 0×0 + 1×0 = 1

Step 4. Plot coordinates: kicked on x, field on y

Target(x,y)
boy(1, 1)
ball(1, 1)
dog(0, 0)
sun(0, 1)
kickedfielddog (0,0)sun (0,1)boy, ball (1,1)
Two coordinates omit tree and ran, so boy and ball overlap here despite different full vectors.

Final answer

boy·dog=1; boy·sun=1. Both share one counted context contribution. Dot-product similarity reflects this corpus and magnitude, not a claim that boy is equally semantically related to dog and sun in general.

","m2":false,"units":"5","marks":10,"sourcePages":[5],"historical":true}]},{"id":"legacy-r24","title":"2024-02-12 · re-exam · recovered AI paper","unit":"PYQ","scope":"Previous examination · complete worked answers","sections":[{"id":"q1a","title":"Q1a · Anaphora versus cataphora","html":"
Question. Compare anaphoric and cataphoric reference with examples. [5 marks]

Answer

Pragmatic analysis interprets an utterance using its situation and the speaker’s intended meaning. Discourse analysis studies how utterances connect across a text or conversation. Syntax and literal word meanings alone cannot settle every interpretation.

TextLiteral contentContextual interpretation
“Can you close the window?”A question about ability.Usually a polite request.
“Meera bought a laptop. It was expensive.”Two sentences.“It” connects the second sentence to the laptop.
“Meet me here tomorrow.”An instruction.Me, here and tomorrow depend on speaker, place and date.

Reference, mention and antecedent

A referring expression points to an entity; a mention is its occurrence in the text. Its referent is the entity being discussed. An antecedent is a preceding expression used to interpret a later referring expression. Reference resolution identifies the intended referent. Coreference resolution groups mentions that refer to the same entity.

“Meera”name mention
→
Person entitysame referent
→
“she” / “the engineer”other mentions
ConceptDefinitionExample
AnaphoraReference back to something introduced earlier.“Riya arrived. She sat down.” She → Riya.
CataphoraReference forward to a later expression.“Before she spoke, Riya checked the microphone.” she → Riya.
ExophoraReference to the situation outside the text.“Put it there”, said while pointing.
CoreferenceTwo or more mentions refer to the same entity.“Dr Rao”, “the lecturer”, “she”.
Pronoun resolutionFind the referent of a pronoun.Resolve “it” in “The laptop stopped because it overheated.”

Coreference is broader than pronoun resolution: repeated names and descriptions also form a chain. Anaphora describes direction/dependence; coreference describes identity. In “I bought a car. The engine is noisy”, engine is related to the car but is not the same entity.

Yaad rakho: Reference = kiski baat? Coreference = alag mentions, same entity. Pronoun resolution = he/she/it kis se linked?

Cohesion, coherence and discourse markers

\n

Cohesion is the explicit linguistic linking of a text through reference, repeated or related words, conjunctions and other devices. In “Rita bought a book. She read it”, she and it connect the second sentence to Rita and the book.

\n

Coherence is the conceptual unity that makes a discourse meaningful. “The road flooded, so the match was cancelled” presents a plausible cause and consequence. A reader relates the propositions using context and knowledge.

\n

Discourse markers signal relationships or organize speech: however signals contrast; therefore signals a result; first and next signal sequence; for example introduces an illustration.

\n

These concepts differ: “I like apples. Therefore the moon is square” has a linking word but no sensible inference. A connector alone does not guarantee coherence.

\n

Yaad rakho: Cohesion = words se link; coherence = meaning se link; marker = relation ka signal.

","m2":false,"units":"6","marks":5,"sourcePages":[1],"historical":true},{"id":"q1b","title":"Q1b · Five corpora","html":"
Question. Define corpora and discuss any five corpora used in NLP. [5 marks]

Answer

A corpus is a collection of language data assembled for study or computational use. Its plural is corpora. A corpus can contain books, news, reviews or conversations. It need not be labelled. Annotation adds linguistic information such as POS tags, entities, parse structures or sense labels.

StructureMeaningUse
PlainCollection without category labelsWord counts, language modelling
CategorizedDocuments grouped by genre/topicClassification
Overlapping categoriesOne document has several categoriesMulti-topic news
TemporalDocuments indexed by timeLanguage change
AnnotatedLinguistic labels attachedTagging, parsing, WSD
ParallelAligned texts in different languagesTranslation

A tagset is the allowed label inventory. A treebank is a corpus annotated with syntax trees. Metadata records source, date, genre, language, annotation method and limitations. A common benchmark supports comparison under shared data and evaluation conditions.

Raw example: “Riya works in Pune.” POS annotation: Riya/PROPN works/VERB in/ADP Pune/PROPN. NER annotation: Riya/PERSON, Pune/LOCATION. Each annotation answers a different task.

A good corpus is relevant to its domain, sufficiently representative, consistently annotated and appropriately split. More data alone do not repair duplicated examples, incorrect labels or a mismatch between training and real use.

Corpus=text collection. Annotation=text ke saath labels. Metadata=collection ki information.

ResourceContentsTypical use
Gutenberg selectionLiterary/book textsLong-form language and vocabulary
Web textInformal domain-specific textStudy usage beyond edited books
Chat corpusConversational postsDialogue, informal forms
BrownEnglish texts across genresGenre comparison
ReutersNews with topic labelsText classification; overlapping topics
InauguralSpeeches indexed by timeVocabulary change across years
WordlistWords/names/stopwordsLexical filtering
Pronouncing dictionaryWords and phonetic formsSpeech, stress, rhyme

Gutenberg versus web/chat: book language does not represent every spelling or conversational pattern in messages. Brown versus Reuters: Brown emphasizes genres; Reuters news can have overlapping topic labels. Inaugural addresses: dates provide a time dimension.

Corpus sizes depend on version, subset and tokenization. Explain their purpose and structure rather than assuming a historical count is a permanent property.

Gutenberg=books; web/chat=informal; Brown=genres; Reuters=news topics; Inaugural=time.

","m2":false,"units":"3","marks":5,"sourcePages":[1],"historical":true},{"id":"q1c","title":"Q1c · Linguistic foundations","html":"
Question. Define linguistic foundation and explain its importance in NLP. [5 marks]

Answer

A phoneme is a sound unit that can distinguish meanings in a language, such as the /p/ and /b/ contrast in “pat” and “bat”. A phoneme is a sound, not necessarily one alphabet letter. A morpheme is the smallest meaning-bearing grammatical unit: cats = cat + plural -s; unhappy = un- + happy.

A lexeme is an abstract vocabulary item grouping its inflected forms: write, writes, wrote and writing belong to WRITE. A phrase is a group of words functioning together; a clause normally contains a predicate and its associated participants; a sentence expresses a complete grammatical unit.

LevelWhat it analysesConcrete example
Phonetics / phonologyPhysical sounds / sound patterns in a languageDistinguish bat from pat
Morphology / lexical analysisWord structure and lexical itemsun + help + ful → unhelpful
SyntaxGrammatical arrangement and relationshipsThe dog chased the cat: subject, verb, object
SemanticsMeaning of words and composed expressionsThe dog is the chaser; the cat is chased
DiscourseConnections across sentences“Riya arrived. She smiled.” → She refers to Riya
PragmaticsIntended meaning in the situation“It is cold here” may request closing the window
word forms
→
grammatical structure
→
sentence meaning
→
cross-sentence context
→
speaker intention

Three worked interpretations

“The cats are sleeping.” Morphology identifies cat + s as plural. Syntax connects “cats” to “are sleeping”. Semantics represents animals engaged in sleeping.

“Aman bought a laptop. It was expensive.” Sentence-level syntax alone does not identify “it”. Discourse connects “it” to the laptop across the sentence boundary.

“Wonderful, another power cut!” The word wonderful is positive literally, but situational context can indicate sarcasm and frustration. This is a pragmatic interpretation.

The analysis levels are conceptual descriptions. Modern systems may learn several jointly rather than executing a compulsory separate module for every level.

Morphology = word ke andar; syntax = sentence ka structure; semantics = meaning; discourse = sentences ka link; pragmatics = intention.

","m2":false,"units":"1","marks":5,"sourcePages":[1],"historical":true},{"id":"q1d","title":"Q1d · Five regex operators","html":"
Question. For word-pattern detection, explain: ^abc, abc$, [A-Z0-9], ed|ing|s, [abc]. [5 marks]

Answer

PatternExact meaningPositive exampleImportant distinction
^abcabc at start of the input/line under chosen modeabcdefNot abc anywhere
abc$abc at endxxabcNot starts with abc
[A-Z0-9]One uppercase ASCII letter or digitQ or 7One character, not the whole string
ed|ing|sOne of the alternatives ed, ing or smatched edWithout anchors this is not necessarily a suffix
[abc]One a, b or cbNot the three-character string abc

Use full-string matching when the entire string must satisfy a pattern; a search may find only a substring.

","m2":false,"units":"3","marks":5,"sourcePages":[1],"historical":true},{"id":"q2a","title":"Q2a · Wordlist puzzle and lexical resource","html":"
Question. Explain lexical resources. Solve the wordlist puzzle with letters E,G,I / V,R,V / O,N,L. Words must have at least four letters, contain centre R, use each tile at most once, and include at least one nine-letter word. [10 marks]

Answer

Given

Column1Column2Column3
Row1EGI
Row2VRV
Row3ONL

To find

Find valid words and explain how a wordlist corpus supplies the candidates.

Formula: which one and why

A lexical resource stores words and linguistic information. Filter an explicitly chosen dictionary by length, mandatory letter and a multiset of tile counts; duplicate V tiles permit two Vs.

Step 1. Count available tiles

E1,G1,I1,V2,R1,O1,N1,L1.

Step 2. Apply all predicates

For each lowercase dictionary word w: length(w)≥4; r occurs; for every letter c, count_w(c)≤count_tiles(c). Deduplicate dictionary entries.

Step 3. Verify examples

WordReason
revolving9 letters: r e v o l v i n g; uses both V tiles and all others once
roving6 letters, includes R, within tile counts
lover5 letters, includes R
giver5 letters, includes R
groin5 letters, includes R
iron4 letters, includes R
ring4 letters, includes R
rove4 letters, includes R

Step 4. Explain the requested count

The count is the number of entries surviving those checks in the chosen wordlist. No wordlist/version is supplied, so a unique total cannot be stated. The examples above are verified examples, not an exhaustive dictionary.

Final answer

REVOLVING is a valid nine-letter answer. Enumerate all other valid entries using the declared wordlist.

Source / convention note. The dictionary is unspecified; an exact number of valid words would be fabricated.

","m2":false,"units":"3","marks":10,"sourcePages":[1,2],"historical":true},{"id":"q2b","title":"Q2b · NLP applications and limitations","html":"
Question. List NLP applications and explain limitations of NLP. [10 marks]

Answer

Natural language processing (NLP) is the field concerned with computational methods for analysing, interpreting and generating human language. Natural languages such as English, Hindi and Marathi evolved through human use; their meanings are not specified by the rigid rules of a programming language.

An NLP system may receive text, speech converted to text, or another linguistic representation. Its output depends on the task: a category, a tag for every word, a translated sentence, a ranked document list or a generated answer. It does not always “understand” in the human sense; its performance must be evaluated for the specific task.

TaskInputOutput
Sentiment analysis“The service was excellent.”Positive sentiment
Text classificationAn emailSpam / not spam
POS tagging“Birds fly.”Birds/NOUN fly/VERB
Named entity recognition“Asha joined Google in Mumbai.”Asha: person; Google: organisation; Mumbai: location
Machine translationA Hindi sentenceA corresponding English sentence
Information retrievalA search queryRanked relevant documents
Question answeringQuestion plus knowledge/contextAn answer
SummarizationA long articleA shorter text preserving key information

How NLP relates to AI, ML and DL

AI is the broader study of systems performing tasks associated with intelligent behaviour. Machine learning learns patterns from data. Deep learning uses multiple trainable neural layers. NLP is an application/research field that can use hand-written rules, statistical ML or deep learning. Therefore, not every NLP system is a neural network.

Linguistics supplies knowledge about language structure. Statistics supplies uncertainty models. Computer science supplies algorithms, storage and efficient implementation. A practical NLP system combines these ideas.

NLP = human language ko computational task mein badlo: label, meaning, search ya generated sentence.

Ambiguity means an expression supports more than one interpretation. Good exam answers name the ambiguity and explain both readings.

ChallengeExampleWhy a system can fail
Lexical/sense ambiguitybank: finance or river edgeOne written word has several senses
Syntactic ambiguityI saw the man with a telescope.The telescope may modify saw or the man
Reference ambiguityRiya told Neha that she won.She may refer to either person without more context
SarcasmGreat, another delay.Literal positive word conveys frustration
NegationThe film was not good.Dropping not reverses meaning
Domain shift“Positive” in a medical reportMeaning/use differs by domain
Code mixingKal meeting reschedule karna.Several languages and spelling conventions
Noisy texttmrw, repeated letters, typosVocabulary/token boundaries become uncertain
Low-resource settingFew labelled examples in a languageModels lack representative training evidence
Bias and privacySkewed data or personal detailsErrors and harms can persist in deployed systems

Homophones sound alike but may differ in spelling (pair/pear). Homographs share spelling but can differ in meaning or pronunciation (lead the team / lead metal). These are different dimensions from syntactic ambiguity.

Long-distance dependencies, cultural knowledge, idioms, morphology and multilingual writing systems also complicate processing. For “kick the bucket”, literal word-by-word interpretation can miss the idiomatic meaning. For “The book that the students recommended is useful”, the verb is agrees with book across an intervening phrase.

Exam mein challenge ka naam + ek sentence + dono meanings/actual failure likho.

","m2":false,"units":"1,7","marks":10,"sourcePages":[2],"historical":true},{"id":"q3a","title":"Q3a · Top-down restaurant parse","html":"
Question. Draw a top-down parse of “A restaurant serves dosas” using the printed grammar: S→NP VP | Aux NP VP | VP; NP→Pronoun | Proper-Noun | Det Nominal; Nominal→Noun | Nominal Noun | Nominal PP; VP→Verb | Verb NP | VP PP; PP→Prep NP. Lexicon: Det=A; Noun=restaurant,dosas; Verb=serves. [10 marks]

Answer

As printed, this grammar cannot derive the complete sentence. The object dosas is a Noun, but NP has no rule for a bare Nominal/Noun. NP→Det Nominal would require a determiner before dosas that is absent.

Clearly stated minimal repair

Add NP→Nominal. The repaired top-down derivation is:

StepSentential form
1S
2NP VP
3Det Nominal VP
4A Noun VP
5A restaurant Verb NP
6A restaurant serves Nominal
7A restaurant serves Noun
8A restaurant serves dosas
SNPDetANominalNounrestaurantVPVerbservesNPNominalNoundosas
[S [NP [Det A] [Nominal [Noun restaurant]]] [VP [Verb serves] [NP [Nominal [Noun dosas]]]]]

This tree uses the added rule only for the object. Do not silently pretend it is derivable under the original rule set. Left-recursive Nominal/VP alternatives require a terminating parser strategy; the displayed successful derivation chooses the needed finite branches.

Source / convention note. Original CFG is incomplete for the given bare-plural object. Exact failure and explicit repaired derivation are both shown.

","m2":false,"units":"4","marks":10,"sourcePages":[2],"historical":true},{"id":"q3b","title":"Q3b · Preprocessing pipeline","html":"
Question. Explain basic preprocessing steps on a blob of text with an example and pipeline diagram. [10 marks]

Answer

An NLP pipeline is an organized sequence of operations that transforms raw language into a task output and checks whether that output is useful. Not every task needs every cleaning operation.

Data acquisition
→
Preprocessing
→
Feature representation
→
Model
→
Evaluation
→
Deployment and monitoring
  1. Data acquisition: collect relevant text and, when needed, labels.
  2. Cleaning and preprocessing: handle markup, tokenize, normalize and decide which information to retain.
  3. Feature engineering: represent text numerically using BoW, TF–IDF or embeddings.
  4. Modelling: apply rules or train a statistical/neural model.
  5. Evaluation: test on held-out data using task-appropriate measures.
  6. Deployment: make the system available, log suitable feedback and monitor errors or data changes.

Concrete spam-filter example

Collect labelled emails → separate train/test data → tokenize text while preserving meaningful terms → fit vocabulary and IDF on training data → train classifier → evaluate spam detection on the held-out set → deploy an email service filter → monitor legitimate emails wrongly blocked.

Intrinsic evaluation measures model/task quality, such as precision, recall or tagging accuracy. Extrinsic evaluation measures effect in the wider application, such as whether users find answers faster. A higher internal metric does not automatically guarantee a better user outcome.

Data leakage: learning vocabulary statistics, labels or preprocessing decisions from the test set can make reported performance too optimistic. Fit learned transformations on training data, then apply them to validation/test data.

Collect → clean → numbers → model → test → use. Test data ko training ki information mat banao.

Normalization reduces superficial variation so comparable forms can be processed consistently. Common operations include lowercasing, Unicode normalization, appropriate spelling correction, contraction handling, and optional stemming or lemmatization.

Stopwords are frequent words selected for removal for a particular task, such as the or a. Removing them may reduce feature size, but removing not, no or nor can destroy sentiment meaning. Punctuation can convey sentence boundaries, emotion or mathematical structure. Lowercasing can erase differences such as US versus us.

Question. Preprocess “The CATS are not sleeping!!!” by lowercasing, removing punctuation, removing only stopwords {the,are}, and lemmatizing plural cats→cat and sleeping→sleep.

Given

Operations and stopword set are fixed. Do not remove not.

To find

The final token list.

Formula and why it applies

Apply the requested operations in order, keeping a visible intermediate result after each transformation.

Step 1. Lowercase

the cats are not sleeping!!!

Step 2. Tokenize and remove punctuation

[the,cats,are,not,sleeping]

Step 3. Remove specified stopwords

Remove the and are → [cats,not,sleeping].

Step 4. Lemmatize

cats→cat; sleeping→sleep → [cat,not,sleep].

Answer. [cat,not,sleep]. The negative meaning is retained.

Task decides preprocessing

For a simple keyword-retrieval model, removing selected function words can help. For machine translation, question answering, grammar analysis or transformer input, aggressive removal may discard necessary syntax or violate the model's expected input format. Emoji can carry sentiment; convert or retain them when relevant instead of always deleting them.

Clean karna = useful signal bachake noise hatao. Har word/punctuation delete karna cleaning nahi.

Stemming applies string-based rules, often stripping suffixes, to reduce related forms to a stem. The result need not be a valid dictionary word. Lemmatization uses vocabulary and morphological analysis, often with POS information, to return a dictionary lemma.

CriterionStemmingLemmatization
BasisSurface form/suffix rulesLexicon and morphological/POS analysis
OutputMay be nonword, e.g. studiDictionary form, e.g. study
Context/POSUsually little or noneOften important
Irregular formsMay miss went→goCan map went→go with verb analysis
CostGenerally simplerUsually more linguistic processing

Porter applies a sequence of suffix-removal rules subject to conditions; Snowball provides related language-specific stemming algorithms; Lancaster is often more aggressive; regex stemming uses explicit patterns; Lovins is another established stemming approach. Exact output depends on the algorithm and implementation.

Three worked reductions

  1. studies, studying: an English stemmer may yield studi/study depending on the rule sequence; verb lemmatization aims at study. State the exact algorithm if exact outputs are asked.
  2. better: adjective-aware lemmatization can map it to good; naïve suffix removal cannot infer that irregular relation.
  3. saw: as a past-tense verb it can become see; as a cutting-tool noun it remains saw. POS and context matter.

Over-stemming merges forms that should remain distinct; under-stemming fails to merge related forms. Normalization can increase recall in search while losing useful distinctions, so it is a task-dependent choice.

Stem = rules se cut; lemma = dictionary form, meaning/POS ka dhyan.

","m2":false,"units":"2","marks":10,"sourcePages":[2],"historical":true},{"id":"q4a","title":"Q4a · Embeddings and Skip-gram","html":"
Question. Explain word embeddings and the steps in Skip-gram with a suitable diagram. [10 marks]

Answer

A sparse representation has many zero entries and often one coordinate per vocabulary item. BoW and TF–IDF are common sparse document representations. A dense embedding has comparatively few learned real-valued coordinates; one word's properties are spread across many coordinates.

RepresentationWhat coordinates meanStrengthLimitation
One-hotOne identity position per vocabulary wordUnambiguous IDDifferent words are orthogonal; no learned similarity
Count-based context vectorContext-word co-occurrence countsTransparent corpus evidenceLarge, sparse; frequent words may dominate
TF–IDF document vectorWeighted vocabulary termsUseful retrieval weightingWord order and contextual meaning are limited
Dense word embeddingLearned latent featuresCompact semantic/syntactic relationshipsCoordinates are not generally human-labelled features

Distributional describes learning from patterns of linguistic context. Distributed describes representing an item across multiple coordinates rather than a single active identity bit. These are related but different ideas. A one-hot ID alone does not learn distributional similarity.

Model design choices

Choose contexts (document, sentence, fixed window), weighting (raw counts, normalization, TF–IDF-like weighting or association scores), dimensionality reduction or learned embeddings, and a similarity measure. Cosine compares direction and is less affected by length than a raw dot product; Euclidean distance also reflects magnitude.

Sparse = bahut zero. Dense = chhoti learned number-list. Similar context se useful similarity seekhte hain.

Word2Vec learns dense word embeddings by predicting words from nearby context. Its two common architectures are CBOW and skip-gram. A vocabulary ID or one-hot vector is the input representation; the learned embedding table converts that identity into a short trainable vector.

ArchitectureInputPredictionMemory cue
CBOWSurrounding context wordsCentre/target wordContext → centre
Skip-gramCentre/target wordSurrounding wordsCentre → context
CBOW: the + sat
→
Average context embeddings
→
Predict: cat
Skip-gram: cat
→
Embedding of cat
→
Predict: the and sat

For vocabulary size \"V\" and embedding dimension \"d\", an input one-hot row is \"1\\times; the input embedding matrix is \"V\\times; the hidden row is \"1\\times; the output matrix is \"d\\times; output scores are \"1\\times. Multiplying a one-hot row by the embedding matrix simply selects one row.

\"P(w_o\\mid

\"v_{w_i}\" is the input word embedding, \"u_w\" an output-context embedding, and the dot product is a compatibility score. Softmax turns all scores into probabilities that sum to one. Training increases the probability of observed context words. Negative sampling or hierarchical softmax can reduce computation in larger vocabularies; use the algorithm specified by the question.

Question. For the sentence “the cat sat on mat”, generate CBOW and skip-gram training data for centre word sat with a window of one token on each side.

Given

Tokens indexed 1 to 5: the, cat, sat, on, mat. Centre = sat.

To find

The CBOW example and the skip-gram pairs.

Formula and why it applies

CBOW predicts the centre from its neighbours; skip-gram predicts each neighbour from the centre.

Step 1. Identify neighbours

Left neighbour = cat; right neighbour = on. “the” and “mat” are outside this window.

Step 2. CBOW example

Input [cat,on] → target sat.

Step 3. Skip-gram examples

(sat,cat) and (sat,on), with first item as input and second as predicted context.

Answer. CBOW: [cat,on]→sat. Skip-gram: sat→cat and sat→on.

Question. The embeddings of context words cat and on are [1,2] and [3,0]. Compute the CBOW hidden vector using mean aggregation. Output vectors for [sat,ran] are [1,0] and [0,1]. Compute softmax probabilities.

Given

Context embeddings [1,2], [3,0]; mean aggregation; two candidate output vectors.

To find

The hidden vector, logits and probabilities.

Formula and why it applies

Mean context vector \"h=(v_1+v_2)/2\". Logit \"z_w=u_w^Th\". Softmax \"P(w)=e^{z_w}/\\sum_j.

Step 1. Add coordinates

\"[1+3,\\,2+0]=[4,2]\".

Step 2. Divide by number of words

\"h=[4/2,2/2]=[2,1]\".

Step 3. Compute logits

\"z_{sat}=1(2)+0(1)=2\". \"z_{ran}=0(2)+1(1)=1\".

Step 4. Exponentials

\"e^2\\approx7.389056\", \"e^1\\approx2.718282\"; sum \"\\approx10.107338\".

Step 5. Divide

\"P(sat)\\approx0.731059\", \"P(ran)\\approx0.268941\".

Answer. h=[2,1]; sat is more probable (0.731059).

Question. A skip-gram centre embedding is v=[1,−1]. Output embeddings for [eat,sleep,drive] are [1,1], [0,1], [1,0]. Calculate the predicted context probabilities.

Given

One centre vector and three output vectors; full softmax over these three words.

To find

Three logits and three probabilities.

Formula and why it applies

\"z_j=u_j^Tv\", then \"P(j)=e^{z_j}/\\sum_k.

Step 1. Eat dot product

\"1(1)+1(-1)=0\".

Step 2. Sleep and drive dot products

\"z_{sleep}=0(1)+1(-1)=-1\". \"z_{drive}=1(1)+0(-1)=1\".

Step 3. Exponentiate and sum

\"e^0=1\", \"e^{-1}\\approx0.367879\", \"e^1\\approx2.718282\". Sum \"\\approx4.086161\".

Step 4. Normalize

\"P(eat)\\approx0.244728\"; \"P(sleep)\\approx0.090031\"; \"P(drive)\\approx0.665241\".

Answer. drive has the highest probability in this toy learned model.

Limitations: a standard static Word2Vec vector merges a word's senses; it has limited representation of subword morphology and out-of-vocabulary words; its local prediction objective does not directly fit a full global co-occurrence table. Similarity is learned from data and is not a guarantee of factual or causal understanding.

CBOW = neighbours se missing centre. Skip-gram = centre se neighbours.

","m2":false,"units":"5","marks":10,"sourcePages":[2],"historical":true},{"id":"q4b","title":"Q4b · POS, NER, chunking and TTR","html":"
Question. Define POS tagging, entity recognition, chunking and type-token ratio with examples. [10 marks]

Answer

Part-of-speech (POS) tagging assigns a grammatical category to every word token in context, such as noun, verb, adjective or determiner. It is a sequence-labeling task: the input is a sequence of words, and the output is a tag for each word. A tag describes the word's role in this sentence, not its meaning in every possible sentence.

TagRoleExample in context
NN / NNSSingular / plural common nouna book / two books
NNPProper nounMary, Mumbai
VB / VBD / VBZBase / past / third-person singular verbread / read / reads
JJAdjectivethe red book
RBAdverbruns quickly
DTDeterminerthe, a, this
INPreposition or subordinating conjunctionon the table
PRPPersonal pronounshe, they
MDModal auxiliarycan, will, should

Open classes readily accept new content words: nouns, verbs, adjectives and adverbs. Examples include chatbot and livestream. Closed classes contain comparatively stable grammatical/function words: determiners, pronouns, prepositions and conjunctions. These are tendencies, not a claim that language never changes.

Input: Book that flight
→
Context: an imperative request
→
Output: Book/VB that/DT flight/NN

Compare “hand me that book”: book is a noun here. Similarly, “I can swim” has can/MD, while “the can is empty” has can/NN. The spelling stays the same; neighboring words and sentence structure change the correct tag. This is POS ambiguity.

SentenceTagged interpretationReason
They watch birds.They/PRP watch/VBP birds/NNSwatch is the action.
The watch stopped.The/DT watch/NN stopped/VBDwatch is the object that stopped.
She has a light bag.She/PRP has/VBZ a/DT light/JJ bag/NNlight describes bag.
Light the lamp.Light/VB the/DT lamp/NNLight is an imperative verb.

POS tags support parsing, named entity recognition, question answering, sentiment analysis and word-sense disambiguation. They do not by themselves identify every entity or fully resolve meaning.

Yaad rakho: Word ka naam same ho sakta hai; sentence mein uska kaam tag decide karta hai.

Named Entity Recognition (NER) identifies text spans referring to entities and assigns their types, such as PERSON, ORGANIZATION and LOCATION. Many practical schemas also include DATE, TIME, MONEY or domain-specific types. The exact allowed labels depend on the application's annotation scheme.

NER has two parts: boundary detection finds where the entity starts and ends; classification assigns the appropriate type. A multiword name should normally be one entity span: “New York” is a location span, not two unrelated location decisions.

Text: Mary joined Google in New York on Monday.
→
Mary → PERSON
→
Google → ORGANIZATION
→
New York → LOCATION
→
Monday → DATE

Not every noun is a named entity. In “a developer works at Google”, developer is an ordinary job description while Google is a named organization. A specialized HR system may define a JOB_TITLE category, but it must state that schema explicitly. NER identifies mentions; entity linking is the separate task of connecting a mention to a unique database identity.

TaskQuestion it answersExample
POSWhat grammatical role does this token have?Google/NNP
ChunkingWhich adjacent words form a phrase?[NP a skilled developer]
NERWhich span names an entity, and of what type?[Google] ORGANIZATION
Entity linkingWhich real entity does this mention denote?Connect Paris to the intended city or person record

Applications. NER structures search results and documents, extracts organizations/people from news, routes support tickets by product or organization, supports recommendation, and extracts specialized entities from healthcare or legal text. Context is essential: “Apple released a laptop” names an organization, while “she ate an apple” describes fruit.

Yaad rakho: NER mein do kaam: naam kahan se kahan tak, aur kis category ka.

Chunking, also called shallow parsing, groups adjacent tagged words into meaningful non-overlapping units, commonly noun phrases. It usually identifies local phrase boundaries without producing the complete hierarchical structure of a sentence. Chinking removes specified tokens from an existing candidate chunk, splitting or trimming it.

The/DT small/JJ cat/NN sat/VBD on/IN the/DT mat/NN
→
NP: [The small cat]
→
Outside: sat on
→
NP: [the mat]

A POS tagger labels individual words. A chunker groups words into phrases. A full parser shows how phrases relate recursively to form the entire sentence. These are related tasks with different outputs.

OperationInputOutput
POS taggingThe small catThe/DT small/JJ cat/NN
NP chunkingTagged tokens[NP The small cat]
ChinkingA broad candidate chunkRemove verbs/prepositions, keep noun-phrase segments
Full parsingSentence + grammarA nested tree with sentence, noun phrase and verb phrase nodes

Rule-based chunking uses tag patterns. Statistical or learned chunkers infer boundaries from labeled examples. A useful NP rule is an optional determiner, followed by zero or more adjectives, followed by a noun. In NLTK-style notation: NP: {<DT>?<JJ>*<NN>}. Here ? means zero or one occurrence; * means zero or more; + means one or more when used. The braces mean “form a chunk around the matched sequence”.

Question. Apply the NP rule optional DT, zero or more JJ, then one NN to “The/DT quick/JJ brown/JJ fox/NN jumps/VBZ over/IN the/DT lazy/JJ dog/NN”.

Given

Tagged input is supplied. Rule: DT? JJ* NN.

To find

All noun-phrase chunks.

Formula and why it applies

NP → DT? JJ* NN

The rule groups a noun with its preceding determiner and adjectives; a verb or preposition does not match this NP pattern.

Solution

Step 1. Start at The

The matches optional DT; quick and brown both match JJ*; fox matches NN.

Step 2. Form the first NP

[NP The quick brown fox]

Step 3. Skip nonmatching tokens

jumps/VBZ and over/IN do not satisfy the NP pattern.

Step 4. Match the second NP

the is DT, lazy is JJ, dog is NN: [NP the lazy dog].

Final answer. [NP The quick brown fox] jumps over [NP the lazy dog].

Question. Using NP → DT? JJ* NN, identify the chunks in “bright/JJ stars/NN shine/VBP and/CC a/DT moon/NN glows/VBZ”. Use the supplied tags exactly.

Given

Input tags label stars as NN for this exercise. Rule requires optional DT, zero or more JJ, then NN.

To find

The matched NP spans.

Formula and why it applies

NP → DT? JJ* NN

The optional determiner may be absent and the adjective repetition may be empty.

Solution

Step 1. Match bright stars

No determiner is present, which DT? permits. bright matches JJ; stars matches NN.

Step 2. Inspect shine and and

VBP and CC fall outside the NP rule.

Step 3. Match a moon

a matches DT; no adjective is present; moon matches NN.

Final answer. Chunks are [NP bright stars] and [NP a moon]. With ordinary Penn tagging, stars is usually NNS; broaden the noun pattern to include NNS when that tag is used.

Question. Begin with one candidate chunk containing “The/DT cat/NN sat/VBD on/IN the/DT mat/NN”. Chink out verbs and prepositions. Show the remaining chunks.

Given

Initial candidate: [The cat sat on the mat]. Remove tags VBD and IN.

To find

The chunk segments after exclusion.

Formula and why it applies

Chinking removes matched tokens from the existing chunk.

Removing the middle verb/preposition region divides the original candidate into two remaining contiguous spans.

Solution

Step 1. Locate the tokens to remove

sat is VBD and on is IN.

Step 2. Remove those tokens from chunk membership

The remaining token groups are “The cat” and “the mat”. sat and on still exist in the sentence; they are simply outside these chunks.

Step 3. Write the final chunked sentence

[The cat] sat on [the mat].

Final answer. Two chunks remain: The cat; the mat.

Yaad rakho: Chunking = andar group karo; chinking = unwanted tokens ko group se bahar karo.

Type-token ratio = number of distinct tokens / total tokens. In “cats chase cats”, types={cats,chase}=2 and tokens=3, so TTR=2/3. This is an authored illustration of the definition, not extra paper data.

","m2":false,"units":"2,4","marks":10,"sourcePages":[2],"historical":true},{"id":"q5a","title":"Q5a · Lexical and vector semantics","html":"
Question. Explain lexical semantics and vector semantics with examples. [10 marks]

Answer

A lexical relation connects words or senses by a meaning relationship. State the relation and its direction before giving the example.

RelationExam definitionExamples and direction
HomonymyThe same form has unrelated meanings.bank: financial institution / river edge; bat: animal / sports implement
PolysemyOne lexical item has multiple related senses.head: body part / head of a department; paper: material / article
SynonymyDifferent words have identical or closely similar meanings in a specified sense.sofa / couch; automobile / car
AntonymyWords have opposite meanings.hot / cold; alive / dead; buy / sell
HyponymyA more specific concept is a type of a more general one.rose is a flower; car is a vehicle
HypernymyA broader concept includes a more specific one.flower is a hypernym of rose
MeronymyA concept is a part of another concept.wheel is part of car; finger is part of hand
HolonymyA concept is the whole containing a part.car is a holonym of wheel

How to distinguish the commonly confused pairs

For homonymy versus polysemy, ask whether the senses share an identifiable meaning. A river bank and a financial bank are unrelated. A financial bank and a blood bank share a repository/storage idea, illustrating polysemy in the supplied notes.

For hyponymy versus meronymy, use the sentence test: “X is a kind of Y” indicates hyponymy; “X is part of Y” indicates meronymy. A wheel is not a type of car.

ExpressionCorrect relationReason
sparrow → birdHyponymyA sparrow is a kind of bird
bird → sparrowHypernymyBird is the broader category
keyboard → computerMeronymyKeyboard is a component
computer → keyboardHolonymyComputer is the containing whole
mouth of a person / mouth of a riverPolysemyBoth senses share the idea of an opening; the river use is a related extension

Synonyms are not always interchangeable in every context or register. “Big mistake” and “large mistake” can sound different even though big and large overlap in meaning. Antonyms can be gradable (hot/cold), complementary (alive/dead) or relational converses (buy/sell).

HYPO = specific type; HYPER = bigger category. MERO = part; HOLO = whole.

The distributional hypothesis states that words occurring in similar contexts tend to have similar meanings or semantic behaviour. “Cat” and “dog” often occur near pet, feed and animal. Their context patterns can therefore be represented and compared numerically.

A vector is an ordered list of numbers. Each position must have a fixed meaning. For context order [pet, food, road], cat=[4,3,0] means four co-occurrences with pet, three with food, none with road. A matrix stacks vectors as rows in a table.

A word–context co-occurrence matrix records how often a target word occurs with each context word under a specified context definition. A window may be a sentence, document, or a fixed number of neighbouring tokens. State its width, direction and whether targets count themselves.

Choose target words
→
Choose context/window
→
Count co-occurrences
→
Store each row as a vector
→
Compare vectors

Question. Build the symmetric co-occurrence matrix for “cats eat fish” using a window of one token on either side, no self-counts and vocabulary [cats,eat,fish].

Given

One sentence, one occurrence of each token. Left and right adjacent tokens are contexts.

To find

A 3×3 target-by-context count matrix.

Formula and why it applies

\"X_{ij}\" counts the number of times context word \"j\" occurs within the chosen window of target \"i\".

Step 1. Target cats

Only eat is adjacent. Row cats = [0,1,0].

Step 2. Target eat

cats is left and fish is right. Row eat = [1,0,1].

Step 3. Target fish

Only eat is adjacent. Row fish = [0,1,0].

Step 4. Assemble the matrix

Target / contextcatseatfish
cats010
eat101
fish010

Answer. Rows are [0,1,0], [1,0,1], [0,1,0]. The symmetry follows from the symmetric window.

Question. Build a target–context matrix for sentences “cats eat fish” and “dogs eat fish”. Target words are cats and dogs; context columns [eat,fish]; use the entire sentence excluding the target.

Given

Each sentence contains one target. Count context token occurrences.

To find

Two context vectors.

Formula and why it applies

Count each context word in every sentence containing the target.

Step 1. Count cats contexts

In its sentence: eat occurs once, fish once → [1,1].

Step 2. Count dogs contexts

In its sentence: eat occurs once, fish once → [1,1].

Step 3. Compare

Both targets have identical observed context patterns in this very small corpus.

Answer. cats=[1,1], dogs=[1,1]. This corpus supplies no context evidence to distinguish them.

Question. For “a b a”, use a right-only window of one token. Build the matrix with rows/columns [a,b].

Given

Count directed target→next-word events, not both directions.

To find

The co-occurrence matrix.

Formula and why it applies

\"X_{ij}=C(\\text{target.

Step 1. First target a

Next token is b → add 1 to X(a,b).

Step 2. Target b

Next token is a → add 1 to X(b,a).

Step 3. Final a

No following token → no addition.

Step 4. Write matrix

Target / nextab
a01
b10

Answer. a=[0,1], b=[1,0]. A different context window would produce a different matrix.

Word ka meaning: uske aas-paas kaunse words kitni baar aaye, uski number-list.

","m2":false,"units":"5","marks":10,"sourcePages":[3],"historical":true},{"id":"q5b","title":"Q5b · Classification label structures","html":"
Question. Compare binary, multiclass and multilabel supervised text classification with examples. [10 marks]

Answer

TypeAllowed outputExampleTypical output rule
BinaryOne of two labelspositive OR negative reviewOne sigmoid or two-class softmax
MulticlassExactly one of several labelssport OR business OR scienceSoftmax over mutually exclusive classes
MultilabelZero, one or several labelsArticle tagged technology AND financeIndependent sigmoid per label

Multiclass and multilabel differ in whether labels may coexist, not merely in the number of labels. A loss and evaluation measure must match the task.

","m2":false,"units":"7","marks":10,"sourcePages":[3],"historical":true},{"id":"q6a","title":"Q6a · TF–IDF in document representation","html":"
Question. Explain TF–IDF in document representation and information retrieval with an example. [10 marks]

Answer

TF–IDF weights a term highly when it is frequent in a particular document but uncommon across the corpus. The supplied Unit 2 slides use normalized term frequency and base-10 inverse document frequency:

\"\\operatorname{TF}(t,d)=\\frac{C(t,d)}{N_d}\"
\"\\operatorname{IDF}(t)=\\log_{10}\\frac{D}{\\operatorname{df}(t)}\"
\"\\operatorname{TFIDF}(t,d)=\\operatorname{TF}(t,d)\\operatorname{IDF}(t).\"

\"N_d\" is the document's token count after the specified preprocessing; \"D\" is number of documents; \"\\operatorname{df}(t)\" is the number of documents containing the term at least once. A word repeated ten times in one document contributes one to document frequency.

Why the log? It compresses the ratio's scale. If a term occurs in all documents, \"D/df=1\", so IDF=0 under this unsmoothed convention. Other libraries use smoothed IDF, different TF or normalization. Use the question's formula; do not silently substitute a library default.

Question. A corpus contains 10 documents. The word model appears in 2 documents and occurs 3 times in a 12-token document. Compute TF, IDF and TF–IDF using the slide convention.

Given

D=10, df(model)=2, C(model,d)=3, Nd=12; log base 10.

To find

The TF–IDF weight.

Formula and why it applies

\"TF=C/N_d\", \"IDF=\\log_{10}(D/df)\", then multiply.

Step 1. TF

\"3/12=0.25\".

Step 2. Document ratio

\"10/2=5\".

Step 3. IDF

\"\\log_{10}5\\approx0.698970\".

Step 4. Weight

\"0.25\\times0.698970\\approx0.174743\".

Answer. TF=0.25, IDF≈0.698970, TF–IDF≈0.174743.

Question. A term occurs 4 times in a 20-token document and is present in all 8 corpus documents. Calculate its TF–IDF.

Given

C=4, Nd=20, D=8, df=8.

To find

Its weight under unsmoothed base-10 IDF.

Formula and why it applies

\"TF=C/N_d\", \"IDF=\\log_{10}(D/df)\".

Step 1. TF

\"4/20=0.2\".

Step 2. IDF

\"\\log_{10}(8/8)=\\log_{10}1=0\".

Step 3. Multiply

\"0.2\\times0=0\".

Answer. Weight 0: this term does not distinguish documents under this convention.

Question. Find TF–IDF vectors for D1=“cat cat dog”, D2=“dog fish”, D3=“cat fish fish”, using [cat,dog,fish], normalized TF and log10 IDF.

Given

No further preprocessing. Each term occurs in exactly two of the three documents.

To find

All three vectors.

Formula and why it applies

First count df by documents, then compute each document's TF, then multiply by its term's IDF.

Step 1. Common IDF

\"D=3\", \"df(cat)=df(dog)=df(fish)=2\". Each IDF \"=\\log_{10}(3/2)\\approx0.176091\".

Step 2. D1 term frequencies

Length 3 → \"[2/3,1/3,0]\".

Step 3. D1 weights

\"[(2/3)0.176091,(1/3)0.176091,0]\\approx[0.117394,0.058697,0]\".

Step 4. D2 weights

Length 2; TF \"[0,1/2,1/2]\". Vector \"\\approx[0,0.088046,0.088046]\".

Step 5. D3 weights

Length 3; TF \"[1/3,0,2/3]\". Vector \"\\approx[0.058697,0,0.117394]\".

Answer. D1≈[0.117394,0.058697,0]; D2≈[0,0.088046,0.088046]; D3≈[0.058697,0,0.117394].

TF: iss document mein kitna. IDF: poore corpus mein kitna rare. Dono multiply.

Information retrieval (IR) locates and ranks items relevant to an information need, usually expressed as a query. The collection may contain documents, product descriptions or scientific papers. Retrieval estimates relevance; it does not itself establish that every retrieved claim is true.

Documents → cleaning
→
Build index
→
Query → same representation
→
Match & rank
→
Return relevant results

The inverted index

An inverted index maps a term to the documents containing it, often with positions or counts. For D1 “cats drink milk” and D2 “dogs drink water”, the posting list for drink is {D1,D2}, while milk maps only to D1. This avoids rescanning every document for every query.

TermPosting list
catsD1
drinkD1, D2
milkD1
dogsD2
waterD2

Matching and ranking

Boolean retrieval combines term membership using AND, OR and NOT. Vector-space retrieval represents documents and queries as weighted vectors, for example with TF-IDF, and compares them with cosine similarity. Learned retrieval can use dense embeddings for meaning-based matching. The representation used at query time must be compatible with the document index.

Search engines and recommendation

A search engine receives an explicit query and ranks matching indexed pages. A recommender may derive an implicit query from a user’s liked items or profile and retrieve similar candidates. Both require candidate retrieval and ranking, but the source of the information need differs.

Evaluate the results

For retrieval, TP is the number of retrieved relevant documents, FP retrieved irrelevant documents, and FN relevant documents not retrieved. Precision and recall use the same formulas as classification. Ranking measures also consider the positions of relevant items; a relevant result at rank 1 is more useful than one far below.

Retrieval evaluation

A search system retrieves 8 documents; 6 are relevant. There are 10 relevant documents in the collection. Find precision, recall and F1.

Given

Retrieved=8; relevant retrieved=6; all relevant=10.

To find

P, R and F1.

Formula and why it applies

\"P=\\frac{\\text{relevant
\"F_1=\\frac{2PR}{P+R}\"

The denominator is 8 for precision and 10 for recall because they ask different questions.

Solution

Step 1. Identify the confusion counts

TP=6; FP=8−6=2; FN=10−6=4.

Step 2. Precision

\"P=\\frac68=0.75\"

Step 3. Recall

\"R=\\frac6{10}=0.6\"

Step 4. F1

\"F_1=\\frac{2(0.75)(0.6)}{0.75+0.6}=\\frac{0.9}{1.35}=\\frac23\\approx0.666667\"
Precision=0.75, recall=0.60, F1≈0.666667.

Yaad rakho: Precision: jo laaye, unmein kitne sahi? Recall: jo sahi the, unmein kitne laaye?

Source / convention note. The prompt requests an example but supplies no corpus. Numbers in the explanation are authored illustrations, not original numerical givens.

","m2":false,"units":"2,7","marks":10,"sourcePages":[3],"historical":true},{"id":"q6b","title":"Q6b · HMM tables and path","html":"
Question. For the four tagged sentences below construct transition and emission tables (3 marks each), then calculate Geeta/N will/M spot/V Rita/N using initial P(N)=0.5, P(M)=0.25, P(V)=0.25. [10 marks]

Answer

Given

SentenceGiven tags
Rita can teach WillN M V N
Spot will see WillN M V N
Will Rita spot GeetaN N V N
Geeta will pat SpotN M V N

Required path: Geeta will spot Rita / N M V N. Given initial probabilities: {'N': Fraction(1, 2), 'M': Fraction(1, 4), 'V': Fraction(1, 4)}

To find

Emission and transition tables and the specified tag-path joint probability

Formula: which one and why

\"P(w\\mid
\"P(t_j\\mid

The second formula is the declared no-END convention. Reset at sentence boundaries; never create a transition from the last word of one training sentence to the first of the next.

Step 1. Count emissions

TagTotal occurrencesRitacanteachWillSpotwillseespotGeetapat
N92003200020
M30100020000
V40010001101

Step 2. Divide each emission by its tag count

TagRitacanteachWillSpotwillseespotGeetapat
N2/9001/32/90002/90
M01/30002/30000
V001/40001/41/401/4

Step 3. Count transitions inside each sentence

FromOutgoing transitionsSentence-final countNMV
N54131
M30003
V40400

Step 4. Convert to a transition table

From / toNMV
N1/53/51/5
M001
V100

Here sentence-final tags are excluded from the no-END outgoing denominator. If END is explicitly modeled, divide all tag→tag and tag→END counts by the total tag count instead; the row then includes END. Do not mix denominators.

Step 5. Multiply the given path, in order

FactorProbabilityRunning product
Initial N1/21 × 1/2 = 1/2
Emit Geeta2/91/2 × 2/9 = 1/9
Transition N→M3/51/9 × 3/5 = 1/15
Emit will2/31/15 × 2/3 = 2/45
Transition M→V12/45 × 1 = 2/45
Emit spot1/42/45 × 1/4 = 1/90
Transition V→N11/90 × 1 = 1/90
Emit Rita2/91/90 × 2/9 = 1/405

Final answer

The two displayed probability tables are the estimated HMM. Zero means unobserved without smoothing, not logically impossible.

No-END path joint probability = 1/405 = 0.00246913580247. If a normalized END-state model is explicitly adopted instead, this complete terminated sequence has probability 4/6561 = 0.000609663161103. These are different model conventions, not contradictory answers.

","m2":false,"units":"4","marks":10,"sourcePages":[3],"historical":true},{"id":"q7a","title":"Q7a · Healthcare and finance applications","html":"
Question. Explain NLP use cases in healthcare and finance. [10 marks]

Answer

Domain / taskInput → method → useful outputExample and limit
Healthcare: clinical extractionNotes → entity/negation/relation extraction → diagnoses, medicines and dosesDistinguish no fever from fever; clinician review is needed
Healthcare: coding and searchClinical text → classify/retrieve → candidate billing codes or evidenceCoding suggestions are not autonomous diagnoses
Healthcare: patient messagingPatient question → intent + retrieval → approved explanationEscalate uncertainty and emergencies; protect identifiers
Finance: sentiment/event analysisNews/filings → entities/events/sentiment → issuer-specific signalsDistinguish rumor from confirmed earnings; not guaranteed returns
Finance: document extractionReports/contracts → fields/relations → searchable structured recordsKeep source spans and verify amounts/dates
Finance: customer support/compliance triageMessages → intent/classification → routed cases or flagged textHuman investigation is required; a language flag is not proof of wrongdoing
Acquire permitted text → Clean and preserve context → Extract / classify / retrieve → Evaluate on domain data → Human-reviewed use
","m2":false,"units":"7","marks":10,"sourcePages":[4],"historical":true},{"id":"q7b","title":"Q7b · Naive Bayes sentiment","html":"
Question. Train Naive Bayes on: positive “amaze wonderful great”; “happy happy great”; “amaze great happy disgust”; “great happy wonderful”. Negative “disgust sad terrible great”; “sad terrible sad happy”. Predict sentence 7: “great happy disgust great”. [10 marks]

Answer

Given

Training textClass
amaze wonderful greatpositive
happy happy greatpositive
amaze great happy disgustpositive
great happy wonderfulpositive
disgust sad terrible greatnegative
sad terrible sad happynegative

Test: great happy disgust great

To find

Classify the test using multinomial Naive Bayes.

Formula: which one and why

Assume add-one smoothing unless the question supplies probabilities. Keep repeated words as repeated factors. Use training vocabulary only; ignore out-of-vocabulary words unless an unknown-token model is defined.

\"\\mathrm{score}(c)=P(c)\\prod_{w\\in

Step 1. Vocabulary and class priors

V = {amaze, disgust, great, happy, sad, terrible, wonderful}; |V|=7.

ClassDocumentsPrior
negative21/3
positive42/3

Step 2. Count every training word within each class

ClassTotal tokensamazedisgustgreathappysadterriblewonderful
negative80111320
positive132144002

Step 3. Count the test words

WordCount
great2
happy1
disgust1

OOV words: none

Step 4. Calculate likelihoods and multiply with the prior

ClassPriorIndividual factorsExact scoreDecimal score
negative1/3great: (1+1)/(8+1×7) = 2/15, used 2 time(s); happy: (1+1)/(8+1×7) = 2/15, used 1 time(s); disgust: (1+1)/(8+1×7) = 2/15, used 1 time(s)16/1518750.000105349794239
positive2/3great: (4+1)/(13+1×7) = 1/4, used 2 time(s); happy: (4+1)/(13+1×7) = 1/4, used 1 time(s); disgust: (1+1)/(13+1×7) = 1/10, used 1 time(s)1/9600.00104166666667

Step 5. Compare and optionally normalize

Normalized scores: negative = 0.091846802, positive = 0.908153198

Division by the same total does not change the winning class.

Final answer

Predicted class: positive.

Yaad rakho: Count → prior → smoothed word chances → repeated factors → compare.

","m2":false,"units":"7","marks":10,"sourcePages":[4],"historical":true}]},{"id":"legacy-f23","title":"2023-12-01 · final · recovered AI paper","unit":"PYQ","scope":"Previous examination · complete worked answers","sections":[{"id":"q1a","title":"Q1a · WSD and pronoun resolution","html":"
Question. Define Word Sense Disambiguation and Pronoun Resolution. Give an example of each. [5 marks]

Answer

Word sense disambiguation (WSD) is the task of selecting the intended sense of an ambiguous word in its context from a set of possible senses. The output is a sense label, not a different replacement word.

Input: “He grilled the bass.”
→
Candidate senses: fish / low sound
→
Context: grilled
→
Output: fish sense

The target word is the ambiguous word. The context is the surrounding evidence. The sense inventory lists candidate meanings, often using WordNet. A feature is a usable clue: nearby word, position, part-of-speech tag or a phrase.

ApproachWhat it needsHow it decidesLimitation
Knowledge basedDictionary, thesaurus or semantic graphDefinition overlap, category evidence or graph scoresShort definitions and missing knowledge
SupervisedExamples already labelled with sensesLearns context → sense mappingLabelling cost, limited rare-sense examples
Semi-supervisedSmall labelled set plus unlabelled dataBootstraps from reliable seed examplesEarly errors can spread
Unsupervised sense inductionUnlabelled contextual occurrencesClusters similar contextsClusters are not automatically named dictionary senses
Neural/contextualLearned word/context representationsPredicts or compares a context representationNeeds suitable data/model and evaluation

Bag-of-words features record nearby words while ignoring exact positions. Collocational features preserve a positional phrase such as “river bank”, “bank account”, “chocolate bar” or “bar exam”.

Why WSD matters

Machine translation needs the correct target-language meaning; information retrieval should distinguish river-bank travel from banking services; text-to-speech may need different pronunciations for “bass”; information extraction should avoid connecting entities through the wrong relation.

WSD: same spelling ke options mein context dekh ke sahi meaning select.

Pronoun resolution identifies the antecedent referred to by a pronoun. In “Rita arrived. She smiled”, she refers to Rita. WSD chooses a word meaning (bank as finance or river edge); pronoun resolution chooses a discourse entity.

","m2":false,"units":"5,6","marks":5,"sourcePages":[1],"historical":true},{"id":"q1b","title":"Q1b · WordNet and its hierarchy","html":"
Question. Define WordNet. Describe the WordNet hierarchy. [5 marks]

Answer

WordNet is a lexical database that groups word senses into synonym sets called synsets and links them through semantic and lexical relations. It is a network of meanings, not merely a list of similar spellings.

A synset represents one concept. Its lemmas are the words that can express that concept. A gloss is a short definition, often accompanied by an example. An ambiguous word can occur in several synsets.

vehicle (general)
→
car / automobile (same concept)
→
sports car (specific)

This is a simplified teaching hierarchy, not a claim about every intermediate WordNet node. Going upward reaches hypernyms; going downward reaches hyponyms. Multiple parents can exist, so the structure need not be a single simple tree.

OperationMeaningIllustration
Get senses of a wordFind candidate synsetsbank → financial sense, river-edge sense, …
Get lemmasFind synonyms within one synsetcar, automobile, motorcar
Get definition/examplesRead the gloss and usesChoose the intended concept
Get hypernymsMove to broader categoriescar → vehicle
Get hyponymsMove to narrower categoriesvehicle → particular vehicle categories
Get meronyms/holonymsFind part/whole linkscar ↔ wheel

WordNet versus a thesaurus

A thesaurus mainly groups related expressions for vocabulary lookup. WordNet explicitly distinguishes senses and labels relations such as hypernymy, hyponymy and meronymy. This supports WSD, query expansion, lexical similarity and semantic search.

Three uses explained

  1. Search expansion: a query for “automobile” can retrieve relevant documents using “car”. Expansion should use the right sense.
  2. WSD: definitions of the senses of “bank” provide words that can be compared with the sentence context.
  3. Similarity: concepts near one another in an “is-a” hierarchy tend to be semantically related. Path-based measures depend on which senses were selected.

WordNet mein spelling ka group nahi—meaning ka synset, aur meanings ke beech labelled links.

","m2":false,"units":"5","marks":5,"sourcePages":[1],"historical":true},{"id":"q1c","title":"Q1c · Sentiment feature extraction","html":"
Question. Extract x₁ = log₁₀(word count), x₂ = count of positive lexicon words and x₃ = count of negative lexicon words from: “I was amazed at how the product quickly it arrived. The customer service was exceptional. The support team responded promptly and professionally. I was not sad with my decision. Overall, my experience with them was outstanding.” Positive lexicon={amazed,promptly,outstanding,great}; negative={horrible,sad,disgusting}. [5 marks]

Answer

Given

I was amazed at how the product quickly it arrived. The customer service was exceptional. The support team responded promptly and professionally. I was not sad with my decision. Overall, my experience with them was outstanding.

To find

The three numerical features.

Formula: which one and why

Use word tokens after stripping punctuation, keep stopwords, and match only the supplied lexicons. Raw lexicon counting does not automatically reverse a word under negation.

Step 1. List and count every word

PositionWord
1i
2was
3amazed
4at
5how
6the
7product
8quickly
9it
10arrived
11the
12customer
13service
14was
15exceptional
16the
17support
18team
19responded
20promptly
21and
22professionally
23i
24was
25not
26sad
27with
28my
29decision
30overall
31my
32experience
33with
34them
35was
36outstanding

Step 2. Apply the logarithm

\"x_1=\\log_{10}36=1.556302501\"

Step 3. Count positive matches

amazed, promptly, outstanding each occur once. great does not occur. x₂=3. exceptional is not in the supplied lexicon, so it is not counted.

Step 4. Count negative matches

sad occurs once; horrible and disgusting do not. x₃=1. The phrase “not sad” remains one raw negative-lexicon occurrence under the specified feature definition.

Final answer

(x₁,x₂,x₃)=(1.556302501,3,1).

","m2":false,"units":"2,7","marks":5,"sourcePages":[1,2],"historical":true},{"id":"q1d","title":"Q1d · Binary, multiclass and multilabel","html":"
Question. Compare binary, multiclass and multilabel supervised text classification with examples. [5 marks]

Answer

TypeAllowed outputExampleTypical output rule
BinaryOne of two labelspositive OR negative reviewOne sigmoid or two-class softmax
MulticlassExactly one of several labelssport OR business OR scienceSoftmax over mutually exclusive classes
MultilabelZero, one or several labelsArticle tagged technology AND financeIndependent sigmoid per label

Multiclass and multilabel differ in whether labels may coexist, not merely in the number of labels. A loss and evaluation measure must match the task.

","m2":false,"units":"7","marks":5,"sourcePages":[2],"historical":true},{"id":"q2a","title":"Q2a · Text corpus structure and applications","html":"
Question. Explain text-corpus structure and the applications of text corpora in NLP. [10 marks]

Answer

A corpus is a collection of language data assembled for study or computational use. Its plural is corpora. A corpus can contain books, news, reviews or conversations. It need not be labelled. Annotation adds linguistic information such as POS tags, entities, parse structures or sense labels.

StructureMeaningUse
PlainCollection without category labelsWord counts, language modelling
CategorizedDocuments grouped by genre/topicClassification
Overlapping categoriesOne document has several categoriesMulti-topic news
TemporalDocuments indexed by timeLanguage change
AnnotatedLinguistic labels attachedTagging, parsing, WSD
ParallelAligned texts in different languagesTranslation

A tagset is the allowed label inventory. A treebank is a corpus annotated with syntax trees. Metadata records source, date, genre, language, annotation method and limitations. A common benchmark supports comparison under shared data and evaluation conditions.

Raw example: “Riya works in Pune.” POS annotation: Riya/PROPN works/VERB in/ADP Pune/PROPN. NER annotation: Riya/PERSON, Pune/LOCATION. Each annotation answers a different task.

A good corpus is relevant to its domain, sufficiently representative, consistently annotated and appropriately split. More data alone do not repair duplicated examples, incorrect labels or a mismatch between training and real use.

Corpus=text collection. Annotation=text ke saath labels. Metadata=collection ki information.

ResourceContentsTypical use
Gutenberg selectionLiterary/book textsLong-form language and vocabulary
Web textInformal domain-specific textStudy usage beyond edited books
Chat corpusConversational postsDialogue, informal forms
BrownEnglish texts across genresGenre comparison
ReutersNews with topic labelsText classification; overlapping topics
InauguralSpeeches indexed by timeVocabulary change across years
WordlistWords/names/stopwordsLexical filtering
Pronouncing dictionaryWords and phonetic formsSpeech, stress, rhyme

Gutenberg versus web/chat: book language does not represent every spelling or conversational pattern in messages. Brown versus Reuters: Brown emphasizes genres; Reuters news can have overlapping topic labels. Inaugural addresses: dates provide a time dimension.

Corpus sizes depend on version, subset and tokenization. Explain their purpose and structure rather than assuming a historical count is a permanent property.

Gutenberg=books; web/chat=informal; Brown=genres; Reuters=news topics; Inaugural=time.

Word-sequence counts support n-gram language models and smoothing. POS-annotated sentences support supervised taggers. Sense-labelled contexts support Naive Bayes WSD. Word-context counts support distributional semantics and GloVe. Parallel aligned sentences support translation.

For a question asking the importance of corpora, give three points: empirical language evidence, examples for training/evaluation, and reproducible comparison. For an annotated-corpus question, define the added labels and give a concrete example. For resource comparisons, distinguish running text, lexical lists, pronunciation entries and semantic relations.

Same text se task ke hisaab se counts, tags, senses ya context patterns nikaalte hain.

","m2":false,"units":"3","marks":10,"sourcePages":[2],"historical":true},{"id":"q2b","title":"Q2b · Lesk versus Walker and bank disambiguation","html":"
Question. Differentiate Lesk and Walker WSD. Calculate sense scores and disambiguate “bank” in “Money in this bank fetches an interest of 8% per annum.” [10 marks]

Answer

The Walker approach uses thesaurus categories as semantic evidence. Associate each candidate sense with a category. Count or weight context words that support that category. Choose the largest total.

\"\\operatorname{score}(s)=\\sum_{w\\in

\"I=1\" if the supplied category mapping says the context word supports the sense, otherwise \"I=0\". The summation symbol means add all those contributions. Weighted questions may replace 1 with a supplied weight. Category membership must be supplied or justified; it is not guessed from word frequency.

Question. Disambiguate bank using Walker scoring. Context words are money, interest, annum, fetch. The finance category supports money, interest and annum; the location category supports none of these words.

Given

Two categories: Finance and Location. Each supported word contributes 1.

To find

The two totals and selected bank sense.

Formula and why it applies

Sum the category indicators for each sense.

Step 1. Make a contribution table

WordFinanceLocation
money10
interest10
annum10
fetch00

Step 2. Add Finance

\"1+1+1+0=3\".

Step 3. Add Location

\"0+0+0+0=0\".

Step 4. Select

3 > 0, so select the financial institution.

Answer. Finance, with score 3 versus 0.

Question. Use Walker scoring for “The fisherman caught bass beside the lake.” Context {fisherman,caught,lake}. The Fish category contains all three clues; the Music category contains none.

Given

Fish and Music senses, binary category membership given.

To find

The correct bass sense.

Formula and why it applies

Score = sum of context-category matches.

Step 1. Record Fish matches

fisherman → 1; caught → 1; lake → 1.

Step 2. Add Fish score

\"1+1+1=3\".

Step 3. Add Music score

\"0+0+0=0\".

Step 4. Choose

Fish has the larger score.

Answer. Bass refers to a fish.

Question. Score “The player used a bat in the match.” Context {player,match}. Sport supports both words with weights 2 and 1; Animal supports neither. Use the supplied weighted Walker scheme.

Given

Sport weights {player:2, match:1}; Animal weights {player:0, match:0}.

To find

Weighted sense scores.

Formula and why it applies

Use the sum of supplied weights, not an unweighted count, because the question explicitly gives weights.

Step 1. Sport contribution of player

Add 2.

Step 2. Sport contribution of match

Add 1. Sport total = 2 + 1 = 3.

Step 3. Animal total

0 + 0 = 0.

Step 4. Compare

3 > 0 selects the sporting implement.

Answer. Bat is the sports implement.

Walker: dictionary ke exact words nahi, context kis topic/category ko support karta hai woh count.

AspectLeskWalker/thesaurus method
EvidenceOverlap of context with dictionary glossesContext-word membership in semantic subject categories
ScoreNumber of shared normalized wordsNumber/weight of context words supporting each sense category
DependencyGloss definitions and preprocessingThesaurus categories and their assignments

Given

Context is the printed sentence. No thesaurus category list is supplied.

To find

State the sense and give a transparent illustrative score.

Formula: which one and why

Declare the categories first; count only declared evidence. A numerical Walker score is not uniquely determined without the thesaurus assignments.

Step 1. Choose explicit illustrative categories

Assume money and interest belong to FINANCE; river, water and shore belong to RIVER. Ignore all other tokens as unassigned.

Step 2. Count support

Candidate bank senseMatching cluesScore
Financial institutionmoney, interest2
River edgenone0

Step 3. Compare

2 > 0, so choose the financial-institution sense under this stated assignment.

Final answer

bank = financial institution. The score 2 versus 0 is conditional on the declared illustrative thesaurus, not supplied numerical data.

Source / convention note. The original gives a sentence but no thesaurus/category mapping, so a unique numerical score cannot be recovered.

","m2":false,"units":"5","marks":10,"sourcePages":[2],"historical":true},{"id":"q3a","title":"Q3a · Hobbs algorithm and limitations","html":"
Question. Explain Hobbs’ algorithm with an example and state its limitations. [10 marks]

Answer

Hobbs’ algorithm is a syntax-based procedure that searches constituent parse trees in a fixed order for a compatible noun-phrase antecedent of a pronoun. The tree determines which candidates are proposed first. Agreement and other relevant constraints determine whether a proposed candidate can be accepted.

It is not simply the rule “choose the closest noun”. A nearby noun may have the wrong number, may be syntactically ineligible, or may be visited after a structurally more prominent noun. The algorithm produces a preference under the supplied parse; ambiguous language can still require context.

The tree vocabulary you need

TermMeaningExample
NPNoun phrase: a phrase referring to an entitythe student; Anita; they
SA sentence/clause nodethe node containing a subject and predicate
VPVerb phrasetold Ravi that she should revise
XThe NP or S node currently being searchedAn algorithm label, not a word
Path pThe branch travelled upward from the pronoun/previous XUsed to distinguish left and right subtrees
N-bar (N′)The noun-centred part of an NP in a grammar using that labelThe nominal-child test depends on the given tree
Breadth-firstVisit shallower nodes before deeper onesLevel 1 before level 2
Left-to-rightVisit equally deep nodes in their written/tree orderEarlier left branch before right branch
Locate pronoun NP
→
Climb to NP/S
→
Propose candidates in tree order
→
Check compatibility
→
Accept or continue searching

The complete search procedure

Step 1. Start at the pronoun

Locate the NP immediately dominating the pronoun. “Dominating” means containing it as a descendant in the tree.

Step 2. Climb to the first NP or S

Move upward to the first NP or S above that starting NP. Call this node X. Keep the upward path p.

Step 3. Search the left branches initially

Below X, traverse branches to the left of p breadth-first and left-to-right. At this initial search, propose an NP only if another NP or S lies between that candidate and X. Test proposed candidates for the required agreement and constraints.

Step 4. If X is the highest sentence node, inspect earlier sentences

Search previous sentence trees from most recent to oldest. Within each, use breadth-first, left-to-right order to propose NPs. Accept the first eligible compatible candidate, or report that no candidate was found in the available context.

Step 5. Otherwise climb again

Move from X to the next NP or S above it. Record this new X and the path followed during the climb.

Step 6. Apply the higher-NP test

If the new X is an NP, propose X when the path did not pass through its nominal N-bar child. Use the given parse labels; do not invent an N-bar branch missing from the question's structure.

Step 7. Search its left branches

Traverse branches below the new X to the left of the path, breadth-first and left-to-right, proposing NPs. The special intervening-NP/S restriction belongs to the initial search in Step 3.

Step 8. If X is S, search the permitted right branches

Search branches to the right of the path breadth-first and left-to-right. Do not descend below an encountered NP or S. Propose eligible NP candidates there.

Step 9. Repeat or stop

Return to the highest-S check and continue the climb/search procedure until a compatible candidate is accepted or the available trees are exhausted. Record the candidate order rather than only writing an unexplained name.

Why the first left search has an extra condition

In a simple clause such as “John saw him”, the subject NP John is directly under the clause S. There is no extra NP or S between John and that S. The initial-search restriction therefore prevents casually selecting John as the ordinary local antecedent of him. Compare the reflexive “John saw himself”, which is a different grammatical case; do not treat every pronoun type as interchangeable.

Worked tree 1 · move to the previous sentence

Question. Apply Hobbs’ search to “The student arrived. She smiled.” Assume the student is compatible with she.

Given

Previous tree: [S [NP The student] [VP arrived]]. Current tree: [S [NP She] [VP smiled]].

To find

The selected antecedent and the search path.

Method and why it applies

Use the supplied constituent trees. The current sentence has no earlier candidate, so the highest-S rule activates previous-sentence search.

Solution

Step 1. Draw both sentence structures

Previous sentence             Current sentence\nS                             S\n├── NP: The student            ├── NP: She ← start\n└── VP: arrived                └── VP: smiled

Step 2. Locate and climb

Start at NP She. Its first higher NP/S is the current S. Set X to that S.

Step 3. Search left of the path

She is the leftmost subject. No NP lies to its left in the current sentence.

Step 4. Use the highest-S rule

This X is the highest S. Inspect the immediately preceding sentence tree.

Step 5. Propose its first NP

Breadth-first, left-to-right finds NP The student.

Step 6. Check compatibility and accept

The question stipulates that the student agrees with she. Accept this candidate.

Answer. She refers to the student under the supplied assumptions.

Worked tree 2 · reject an incompatible subject

Question. Resolve they in “Riya spoke to the students. They laughed.” Use a previous-sentence search and interpret they as plural.

Given

Previous tree has subject NP Riya, then VP spoke [PP to [NP the students]]. Current subject is NP They.

To find

The candidate order and accepted antecedent.

Method and why it applies

Breadth-first search favours a shallow subject, but agreement must still be applied; a first candidate is not automatically a valid answer.

Solution

Step 1. Draw the useful part of the previous tree

S\n├── NP: Riya                 ← candidate 1\n└── VP\n    ├── V: spoke\n    └── PP\n        ├── P: to\n        └── NP: the students ← candidate 2

Step 2. Search the current sentence

From NP They climb to its S; there is no earlier NP in “They laughed”. This is the highest S, so move to the previous tree.

Step 3. Visit Riya first

Riya is the shallower leftmost NP. Under the stated plural interpretation, this singular candidate fails number agreement.

Step 4. Visit the students

Continue breadth-first. The next relevant NP is the students.

Step 5. Apply number agreement

Students is plural and is an eligible candidate. It matches plural they.

Step 6. State the resolution

Accept the students. The method did not simply accept the first noun it saw.

Answer. They → the students. Candidate order: Riya rejected; the students accepted.

Worked tree 3 · climb out of an embedded clause

Question. Resolve she in “Anita told Ravi that she should revise.” Assume Anita is compatible with she and Ravi is not. Use the displayed parse.

Given

The pronoun is the subject of an embedded S inside the main VP.

To find

The preferred antecedent found by the climb and left-branch search.

Method and why it applies

The embedded clause has no left candidate. Climbing to the main S exposes its subject Anita as the first suitable higher-level candidate.

Solution

Step 1. Show the clause structure

S (main)\n├── NP: Anita                  ← accepted candidate\n└── VP\n    ├── V: told\n    ├── NP: Ravi\n    └── SBAR: that\n        └── S (embedded)       ← first X\n            ├── NP: she        ← start\n            └── VP: should revise

Step 2. Start inside the embedded clause

From NP she climb to the embedded S. There is no candidate to the left of she within this embedded S.

Step 3. Climb to the next NP/S

The embedded S is not the highest S. Move upward through SBAR and VP to the main S. These intermediate nodes are not NP/S search nodes.

Step 4. Search left of the upward path

At the main S, the subject NP Anita is a shallow left candidate. The object Ravi is deeper within the VP. Breadth-first order proposes Anita first.

Step 5. Check the stated compatibility

Anita is compatible with she by the question's assumption. Accept it; no need to choose a later incompatible candidate.

Step 6. Give the result and its limit

This is the algorithm's result under the displayed parse and agreement assumptions. It does not prove the intention of every speaker using an otherwise ambiguous sentence.

Answer. She → Anita under the stated tree and compatibility information.

Advantages, limitations and a complete exam answer

AdvantagesLimitations
Explicit candidate order makes the decision traceable.Wrong parse structure can produce the wrong search order.
Requires no labelled coreference-training corpus for its basic rules.World knowledge and discourse intention may override syntactic preference.
Uses structure rather than only surface distance.Some ambiguous, long-distance or unusual constructions need additional treatment.

For a longer answer, write the definition, inputs, tree notation, ordered algorithm, one labelled tree with a candidate table, final antecedent, and limitations. In a worked question show separately: candidate → why proposed → agreement result → accept/reject. Do not equate the selected candidate with a logically guaranteed interpretation when the sentence is ambiguous.

Yaad rakho: Tree order candidate deta hai; agreement usko allow/reject karta hai. Nearest noun automatically answer nahi.

","m2":false,"units":"6","marks":10,"sourcePages":[2],"historical":true},{"id":"q3b","title":"Q3b · Stemming and lemmatization","html":"
Question. Discuss the need for stemming, list different types of stemming, and differentiate stemming from lemmatization. [10 marks]

Answer

Stemming merges inflected/derived surface variants for matching and reduces vocabulary size. Types include rule/affix-removal stemmers (Porter, Snowball, Lancaster), dictionary/lookup stemmers, statistical stemmers and hybrid methods. More aggressive reduction increases both merging and overstemming risk.

Stemming applies string-based rules, often stripping suffixes, to reduce related forms to a stem. The result need not be a valid dictionary word. Lemmatization uses vocabulary and morphological analysis, often with POS information, to return a dictionary lemma.

CriterionStemmingLemmatization
BasisSurface form/suffix rulesLexicon and morphological/POS analysis
OutputMay be nonword, e.g. studiDictionary form, e.g. study
Context/POSUsually little or noneOften important
Irregular formsMay miss went→goCan map went→go with verb analysis
CostGenerally simplerUsually more linguistic processing

Porter applies a sequence of suffix-removal rules subject to conditions; Snowball provides related language-specific stemming algorithms; Lancaster is often more aggressive; regex stemming uses explicit patterns; Lovins is another established stemming approach. Exact output depends on the algorithm and implementation.

Three worked reductions

  1. studies, studying: an English stemmer may yield studi/study depending on the rule sequence; verb lemmatization aims at study. State the exact algorithm if exact outputs are asked.
  2. better: adjective-aware lemmatization can map it to good; naïve suffix removal cannot infer that irregular relation.
  3. saw: as a past-tense verb it can become see; as a cutting-tool noun it remains saw. POS and context matter.

Over-stemming merges forms that should remain distinct; under-stemming fails to merge related forms. Normalization can increase recall in search while losing useful distinctions, so it is a task-dependent choice.

Stem = rules se cut; lemma = dictionary form, meaning/POS ka dhyan.

","m2":false,"units":"2","marks":10,"sourcePages":[2],"historical":true},{"id":"q4a","title":"Q4a · Lexical and vector semantics","html":"
Question. Explain lexical semantics and vector semantics with suitable examples. [10 marks]

Answer

A lexical relation connects words or senses by a meaning relationship. State the relation and its direction before giving the example.

RelationExam definitionExamples and direction
HomonymyThe same form has unrelated meanings.bank: financial institution / river edge; bat: animal / sports implement
PolysemyOne lexical item has multiple related senses.head: body part / head of a department; paper: material / article
SynonymyDifferent words have identical or closely similar meanings in a specified sense.sofa / couch; automobile / car
AntonymyWords have opposite meanings.hot / cold; alive / dead; buy / sell
HyponymyA more specific concept is a type of a more general one.rose is a flower; car is a vehicle
HypernymyA broader concept includes a more specific one.flower is a hypernym of rose
MeronymyA concept is a part of another concept.wheel is part of car; finger is part of hand
HolonymyA concept is the whole containing a part.car is a holonym of wheel

How to distinguish the commonly confused pairs

For homonymy versus polysemy, ask whether the senses share an identifiable meaning. A river bank and a financial bank are unrelated. A financial bank and a blood bank share a repository/storage idea, illustrating polysemy in the supplied notes.

For hyponymy versus meronymy, use the sentence test: “X is a kind of Y” indicates hyponymy; “X is part of Y” indicates meronymy. A wheel is not a type of car.

ExpressionCorrect relationReason
sparrow → birdHyponymyA sparrow is a kind of bird
bird → sparrowHypernymyBird is the broader category
keyboard → computerMeronymyKeyboard is a component
computer → keyboardHolonymyComputer is the containing whole
mouth of a person / mouth of a riverPolysemyBoth senses share the idea of an opening; the river use is a related extension

Synonyms are not always interchangeable in every context or register. “Big mistake” and “large mistake” can sound different even though big and large overlap in meaning. Antonyms can be gradable (hot/cold), complementary (alive/dead) or relational converses (buy/sell).

HYPO = specific type; HYPER = bigger category. MERO = part; HOLO = whole.

The distributional hypothesis states that words occurring in similar contexts tend to have similar meanings or semantic behaviour. “Cat” and “dog” often occur near pet, feed and animal. Their context patterns can therefore be represented and compared numerically.

A vector is an ordered list of numbers. Each position must have a fixed meaning. For context order [pet, food, road], cat=[4,3,0] means four co-occurrences with pet, three with food, none with road. A matrix stacks vectors as rows in a table.

A word–context co-occurrence matrix records how often a target word occurs with each context word under a specified context definition. A window may be a sentence, document, or a fixed number of neighbouring tokens. State its width, direction and whether targets count themselves.

Choose target words
→
Choose context/window
→
Count co-occurrences
→
Store each row as a vector
→
Compare vectors

Question. Build the symmetric co-occurrence matrix for “cats eat fish” using a window of one token on either side, no self-counts and vocabulary [cats,eat,fish].

Given

One sentence, one occurrence of each token. Left and right adjacent tokens are contexts.

To find

A 3×3 target-by-context count matrix.

Formula and why it applies

\"X_{ij}\" counts the number of times context word \"j\" occurs within the chosen window of target \"i\".

Step 1. Target cats

Only eat is adjacent. Row cats = [0,1,0].

Step 2. Target eat

cats is left and fish is right. Row eat = [1,0,1].

Step 3. Target fish

Only eat is adjacent. Row fish = [0,1,0].

Step 4. Assemble the matrix

Target / contextcatseatfish
cats010
eat101
fish010

Answer. Rows are [0,1,0], [1,0,1], [0,1,0]. The symmetry follows from the symmetric window.

Question. Build a target–context matrix for sentences “cats eat fish” and “dogs eat fish”. Target words are cats and dogs; context columns [eat,fish]; use the entire sentence excluding the target.

Given

Each sentence contains one target. Count context token occurrences.

To find

Two context vectors.

Formula and why it applies

Count each context word in every sentence containing the target.

Step 1. Count cats contexts

In its sentence: eat occurs once, fish once → [1,1].

Step 2. Count dogs contexts

In its sentence: eat occurs once, fish once → [1,1].

Step 3. Compare

Both targets have identical observed context patterns in this very small corpus.

Answer. cats=[1,1], dogs=[1,1]. This corpus supplies no context evidence to distinguish them.

Question. For “a b a”, use a right-only window of one token. Build the matrix with rows/columns [a,b].

Given

Count directed target→next-word events, not both directions.

To find

The co-occurrence matrix.

Formula and why it applies

\"X_{ij}=C(\\text{target.

Step 1. First target a

Next token is b → add 1 to X(a,b).

Step 2. Target b

Next token is a → add 1 to X(b,a).

Step 3. Final a

No following token → no addition.

Step 4. Write matrix

Target / nextab
a01
b10

Answer. a=[0,1], b=[1,0]. A different context window would produce a different matrix.

Word ka meaning: uske aas-paas kaunse words kitni baar aaye, uski number-list.

","m2":false,"units":"5","marks":10,"sourcePages":[2],"historical":true},{"id":"q4b","title":"Q4b · Three-document TF–IDF","html":"
Question. Calculate TF–IDF for: (1) Inflation increased unemployment (2) Company increased sales (3) Fear increased pulse. [10 marks]

Answer

Given

DocumentOriginal text
1Inflation increased unemployment
2Company increased sales
3Fear increased pulse

Stopwords: none supplied

To find

BoW, IDF and TF–IDF

Formula: which one and why

Use lowercase alphabetic tokens and exactly the stated stoplist. If no stoplist is supplied, retain function words. Adopt raw-count TF and natural-log IDF; the paper does not prescribe another TF/log convention.

\"\\mathrm{TFIDF}(w,d)=c(w,d)\\ln\\frac{N}{\\mathrm{df}(w)}\"

A term shared by all N documents has IDF ln(1)=0. Counts and TF–IDF weights are different representations.

Step 1. Tokenize each document

DocumentTokens
1inflation, increased, unemployment
2company, increased, sales
3fear, increased, pulse

Step 2. Fix the vocabulary order

company, fear, increased, inflation, pulse, sales, unemployment

Step 3. Count occurrences (Bag of Words)

Documentcompanyfearincreasedinflationpulsesalesunemployment
10011001
21010010
30110100

Step 4. Calculate document frequency and IDF

TermdfN / dfln(N/df)
company131.098612289
fear131.098612289
increased310.000000000
inflation131.098612289
pulse131.098612289
sales131.098612289
unemployment131.098612289

Step 5. Multiply each TF by its IDF

Documentcompanyfearincreasedinflationpulsesalesunemployment
10.0000000.0000000.0000001.0986120.0000000.0000001.098612
21.0986120.0000000.0000000.0000000.0000001.0986120.000000
30.0000001.0986120.0000000.0000001.0986120.0000000.000000

Final answer

The final table is the TF–IDF matrix in the declared vocabulary order. A different explicitly stated log base rescales all IDFs; length-normalized TF changes each document scale.

","m2":false,"units":"2","marks":10,"sourcePages":[2],"historical":true},{"id":"q5a","title":"Q5a · Word embeddings and CBOW","html":"
Question. Elaborate word embeddings. Explain the steps in CBOW with a suitable diagram. [10 marks]

Answer

A sparse representation has many zero entries and often one coordinate per vocabulary item. BoW and TF–IDF are common sparse document representations. A dense embedding has comparatively few learned real-valued coordinates; one word's properties are spread across many coordinates.

RepresentationWhat coordinates meanStrengthLimitation
One-hotOne identity position per vocabulary wordUnambiguous IDDifferent words are orthogonal; no learned similarity
Count-based context vectorContext-word co-occurrence countsTransparent corpus evidenceLarge, sparse; frequent words may dominate
TF–IDF document vectorWeighted vocabulary termsUseful retrieval weightingWord order and contextual meaning are limited
Dense word embeddingLearned latent featuresCompact semantic/syntactic relationshipsCoordinates are not generally human-labelled features

Distributional describes learning from patterns of linguistic context. Distributed describes representing an item across multiple coordinates rather than a single active identity bit. These are related but different ideas. A one-hot ID alone does not learn distributional similarity.

Model design choices

Choose contexts (document, sentence, fixed window), weighting (raw counts, normalization, TF–IDF-like weighting or association scores), dimensionality reduction or learned embeddings, and a similarity measure. Cosine compares direction and is less affected by length than a raw dot product; Euclidean distance also reflects magnitude.

Sparse = bahut zero. Dense = chhoti learned number-list. Similar context se useful similarity seekhte hain.

Word2Vec learns dense word embeddings by predicting words from nearby context. Its two common architectures are CBOW and skip-gram. A vocabulary ID or one-hot vector is the input representation; the learned embedding table converts that identity into a short trainable vector.

ArchitectureInputPredictionMemory cue
CBOWSurrounding context wordsCentre/target wordContext → centre
Skip-gramCentre/target wordSurrounding wordsCentre → context
CBOW: the + sat
→
Average context embeddings
→
Predict: cat
Skip-gram: cat
→
Embedding of cat
→
Predict: the and sat

For vocabulary size \"V\" and embedding dimension \"d\", an input one-hot row is \"1\\times; the input embedding matrix is \"V\\times; the hidden row is \"1\\times; the output matrix is \"d\\times; output scores are \"1\\times. Multiplying a one-hot row by the embedding matrix simply selects one row.

\"P(w_o\\mid

\"v_{w_i}\" is the input word embedding, \"u_w\" an output-context embedding, and the dot product is a compatibility score. Softmax turns all scores into probabilities that sum to one. Training increases the probability of observed context words. Negative sampling or hierarchical softmax can reduce computation in larger vocabularies; use the algorithm specified by the question.

Question. For the sentence “the cat sat on mat”, generate CBOW and skip-gram training data for centre word sat with a window of one token on each side.

Given

Tokens indexed 1 to 5: the, cat, sat, on, mat. Centre = sat.

To find

The CBOW example and the skip-gram pairs.

Formula and why it applies

CBOW predicts the centre from its neighbours; skip-gram predicts each neighbour from the centre.

Step 1. Identify neighbours

Left neighbour = cat; right neighbour = on. “the” and “mat” are outside this window.

Step 2. CBOW example

Input [cat,on] → target sat.

Step 3. Skip-gram examples

(sat,cat) and (sat,on), with first item as input and second as predicted context.

Answer. CBOW: [cat,on]→sat. Skip-gram: sat→cat and sat→on.

Question. The embeddings of context words cat and on are [1,2] and [3,0]. Compute the CBOW hidden vector using mean aggregation. Output vectors for [sat,ran] are [1,0] and [0,1]. Compute softmax probabilities.

Given

Context embeddings [1,2], [3,0]; mean aggregation; two candidate output vectors.

To find

The hidden vector, logits and probabilities.

Formula and why it applies

Mean context vector \"h=(v_1+v_2)/2\". Logit \"z_w=u_w^Th\". Softmax \"P(w)=e^{z_w}/\\sum_j.

Step 1. Add coordinates

\"[1+3,\\,2+0]=[4,2]\".

Step 2. Divide by number of words

\"h=[4/2,2/2]=[2,1]\".

Step 3. Compute logits

\"z_{sat}=1(2)+0(1)=2\". \"z_{ran}=0(2)+1(1)=1\".

Step 4. Exponentials

\"e^2\\approx7.389056\", \"e^1\\approx2.718282\"; sum \"\\approx10.107338\".

Step 5. Divide

\"P(sat)\\approx0.731059\", \"P(ran)\\approx0.268941\".

Answer. h=[2,1]; sat is more probable (0.731059).

Question. A skip-gram centre embedding is v=[1,−1]. Output embeddings for [eat,sleep,drive] are [1,1], [0,1], [1,0]. Calculate the predicted context probabilities.

Given

One centre vector and three output vectors; full softmax over these three words.

To find

Three logits and three probabilities.

Formula and why it applies

\"z_j=u_j^Tv\", then \"P(j)=e^{z_j}/\\sum_k.

Step 1. Eat dot product

\"1(1)+1(-1)=0\".

Step 2. Sleep and drive dot products

\"z_{sleep}=0(1)+1(-1)=-1\". \"z_{drive}=1(1)+0(-1)=1\".

Step 3. Exponentiate and sum

\"e^0=1\", \"e^{-1}\\approx0.367879\", \"e^1\\approx2.718282\". Sum \"\\approx4.086161\".

Step 4. Normalize

\"P(eat)\\approx0.244728\"; \"P(sleep)\\approx0.090031\"; \"P(drive)\\approx0.665241\".

Answer. drive has the highest probability in this toy learned model.

Limitations: a standard static Word2Vec vector merges a word's senses; it has limited representation of subword morphology and out-of-vocabulary words; its local prediction objective does not directly fit a full global co-occurrence table. Similarity is learned from data and is not a guarantee of factual or causal understanding.

CBOW = neighbours se missing centre. Skip-gram = centre se neighbours.

","m2":false,"units":"5","marks":10,"sourcePages":[3],"historical":true},{"id":"q5b","title":"Q5b · HMM tables and given path","html":"
Question. For the four tagged sentences below, construct transition (3 marks) and emission (3 marks) tables. Calculate the path Geeta/N will/M spot/V Rita/N, given initial P(N)=0.5, P(M)=0.25, P(V)=0.25 (4 marks). [10 marks]

Answer

Given

SentenceGiven tags
Rita can teach WillN M V N
Spot will see WillN M V N
Will Rita spot GeetaN N V N
Geeta will pat SpotN M V N

Required path: Geeta will spot Rita / N M V N. Given initial probabilities: {'N': Fraction(1, 2), 'M': Fraction(1, 4), 'V': Fraction(1, 4)}

To find

Emission and transition tables and the specified tag-path joint probability

Formula: which one and why

\"P(w\\mid
\"P(t_j\\mid

The second formula is the declared no-END convention. Reset at sentence boundaries; never create a transition from the last word of one training sentence to the first of the next.

Step 1. Count emissions

TagTotal occurrencesRitacanteachWillSpotwillseespotGeetapat
N92003200020
M30100020000
V40010001101

Step 2. Divide each emission by its tag count

TagRitacanteachWillSpotwillseespotGeetapat
N2/9001/32/90002/90
M01/30002/30000
V001/40001/41/401/4

Step 3. Count transitions inside each sentence

FromOutgoing transitionsSentence-final countNMV
N54131
M30003
V40400

Step 4. Convert to a transition table

From / toNMV
N1/53/51/5
M001
V100

Here sentence-final tags are excluded from the no-END outgoing denominator. If END is explicitly modeled, divide all tag→tag and tag→END counts by the total tag count instead; the row then includes END. Do not mix denominators.

Step 5. Multiply the given path, in order

FactorProbabilityRunning product
Initial N1/21 × 1/2 = 1/2
Emit Geeta2/91/2 × 2/9 = 1/9
Transition N→M3/51/9 × 3/5 = 1/15
Emit will2/31/15 × 2/3 = 2/45
Transition M→V12/45 × 1 = 2/45
Emit spot1/42/45 × 1/4 = 1/90
Transition V→N11/90 × 1 = 1/90
Emit Rita2/91/90 × 2/9 = 1/405

Final answer

The two displayed probability tables are the estimated HMM. Zero means unobserved without smoothing, not logically impossible.

No-END path joint probability = 1/405 = 0.00246913580247. If a normalized END-state model is explicitly adopted instead, this complete terminated sequence has probability 4/6561 = 0.000609663161103. These are different model conventions, not contradictory answers.

","m2":false,"units":"4","marks":10,"sourcePages":[3],"historical":true},{"id":"q6a","title":"Q6a · Text classification pipeline","html":"
Question. Explain the text-classification pipeline in detail with a suitable block diagram. [10 marks]

Answer

Text classification assigns a document to one or more predefined categories. Spam detection usually uses two classes; news classification may use several topics. In single-label multiclass classification, choose one of several categories. In multilabel classification, several labels may apply to the same document.

Define labels
→
Prepare labelled examples
→
Train representation + model
→
Evaluate on unseen text
→
Predict new labels

An exam-ready pipeline

StageWhat you doWhy it matters
Task definitionDecide what counts as spam or which news topics exist.Unclear labels create inconsistent training data.
Data collection and annotationCollect representative examples and assign correct labels.The model learns from these examples.
Split dataSeparate training, validation and test data.Test examples must not influence training choices.
PreprocessTokenize and normalise where suitable; retain meaningful negation.Reduce irrelevant variation without deleting signal.
RepresentUse BoW, TF-IDF or learned embeddings.Models need numerical features.
TrainFit Naive Bayes, logistic regression or another appropriate model.Learn how features predict categories.
EvaluateMeasure relevant precision, recall and F1 on held-out examples.Accuracy alone may hide poor minority-class performance.
Deploy and monitorApply the same preprocessing and monitor changing language.Incoming data can differ from old examples.

Naive Bayes is especially useful for hand calculations. With a bag-of-words representation, its class score is the prior multiplied by the conditional probability of each token; repeated tokens repeat the factor. The “naive” assumption is conditional independence of features given the class, not that words are independent in real language.

\"s(c)=P(c)\\prod_{w\\in
\"P(w\\mid

C(w,d) counts word w in document d. C(w,c) counts it in training documents belonging to class c. Nc is the total token count for class c. Add-one smoothing adds one to every vocabulary count and therefore adds |V| to the denominator. This is the same arithmetic taught for WSD, with document classes instead of senses.

Worked classification · two-token message

A spam classifier has vocabulary {win, prize, meeting}. Priors are P(spam)=P(ham)=0.5. Spam token counts are [3,2,0]; ham counts are [0,0,5]. With add-one smoothing, classify “win prize”.

Given

|V|=3; N(spam)=5; N(ham)=5; message has win once and prize once.

To find

Class scores and predicted label.

Formula and why it applies

\"s(c)=P(c)P(\\text{win}\\mid
\"P(w\\mid

The model is multinomial Naive Bayes; multiply the two token likelihoods and the class prior.

Solution

Step 1. Compute the smoothed denominators

\"5+3=8\"

Step 2. Compute spam likelihoods

\"P(\\text{win}\\mid
\"P(\\text{prize}\\mid

Step 3. Compute ham likelihoods

\"P(\\text{win}\\mid

Step 4. Multiply the spam score

\"s(S)=\\frac12\\times\\frac12\\times\\frac38=\\frac{3}{32}=0.09375\"

Step 5. Multiply the ham score

\"s(H)=\\frac12\\times\\frac18\\times\\frac18=\\frac{1}{128}=0.0078125\"

Step 6. Compare

0.09375 > 0.0078125. These are unnormalised scores; the label is chosen by the larger score.

Predicted label: spam.

Yaad rakho: Class prior × har word ki class-likelihood; sabse bada score wins.

","m2":false,"units":"7","marks":10,"sourcePages":[3],"historical":true},{"id":"q6b","title":"Q6b · VSM, TF and IDF in retrieval","html":"
Question. Define Vector Space Model, Term Frequency and Inverse Document Frequency with reference to information retrieval. [10 marks]

Answer

A vocabulary is the ordered collection of distinct terms allowed by a representation. Its order defines what every coordinate means. A vector is simply an ordered number-list. A document–term matrix puts one document in each row and one vocabulary term in each column.

With vocabulary [cat,dog,fish], the count vector [2,0,1] means cat appears twice, dog zero times and fish once. With [dog,cat,fish], the same text becomes [0,2,1]. A vector without its vocabulary order is incomplete.

Documentcatdogfish
cat cat fish201
dog fish011

Nominal categories have no inherent order, such as cities. Ordinal categories do have an order, such as low < medium < high. Assigning arbitrary word IDs does not create meaningful numerical distances between words.

Vector ka har box ek fixed word ka hai. Pehle column order fix, phir numbers.

TF–IDF weights a term highly when it is frequent in a particular document but uncommon across the corpus. The supplied Unit 2 slides use normalized term frequency and base-10 inverse document frequency:

\"\\operatorname{TF}(t,d)=\\frac{C(t,d)}{N_d}\"
\"\\operatorname{IDF}(t)=\\log_{10}\\frac{D}{\\operatorname{df}(t)}\"
\"\\operatorname{TFIDF}(t,d)=\\operatorname{TF}(t,d)\\operatorname{IDF}(t).\"

\"N_d\" is the document's token count after the specified preprocessing; \"D\" is number of documents; \"\\operatorname{df}(t)\" is the number of documents containing the term at least once. A word repeated ten times in one document contributes one to document frequency.

Why the log? It compresses the ratio's scale. If a term occurs in all documents, \"D/df=1\", so IDF=0 under this unsmoothed convention. Other libraries use smoothed IDF, different TF or normalization. Use the question's formula; do not silently substitute a library default.

Question. A corpus contains 10 documents. The word model appears in 2 documents and occurs 3 times in a 12-token document. Compute TF, IDF and TF–IDF using the slide convention.

Given

D=10, df(model)=2, C(model,d)=3, Nd=12; log base 10.

To find

The TF–IDF weight.

Formula and why it applies

\"TF=C/N_d\", \"IDF=\\log_{10}(D/df)\", then multiply.

Step 1. TF

\"3/12=0.25\".

Step 2. Document ratio

\"10/2=5\".

Step 3. IDF

\"\\log_{10}5\\approx0.698970\".

Step 4. Weight

\"0.25\\times0.698970\\approx0.174743\".

Answer. TF=0.25, IDF≈0.698970, TF–IDF≈0.174743.

Question. A term occurs 4 times in a 20-token document and is present in all 8 corpus documents. Calculate its TF–IDF.

Given

C=4, Nd=20, D=8, df=8.

To find

Its weight under unsmoothed base-10 IDF.

Formula and why it applies

\"TF=C/N_d\", \"IDF=\\log_{10}(D/df)\".

Step 1. TF

\"4/20=0.2\".

Step 2. IDF

\"\\log_{10}(8/8)=\\log_{10}1=0\".

Step 3. Multiply

\"0.2\\times0=0\".

Answer. Weight 0: this term does not distinguish documents under this convention.

Question. Find TF–IDF vectors for D1=“cat cat dog”, D2=“dog fish”, D3=“cat fish fish”, using [cat,dog,fish], normalized TF and log10 IDF.

Given

No further preprocessing. Each term occurs in exactly two of the three documents.

To find

All three vectors.

Formula and why it applies

First count df by documents, then compute each document's TF, then multiply by its term's IDF.

Step 1. Common IDF

\"D=3\", \"df(cat)=df(dog)=df(fish)=2\". Each IDF \"=\\log_{10}(3/2)\\approx0.176091\".

Step 2. D1 term frequencies

Length 3 → \"[2/3,1/3,0]\".

Step 3. D1 weights

\"[(2/3)0.176091,(1/3)0.176091,0]\\approx[0.117394,0.058697,0]\".

Step 4. D2 weights

Length 2; TF \"[0,1/2,1/2]\". Vector \"\\approx[0,0.088046,0.088046]\".

Step 5. D3 weights

Length 3; TF \"[1/3,0,2/3]\". Vector \"\\approx[0.058697,0,0.117394]\".

Answer. D1≈[0.117394,0.058697,0]; D2≈[0,0.088046,0.088046]; D3≈[0.058697,0,0.117394].

TF: iss document mein kitna. IDF: poore corpus mein kitna rare. Dono multiply.

Information retrieval (IR) locates and ranks items relevant to an information need, usually expressed as a query. The collection may contain documents, product descriptions or scientific papers. Retrieval estimates relevance; it does not itself establish that every retrieved claim is true.

Documents → cleaning
→
Build index
→
Query → same representation
→
Match & rank
→
Return relevant results

The inverted index

An inverted index maps a term to the documents containing it, often with positions or counts. For D1 “cats drink milk” and D2 “dogs drink water”, the posting list for drink is {D1,D2}, while milk maps only to D1. This avoids rescanning every document for every query.

TermPosting list
catsD1
drinkD1, D2
milkD1
dogsD2
waterD2

Matching and ranking

Boolean retrieval combines term membership using AND, OR and NOT. Vector-space retrieval represents documents and queries as weighted vectors, for example with TF-IDF, and compares them with cosine similarity. Learned retrieval can use dense embeddings for meaning-based matching. The representation used at query time must be compatible with the document index.

Search engines and recommendation

A search engine receives an explicit query and ranks matching indexed pages. A recommender may derive an implicit query from a user’s liked items or profile and retrieve similar candidates. Both require candidate retrieval and ranking, but the source of the information need differs.

Evaluate the results

For retrieval, TP is the number of retrieved relevant documents, FP retrieved irrelevant documents, and FN relevant documents not retrieved. Precision and recall use the same formulas as classification. Ranking measures also consider the positions of relevant items; a relevant result at rank 1 is more useful than one far below.

Retrieval evaluation

A search system retrieves 8 documents; 6 are relevant. There are 10 relevant documents in the collection. Find precision, recall and F1.

Given

Retrieved=8; relevant retrieved=6; all relevant=10.

To find

P, R and F1.

Formula and why it applies

\"P=\\frac{\\text{relevant
\"F_1=\\frac{2PR}{P+R}\"

The denominator is 8 for precision and 10 for recall because they ask different questions.

Solution

Step 1. Identify the confusion counts

TP=6; FP=8−6=2; FN=10−6=4.

Step 2. Precision

\"P=\\frac68=0.75\"

Step 3. Recall

\"R=\\frac6{10}=0.6\"

Step 4. F1

\"F_1=\\frac{2(0.75)(0.6)}{0.75+0.6}=\\frac{0.9}{1.35}=\\frac23\\approx0.666667\"
Precision=0.75, recall=0.60, F1≈0.666667.

Yaad rakho: Precision: jo laaye, unmein kitne sahi? Recall: jo sahi the, unmein kitne laaye?

","m2":false,"units":"2,7","marks":10,"sourcePages":[3],"historical":true}]},{"id":"legacy-r23","title":"2023-01-30 · re-exam · recovered AI paper","unit":"PYQ","scope":"Previous examination · complete worked answers","sections":[{"id":"q1a","title":"Q1a · Normalization changes vocabulary","html":"
Question. “The vocabulary of a corpus remains the same before and after normalization.” State true or false and justify with an example. [5 marks]

Answer

False in general. Vocabulary is the set of distinct tokens under a chosen tokenizer. In “Cats cats running”, raw case-sensitive vocabulary={Cats,cats,running}, size3. Lowercase merges Cats/cats: {cats,running}, size2. Lemmatization may yield {cat,run}; stopword removal can remove entries. For an already normalized corpus the set may happen to remain unchanged, so the correct claim is that normalization can change it, not that it always must.

","m2":false,"units":"2","marks":5,"sourcePages":[1],"historical":true},{"id":"q1b","title":"Q1b · Good corpus characteristics","html":"
Question. State the characteristics of a good corpus. [5 marks]

Answer

A useful corpus is suited to its task and population: representative genres/languages, adequate size and diversity, balanced or intentionally documented sampling, reliable consistent annotations, clean encoding/tokenization, metadata and provenance, legal/ethical permission, documented train/test splits, and versioned reproducibility. “Large” alone does not imply representative. Example: a medical-tagging corpus needs clinical language and negation annotation; only movie reviews would be unsuitable.

","m2":false,"units":"3","marks":5,"sourcePages":[1],"historical":true},{"id":"q1c","title":"Q1c · Word2Vec versus Doc2Vec","html":"
Question. Differentiate Word2Vec and Doc2Vec. [5 marks]

Answer

AspectWord2VecDoc2Vec
RepresentationOne learned vector per word typeLearns document/paragraph vectors as well as, in variants, word vectors
TrainingCBOW predicts centre from context; Skip-gram predicts context from centrePV-DM combines document/context to predict words; PV-DBOW predicts sampled words from document ID
UnitWord/token typeSentence, paragraph or document
New textRetrieve known word vectors; combine via a separate pooling choiceInfer a new document vector while learned model weights stay fixed
UseWord similarity, featuresDocument similarity, retrieval or classification

Averaging Word2Vec word vectors is a possible document representation, but it is not the Doc2Vec training algorithm. This older paper names Doc2Vec explicitly; the current course names word and sentence embeddings but does not explicitly list Doc2Vec.

","m2":false,"units":"5","marks":5,"sourcePages":[1],"historical":true},{"id":"q1d","title":"Q1d · Ambiguity in NLP","html":"
Question. Explain the types of ambiguity in NLP with examples. [5 marks]

Answer

Ambiguity means an expression supports more than one interpretation. Good exam answers name the ambiguity and explain both readings.

ChallengeExampleWhy a system can fail
Lexical/sense ambiguitybank: finance or river edgeOne written word has several senses
Syntactic ambiguityI saw the man with a telescope.The telescope may modify saw or the man
Reference ambiguityRiya told Neha that she won.She may refer to either person without more context
SarcasmGreat, another delay.Literal positive word conveys frustration
NegationThe film was not good.Dropping not reverses meaning
Domain shift“Positive” in a medical reportMeaning/use differs by domain
Code mixingKal meeting reschedule karna.Several languages and spelling conventions
Noisy texttmrw, repeated letters, typosVocabulary/token boundaries become uncertain
Low-resource settingFew labelled examples in a languageModels lack representative training evidence
Bias and privacySkewed data or personal detailsErrors and harms can persist in deployed systems

Homophones sound alike but may differ in spelling (pair/pear). Homographs share spelling but can differ in meaning or pronunciation (lead the team / lead metal). These are different dimensions from syntactic ambiguity.

Long-distance dependencies, cultural knowledge, idioms, morphology and multilingual writing systems also complicate processing. For “kick the bucket”, literal word-by-word interpretation can miss the idiomatic meaning. For “The book that the students recommended is useful”, the verb is agrees with book across an intervening phrase.

Exam mein challenge ka naam + ek sentence + dono meanings/actual failure likho.

Structural ambiguity occurs when a sentence has more than one valid parse tree. In “I saw the man with a telescope”, the prepositional phrase “with a telescope” can attach to saw (the instrument I used) or to the man (the man possessing the telescope). The words and their order stay the same, but the grouping changes.

ReadingAttachmentMeaning
I [saw the man] [with a telescope]PP modifies VPI used a telescope to see the man.
I saw [the man with a telescope]PP modifies NPThe man had a telescope.

A parser may need context, probabilities or semantic constraints to choose between trees. A plain CFG describes permitted structures but does not itself assign a probability to each tree. Do not invent rule probabilities for an ordinary CFG question.

Lexical ambiguity also matters: book can be a noun or verb. In “Book the flight”, the imperative analysis begins S→VP, followed by VP→V NP. A parser that initially chooses book as a noun may reach a dead end and must try another permitted analysis.

Yaad rakho: Same words, alag grouping → alag meaning.

","m2":false,"units":"1,4","marks":5,"sourcePages":[1],"historical":true},{"id":"q2a","title":"Q2a · Normalize the Tanisha–Simran passage","html":"
Question. Perform sentence segmentation, word tokenization, stopword removal, lowercasing and stemming on the following paragraph; show vocabulary before and after: “Tanisha and Simran are best friends. They play together with other friends. Tanisha likes to play badminton but Simran prefers to play online games. Tanisha wants to be a badminton player. Simran wants to become an online gamer.” [10 marks]

Answer

No stoplist/stemmer is specified. For a reproducible answer choose stoplist {and,are,they,with,other,to,but,be,a,an} after lowercase; use the Porter stemmer. Other declared lists produce different valid counts.

StageResult
Sentence segmentationTanisha and Simran are best friends.
They play together with other friends.
Tanisha likes to play badminton but Simran prefers to play online games.
Tanisha wants to be a badminton player.
Simran wants to become an online gamer.
Word tokens (punctuation omitted)Tanisha, and, Simran, are, best, friends, They, play, together, with, other, friends, Tanisha, likes, to, play, badminton, but, Simran, prefers, to, play, online, games, Tanisha, wants, to, be, a, badminton, player, Simran, wants, to, become, an, online, gamer
Lowercasetanisha, and, simran, are, best, friends, they, play, together, with, other, friends, tanisha, likes, to, play, badminton, but, simran, prefers, to, play, online, games, tanisha, wants, to, be, a, badminton, player, simran, wants, to, become, an, online, gamer
Declared stoplista, an, and, are, be, but, other, they, to, with
After exact stop removaltanisha, simran, best, friends, play, together, friends, tanisha, likes, play, badminton, simran, prefers, play, online, games, tanisha, wants, badminton, player, simran, wants, become, online, gamer
Retained wordPorter stem
tanishatanisha
simransimran
bestbest
friendsfriend
playplay
togethertogeth
likeslike
badmintonbadminton
prefersprefer
onlineonlin
gamesgame
wantswant
playerplayer
becomebecom
gamergamer

Full final token sequence: tanisha, simran, best, friend, play, togeth, friend, tanisha, like, play, badminton, simran, prefer, play, onlin, game, tanisha, want, badminton, player, simran, want, becom, onlin, gamer

Raw case-sensitive vocabulary (25): Simran, Tanisha, They, a, an, and, are, badminton, be, become, best, but, friends, gamer, games, likes, online, other, play, player, prefers, to, together, wants, with. After lowercase only: 25 distinct words. Final stem vocabulary (15): badminton, becom, best, friend, game, gamer, like, onlin, play, player, prefer, simran, tanisha, togeth, want.

Source / convention note. Stoplist and stemming algorithm are choices, not supplied numerical givens; both are stated.

","m2":false,"units":"2","marks":10,"sourcePages":[1,2],"historical":true},{"id":"q2b","title":"Q2b · Text vectorization alternatives","html":"
Question. Define word embeddings and explain one-hot, Bag of Words, bag of n-grams and TF–IDF with advantages and disadvantages. [10 marks]

Answer

A sparse representation has many zero entries and often one coordinate per vocabulary item. BoW and TF–IDF are common sparse document representations. A dense embedding has comparatively few learned real-valued coordinates; one word's properties are spread across many coordinates.

RepresentationWhat coordinates meanStrengthLimitation
One-hotOne identity position per vocabulary wordUnambiguous IDDifferent words are orthogonal; no learned similarity
Count-based context vectorContext-word co-occurrence countsTransparent corpus evidenceLarge, sparse; frequent words may dominate
TF–IDF document vectorWeighted vocabulary termsUseful retrieval weightingWord order and contextual meaning are limited
Dense word embeddingLearned latent featuresCompact semantic/syntactic relationshipsCoordinates are not generally human-labelled features

Distributional describes learning from patterns of linguistic context. Distributed describes representing an item across multiple coordinates rather than a single active identity bit. These are related but different ideas. A one-hot ID alone does not learn distributional similarity.

Model design choices

Choose contexts (document, sentence, fixed window), weighting (raw counts, normalization, TF–IDF-like weighting or association scores), dimensionality reduction or learned embeddings, and a similarity measure. Cosine compares direction and is less affected by length than a raw dot product; Euclidean distance also reflects magnitude.

Sparse = bahut zero. Dense = chhoti learned number-list. Similar context se useful similarity seekhte hain.

MethodConstruction and exampleStrengthLimitation
One-hotOne1in vocabulary coordinate: cat=(1,0,0)Simple unique identitySparse; all distinct words equally distant
BoWCounts per vocabulary coordinateSimple interpretable frequencyLoses order and context
Bag of n-gramsCounts contiguous word groups, e.g. not goodRetains local order/phrasesVocabulary grows; sparse unseen phrases
TF–IDFTerm frequency weighted by inverse document frequencyDownweights ubiquitous terms for retrievalNo contextual meaning; depends on corpus statistics

Dense learned embeddings differ from these sparse count encodings: similarity may reflect learned context. Explain the representation before computing any metric.

","m2":false,"units":"2,5","marks":10,"sourcePages":[2],"historical":true},{"id":"q3a","title":"Q3a · Semantics, pragmatics and two ambiguities","html":"
Question. Differentiate semantics and pragmatics. Explain ambiguity in “Time flies like an arrow” and “He crushed the key to my heart”. [10 marks]

Answer

Semantics concerns conventional meaning and composition; pragmatics interprets an utterance using speaker intention and situation. “It is cold here” can semantically report temperature and pragmatically request closing a window.

SentenceReadings / cause
Time flies like an arrowOrdinary reading: time(N) flies(V), with like an arrow as comparison. Alternative imperative: time(V) the flies(N) as one times an arrow; lexical-category and structural ambiguity.
He crushed the key to my heartLiteral key/crushing versus figurative destruction of affection or emotional access. Lexical/semantic ambiguity and metaphor need pragmatic context; there is no supplied situation proving a single intended reading.
","m2":false,"units":"1,5,6","marks":10,"sourcePages":[2],"historical":true},{"id":"q3b","title":"Q3b · Naive Bayes text classifier","html":"
Question. Explain Naive Bayes text classification with a suitable example. [10 marks]

Answer

The paper supplies no numerical training corpus for this part. The following clearly labelled illustrative corpus demonstrates the complete calculation.

Given

Training textClass
Chinese Beijing Chineseyes
Chinese Chinese Shanghaiyes
Chinese Macaoyes
Tokyo Japan Chineseno

Test: Chinese Chinese Chinese Tokyo Japan

To find

Classify the test using multinomial Naive Bayes.

Formula: which one and why

Assume add-one smoothing unless the question supplies probabilities. Keep repeated words as repeated factors. Use training vocabulary only; ignore out-of-vocabulary words unless an unknown-token model is defined.

\"\\mathrm{score}(c)=P(c)\\prod_{w\\in

Step 1. Vocabulary and class priors

V = {beijing, chinese, japan, macao, shanghai, tokyo}; |V|=6.

ClassDocumentsPrior
no11/4
yes33/4

Step 2. Count every training word within each class

ClassTotal tokensbeijingchinesejapanmacaoshanghaitokyo
no3011001
yes8150110

Step 3. Count the test words

WordCount
chinese3
tokyo1
japan1

OOV words: none

Step 4. Calculate likelihoods and multiply with the prior

ClassPriorIndividual factorsExact scoreDecimal score
no1/4chinese: (1+1)/(3+1×6) = 2/9, used 3 time(s); tokyo: (1+1)/(3+1×6) = 2/9, used 1 time(s); japan: (1+1)/(3+1×6) = 2/9, used 1 time(s)8/590490.000135480702467
yes3/4chinese: (5+1)/(8+1×6) = 3/7, used 3 time(s); tokyo: (0+1)/(8+1×6) = 1/14, used 1 time(s); japan: (0+1)/(8+1×6) = 1/14, used 1 time(s)81/2689120.000301213779973

Step 5. Compare and optionally normalize

Normalized scores: no = 0.310241388, yes = 0.689758612

Division by the same total does not change the winning class.

Final answer

Predicted class: yes.

Yaad rakho: Count → prior → smoothed word chances → repeated factors → compare.

Source / convention note. Chinese corpus is an authored/reused teaching illustration here; it is actual given data in F22 Q5b, not this prompt.

","m2":false,"units":"7","marks":10,"sourcePages":[2],"historical":true},{"id":"q4a","title":"Q4a · Jaccard, cosine and Euclidean without preprocessing","html":"
Question. Explain Jaccard, cosine and Euclidean measures. Calculate them without preprocessing for D₁=“Data is the new oil of the digital economy” and D₂=“Data is a new oil”. [10 marks]

Answer

Given

Use exactly the given case and words; do not remove stopwords.

To find

Three measures in explicitly stated representations.

Formula: which one and why

\"J(A,B)=\\frac{|A\\cap

Jaccard uses sets of distinct tokens. Cosine and Euclidean use raw-count BoW vectors in one shared order; do not secretly turn repeated the into a single count.

Step 1. Build the common vocabulary and counts

DocumentDataisthenewoilofdigitaleconomya
D1112111110
D2110110001

Step 2. Jaccard sets

Intersection={Data,is,new,oil}, size4; union size9. J=4/9=0.444444; Jaccard distance=1−J=5/9.

Step 3. Cosine

Dot product=4. D1 squared length=1+1+4+1+1+1+1+1=11; D2 squared length=5. Cosine=4/√55=0.539359890.

Step 4. Euclidean

Coordinate differences: [0, 0, 2, 0, 0, 1, 1, 1, -1]. Sum of squares=8. Distance=√8=2.828427125. Smaller distance means greater proximity; this is a distance, not a probability.

Final answer

Jaccard=0.444444; cosine=0.539360; Euclidean distance=2.828427.

","m2":false,"units":"2","marks":10,"sourcePages":[2],"historical":true},{"id":"q4b","title":"Q4b · CBOW and Skip-gram","html":"
Question. Explain the CBOW and Skip-gram architectures. [10 marks]

Answer

Word2Vec learns dense word embeddings by predicting words from nearby context. Its two common architectures are CBOW and skip-gram. A vocabulary ID or one-hot vector is the input representation; the learned embedding table converts that identity into a short trainable vector.

ArchitectureInputPredictionMemory cue
CBOWSurrounding context wordsCentre/target wordContext → centre
Skip-gramCentre/target wordSurrounding wordsCentre → context
CBOW: the + sat
→
Average context embeddings
→
Predict: cat
Skip-gram: cat
→
Embedding of cat
→
Predict: the and sat

For vocabulary size \"V\" and embedding dimension \"d\", an input one-hot row is \"1\\times; the input embedding matrix is \"V\\times; the hidden row is \"1\\times; the output matrix is \"d\\times; output scores are \"1\\times. Multiplying a one-hot row by the embedding matrix simply selects one row.

\"P(w_o\\mid

\"v_{w_i}\" is the input word embedding, \"u_w\" an output-context embedding, and the dot product is a compatibility score. Softmax turns all scores into probabilities that sum to one. Training increases the probability of observed context words. Negative sampling or hierarchical softmax can reduce computation in larger vocabularies; use the algorithm specified by the question.

Question. For the sentence “the cat sat on mat”, generate CBOW and skip-gram training data for centre word sat with a window of one token on each side.

Given

Tokens indexed 1 to 5: the, cat, sat, on, mat. Centre = sat.

To find

The CBOW example and the skip-gram pairs.

Formula and why it applies

CBOW predicts the centre from its neighbours; skip-gram predicts each neighbour from the centre.

Step 1. Identify neighbours

Left neighbour = cat; right neighbour = on. “the” and “mat” are outside this window.

Step 2. CBOW example

Input [cat,on] → target sat.

Step 3. Skip-gram examples

(sat,cat) and (sat,on), with first item as input and second as predicted context.

Answer. CBOW: [cat,on]→sat. Skip-gram: sat→cat and sat→on.

Question. The embeddings of context words cat and on are [1,2] and [3,0]. Compute the CBOW hidden vector using mean aggregation. Output vectors for [sat,ran] are [1,0] and [0,1]. Compute softmax probabilities.

Given

Context embeddings [1,2], [3,0]; mean aggregation; two candidate output vectors.

To find

The hidden vector, logits and probabilities.

Formula and why it applies

Mean context vector \"h=(v_1+v_2)/2\". Logit \"z_w=u_w^Th\". Softmax \"P(w)=e^{z_w}/\\sum_j.

Step 1. Add coordinates

\"[1+3,\\,2+0]=[4,2]\".

Step 2. Divide by number of words

\"h=[4/2,2/2]=[2,1]\".

Step 3. Compute logits

\"z_{sat}=1(2)+0(1)=2\". \"z_{ran}=0(2)+1(1)=1\".

Step 4. Exponentials

\"e^2\\approx7.389056\", \"e^1\\approx2.718282\"; sum \"\\approx10.107338\".

Step 5. Divide

\"P(sat)\\approx0.731059\", \"P(ran)\\approx0.268941\".

Answer. h=[2,1]; sat is more probable (0.731059).

Question. A skip-gram centre embedding is v=[1,−1]. Output embeddings for [eat,sleep,drive] are [1,1], [0,1], [1,0]. Calculate the predicted context probabilities.

Given

One centre vector and three output vectors; full softmax over these three words.

To find

Three logits and three probabilities.

Formula and why it applies

\"z_j=u_j^Tv\", then \"P(j)=e^{z_j}/\\sum_k.

Step 1. Eat dot product

\"1(1)+1(-1)=0\".

Step 2. Sleep and drive dot products

\"z_{sleep}=0(1)+1(-1)=-1\". \"z_{drive}=1(1)+0(-1)=1\".

Step 3. Exponentiate and sum

\"e^0=1\", \"e^{-1}\\approx0.367879\", \"e^1\\approx2.718282\". Sum \"\\approx4.086161\".

Step 4. Normalize

\"P(eat)\\approx0.244728\"; \"P(sleep)\\approx0.090031\"; \"P(drive)\\approx0.665241\".

Answer. drive has the highest probability in this toy learned model.

Limitations: a standard static Word2Vec vector merges a word's senses; it has limited representation of subword morphology and out-of-vocabulary words; its local prediction objective does not directly fit a full global co-occurrence table. Similarity is learned from data and is not a guarantee of factual or causal understanding.

CBOW = neighbours se missing centre. Skip-gram = centre se neighbours.

","m2":false,"units":"5","marks":10,"sourcePages":[2],"historical":true},{"id":"q5a","title":"Q5a · Password regex","html":"
Question. Write a regex for an 8–15 character password containing letters and digits, at least one uppercase letter and at least one special character from !%@*, with no other characters. [10 marks]

Answer

^(?=.{8,15}$)(?=.*[A-Z])(?=.*[0-9])(?=.*[!%@*])[A-Za-z0-9!%@*]+$

PartPurpose
(?=.{8,15}$)Length8–15
(?=.*[A-Z])At least one uppercase; this also meets the letter requirement
(?=.*[0-9])At least one digit
(?=.*[!%@*])At least one permitted special
[A-Za-z0-9!%@*]+Only permitted characters, including lowercase if desired

Use full-string matching. Abcdef1! passes; abcdef1! fails uppercase, Abcdefgh fails digit/special, Abc1! is too short, Abcdef1# contains forbidden #. A lowercase character is not separately required by the printed conditions.

","m2":false,"units":"3","marks":10,"sourcePages":[2,3],"historical":true},{"id":"q5b","title":"Q5b · Two NLP applications","html":"
Question. Briefly explain any two applications of NLP. [10 marks]

Answer

1. Machine translation

Translate meaning from source to target language using a model conditioned on the source sequence. Example: English “Where is the station?” to Hindi; word order and ambiguity require context. Train on parallel corpora and evaluate adequacy/fluency.

2. Information extraction

Convert text into structured entities and relations. Example: “Acme acquired Beta in 2022” becomes acquirer=Acme, acquired=Beta, year=2022. Pipeline: tokenize → NER → relation extraction → validate output with source spans.

","m2":false,"units":"7","marks":10,"sourcePages":[3],"historical":true},{"id":"q6a","title":"Q6a · Mention-pair model","html":"
Question. Explain the mention-pair model for reference resolution. [10 marks]

Answer

A mention is a span referring to an entity. A mention-pair model classifies a pair (earlier candidate antecedent, later mention) as coreferent or not. Pairwise decisions are subsequently linked into entity clusters; a positive score alone does not ensure globally consistent clusters.

Detect mentions → Generate earlier candidate pairs → Extract pair features → Score same-entity relation → Select/link antecedents → Form entity clusters
  1. Text: “Rita bought a car. She likes it.” Detect Rita, a car, She, it.
  2. For She, compare (Rita,She) and (a car,She). Features include person/number compatibility, animate type, syntactic role, sentence distance and lexical agreement.
  3. Rita–She has compatible singular human features; car–She fails the ordinary inanimate reading. Select Rita.
  4. For it, compare earlier mentions: car–it is singular/inanimate and contextually fits likes. Cluster {Rita,She} and {a car,it}.
  5. Train a classifier on labelled mention pairs; at prediction, choose a sufficiently supported antecedent or leave a new entity. Avoid reflexive/binding violations and document leakage during evaluation.

Advantages: interpretable pair features and supervised learning. Limitations: quadratic candidate growth, error propagation from mention detection, non-transitive pair predictions, and weak long-range/world-knowledge reasoning. A greedy nearest-candidate choice is not the same as a learned mention-pair model.

","m2":false,"units":"6","marks":10,"sourcePages":[3],"historical":true},{"id":"q6b","title":"Q6b · Classification evaluation including ROC/AUC","html":"
Question. Explain accuracy, precision, recall, F1 score, ROC and AUC. [10 marks]

Answer

Evaluation compares predicted results with known answers. A true positive is a correctly predicted positive; a false positive is a negative item predicted positive; a false negative is a missed positive; a true negative is a correctly rejected negative.

\"\\text{Precision}=\\frac{TP}{TP+FP}\"
\"\\text{Recall}=\\frac{TP}{TP+FN}\"
\"F_1=\\frac{2PR}{P+R}=\\frac{2TP}{2TP+FP+FN}\"
\"\\text{Accuracy}=\\frac{TP+TN}{TP+TN+FP+FN}\"

Precision asks how many predicted positives were correct. Recall asks how many actual positives were found. F1 balances them by a harmonic mean. If a denominator is zero, state the convention rather than dividing by zero.

Example 1 · evaluate a classifier

A binary text classifier gives TP=6, FP=2, FN=4, TN=8. Compute precision, recall, F1 and accuracy.

Given

TP=6; FP=2; FN=4; TN=8.

To find

Four evaluation measures.

Formula and why it applies

\"P=\\frac{TP}{TP+FP},\\quad
\"F_1=\\frac{2TP}{2TP+FP+FN},\\quad

Use predicted positives in precision, actual positives in recall, and all examples in accuracy.

Solution

Step 1. Precision

\"P=\\frac{6}{6+2}=0.750000\"

Step 2. Recall

\"R=\\frac{6}{6+4}=0.600000\"

Step 3. F1

\"F_1=\\frac{12}{12+2+4}=0.666667\"

Step 4. Accuracy

\"A=\\frac{6+8}{6+2+4+8}=0.700000\"
Precision 0.7500; recall 0.6000; F1 0.6667; accuracy 0.7000.

Example 2 · evaluate a classifier

A binary text classifier gives TP=9, FP=3, FN=1, TN=7. Compute precision, recall, F1 and accuracy.

Given

TP=9; FP=3; FN=1; TN=7.

To find

Four evaluation measures.

Formula and why it applies

\"P=\\frac{TP}{TP+FP},\\quad
\"F_1=\\frac{2TP}{2TP+FP+FN},\\quad

Use predicted positives in precision, actual positives in recall, and all examples in accuracy.

Solution

Step 1. Precision

\"P=\\frac{9}{9+3}=0.750000\"

Step 2. Recall

\"R=\\frac{9}{9+1}=0.900000\"

Step 3. F1

\"F_1=\\frac{18}{18+3+1}=0.818182\"

Step 4. Accuracy

\"A=\\frac{9+7}{9+3+1+7}=0.800000\"
Precision 0.7500; recall 0.9000; F1 0.8182; accuracy 0.8000.

Example 3 · evaluate a classifier

A binary text classifier gives TP=4, FP=0, FN=4, TN=12. Compute precision, recall, F1 and accuracy.

Given

TP=4; FP=0; FN=4; TN=12.

To find

Four evaluation measures.

Formula and why it applies

\"P=\\frac{TP}{TP+FP},\\quad
\"F_1=\\frac{2TP}{2TP+FP+FN},\\quad

Use predicted positives in precision, actual positives in recall, and all examples in accuracy.

Solution

Step 1. Precision

\"P=\\frac{4}{4+0}=1.000000\"

Step 2. Recall

\"R=\\frac{4}{4+4}=0.500000\"

Step 3. F1

\"F_1=\\frac{8}{8+0+4}=0.666667\"

Step 4. Accuracy

\"A=\\frac{4+12}{4+0+4+12}=0.800000\"
Precision 1.0000; recall 0.5000; F1 0.6667; accuracy 0.8000.

ROC and AUC

\"\\mathrm{TPR}=\\frac{TP}{TP+FN},\\qquad\\mathrm{FPR}=\\frac{FP}{FP+TN}\"

A classifier produces scores. Sweep the decision threshold; at each threshold calculate TPR and FPR. Plot FPR horizontally and TPR vertically. AUC is the area under that curve, equivalently the probability that a randomly chosen positive has a higher score than a randomly chosen negative (ties conventionally contribute half). AUC1 is perfect ranking;0.5 is chance ranking. This evaluates ranking over thresholds, not the accuracy at one threshold.

Illustrative thresholdTPRFPR
Above all scores00
Middle threshold0.80.2
Below all scores11

For straight segments through (0,0),(.2,.8),(1,1), trapezoid area=.2×(0+.8)/2+.8×(.8+1)/2=.08+.72=.80. These points illustrate integration; they are not supplied exam data. Precision–recall may be more informative for rare positives.

","m2":false,"units":"7","marks":10,"sourcePages":[3],"historical":true},{"id":"q7a","title":"Q7a · Organization and facility mentions","html":"
Question. Identify organization and facility mentions in this White House visitor passage using the supplied ACE categories: “After the 9/11 attacks, the system changed radically. Now, anyone who wants to tour the White House must apply through the office of his or her representative in Congress, which forwards the names to the White House for clearance. Once they get the green light, visitors show up at the appointed time on 15th Street between E and F Streets and join the line to enter through the southeast gate. Anyone who has flown on an airline in recent years will recognize the familiar territory of identity checks and electronic scans, although you do get to keep your shoes on. At the head of the line, rangers from the National Park Service check photo IDs against a list of names.” [10 marks]

Answer

Mention in orderTypeReason
White House (tour)FACPhysical building being toured
office of his or her representativeORG under administrative-office readingStaff/organizational office receiving applications; no visit to the physical office is described
CongressORGLegislative organization
White House (clearance)ORGInstitution/staff granting clearance, a metonymic use
15th StreetFACTransport infrastructure
E and F StreetsFAC (two coordinated streets)Roads, not organizations
southeast gateFACPhysical entrance component
an airlineORG generic mentionOrganization operating flights; generic, not a proper named company
National Park ServiceORGGovernment agency

The prompt uses ACE entity mentions: nominal/generic mentions can be included, unlike a named-only tagger. 9/11 is an event/date mention, not ORG/FAC; visitors and rangers are people. If a narrower classroom guideline asks only named entities, omit the generic office/airline/gate spans and explicitly state that convention. Context decides whether White House means building or administration.

Source / convention note. Boundary and generic-mention decisions depend on the provided ACE annotation convention; the answer distinguishes them from a named-only interpretation.

","m2":false,"units":"4","marks":10,"sourcePages":[3,4],"historical":true},{"id":"q7b","title":"Q7b · Tagging and two complete parse trees","html":"
Question. Explain POS tagging. Tag and parse (1) John and Mary bought a refrigerator with three doors. (2) It was purchased from a very small store near their house. [10 marks]

Answer

POS tagging assigns grammatical categories in context; constituency parsing groups those words hierarchically. Here with three doors modifies refrigerator; near their house modifies store. These attachments fit the natural reading; the paper gives no fixed CFG.

John/NNP and/CC Mary/NNP bought/VBD a/DT refrigerator/NN with/IN three/CD doors/NNS

SNPNNPJohnCCandNNPMaryVPVBDboughtNPDTaNNrefrigeratorPPINwithNPCDthreeNNSdoors
[S [NP [NNP John] [CC and] [NNP Mary]] [VP [VBD bought] [NP [DT a] [NN refrigerator] [PP [IN with] [NP [CD three] [NNS doors]]]]]]

It/PRP was/VBD purchased/VBN from/IN a/DT very/RB small/JJ store/NN near/IN their/PRP$ house/NN

SNPPRPItVPVBDwasVPVBNpurchasedPPINfromNPDTaADJPRBveryJJsmallNNstorePPINnearNPPRP$theirNNhouse
[S [NP [PRP It]] [VP [VBD was] [VP [VBN purchased] [PP [IN from] [NP [DT a] [ADJP [RB very] [JJ small]] [NN store] [PP [IN near] [NP [PRP$ their] [NN house]]]]]]]]

The first subject is coordinated; bought is past tense. The second VP is passive auxiliary was plus past participle purchased. Very modifies small, and their is a possessive determiner.

","m2":false,"units":"4","marks":10,"sourcePages":[4],"historical":true}]},{"id":"legacy-f22","title":"2022-11-22 · final · recovered AI paper","unit":"PYQ","scope":"Previous examination · complete worked answers","sections":[{"id":"q1a","title":"Q1a · Need and difficulties of NLP","html":"
Question. Explain the need for NLP and the difficulties in natural language processing. [5 marks]

Answer

Natural language processing (NLP) is the field concerned with computational methods for analysing, interpreting and generating human language. Natural languages such as English, Hindi and Marathi evolved through human use; their meanings are not specified by the rigid rules of a programming language.

An NLP system may receive text, speech converted to text, or another linguistic representation. Its output depends on the task: a category, a tag for every word, a translated sentence, a ranked document list or a generated answer. It does not always “understand” in the human sense; its performance must be evaluated for the specific task.

TaskInputOutput
Sentiment analysis“The service was excellent.”Positive sentiment
Text classificationAn emailSpam / not spam
POS tagging“Birds fly.”Birds/NOUN fly/VERB
Named entity recognition“Asha joined Google in Mumbai.”Asha: person; Google: organisation; Mumbai: location
Machine translationA Hindi sentenceA corresponding English sentence
Information retrievalA search queryRanked relevant documents
Question answeringQuestion plus knowledge/contextAn answer
SummarizationA long articleA shorter text preserving key information

How NLP relates to AI, ML and DL

AI is the broader study of systems performing tasks associated with intelligent behaviour. Machine learning learns patterns from data. Deep learning uses multiple trainable neural layers. NLP is an application/research field that can use hand-written rules, statistical ML or deep learning. Therefore, not every NLP system is a neural network.

Linguistics supplies knowledge about language structure. Statistics supplies uncertainty models. Computer science supplies algorithms, storage and efficient implementation. A practical NLP system combines these ideas.

NLP = human language ko computational task mein badlo: label, meaning, search ya generated sentence.

Ambiguity means an expression supports more than one interpretation. Good exam answers name the ambiguity and explain both readings.

ChallengeExampleWhy a system can fail
Lexical/sense ambiguitybank: finance or river edgeOne written word has several senses
Syntactic ambiguityI saw the man with a telescope.The telescope may modify saw or the man
Reference ambiguityRiya told Neha that she won.She may refer to either person without more context
SarcasmGreat, another delay.Literal positive word conveys frustration
NegationThe film was not good.Dropping not reverses meaning
Domain shift“Positive” in a medical reportMeaning/use differs by domain
Code mixingKal meeting reschedule karna.Several languages and spelling conventions
Noisy texttmrw, repeated letters, typosVocabulary/token boundaries become uncertain
Low-resource settingFew labelled examples in a languageModels lack representative training evidence
Bias and privacySkewed data or personal detailsErrors and harms can persist in deployed systems

Homophones sound alike but may differ in spelling (pair/pear). Homographs share spelling but can differ in meaning or pronunciation (lead the team / lead metal). These are different dimensions from syntactic ambiguity.

Long-distance dependencies, cultural knowledge, idioms, morphology and multilingual writing systems also complicate processing. For “kick the bucket”, literal word-by-word interpretation can miss the idiomatic meaning. For “The book that the students recommended is useful”, the verb is agrees with book across an intervening phrase.

Exam mein challenge ka naam + ek sentence + dono meanings/actual failure likho.

","m2":false,"units":"1","marks":5,"sourcePages":[1],"historical":true},{"id":"q1b","title":"Q1b · Levels of language analysis","html":"
Question. Explain the levels of language analysis with examples. [5 marks]

Answer

A phoneme is a sound unit that can distinguish meanings in a language, such as the /p/ and /b/ contrast in “pat” and “bat”. A phoneme is a sound, not necessarily one alphabet letter. A morpheme is the smallest meaning-bearing grammatical unit: cats = cat + plural -s; unhappy = un- + happy.

A lexeme is an abstract vocabulary item grouping its inflected forms: write, writes, wrote and writing belong to WRITE. A phrase is a group of words functioning together; a clause normally contains a predicate and its associated participants; a sentence expresses a complete grammatical unit.

LevelWhat it analysesConcrete example
Phonetics / phonologyPhysical sounds / sound patterns in a languageDistinguish bat from pat
Morphology / lexical analysisWord structure and lexical itemsun + help + ful → unhelpful
SyntaxGrammatical arrangement and relationshipsThe dog chased the cat: subject, verb, object
SemanticsMeaning of words and composed expressionsThe dog is the chaser; the cat is chased
DiscourseConnections across sentences“Riya arrived. She smiled.” → She refers to Riya
PragmaticsIntended meaning in the situation“It is cold here” may request closing the window
word forms
→
grammatical structure
→
sentence meaning
→
cross-sentence context
→
speaker intention

Three worked interpretations

“The cats are sleeping.” Morphology identifies cat + s as plural. Syntax connects “cats” to “are sleeping”. Semantics represents animals engaged in sleeping.

“Aman bought a laptop. It was expensive.” Sentence-level syntax alone does not identify “it”. Discourse connects “it” to the laptop across the sentence boundary.

“Wonderful, another power cut!” The word wonderful is positive literally, but situational context can indicate sarcasm and frustration. This is a pragmatic interpretation.

The analysis levels are conceptual descriptions. Modern systems may learn several jointly rather than executing a compulsory separate module for every level.

Morphology = word ke andar; syntax = sentence ka structure; semantics = meaning; discourse = sentences ka link; pragmatics = intention.

","m2":false,"units":"1","marks":5,"sourcePages":[1],"historical":true},{"id":"q1c","title":"Q1c · Medical chatbot","html":"
Question. Briefly explain a medical chatbot and its NLP processing. [5 marks]

Answer

A medical chatbot accepts a patient’s natural-language message and provides an appropriately bounded response, such as appointment information or approved health education. It must distinguish intents and preserve symptoms, negations and durations.

Patient message → Tokenization + intent/entity detection → Retrieve approved information → Generate grounded response → Escalate to clinician when needed

Example: “I have no fever but a cough for three days” yields cough present, fever absent and duration three days. A safe administrative bot can suggest an appointment; it must not turn uncertain text into a definitive diagnosis. Evaluate intent accuracy, entity/negation accuracy, response correctness, privacy and escalation behaviour.

","m2":false,"units":"7","marks":5,"sourcePages":[1],"historical":true},{"id":"q1d","title":"Q1d · Sentiment with logistic regression","html":"
Question. Explain sentiment analysis using logistic regression with an example. [5 marks]

Answer

Binary logistic regression maps text features x to a probability:

\"z=w^Tx+b,\\qquad

Use positive-word count and negative-word count as an illustrative feature vector. With x=(2,1), w=(1,−1), b=0: z=1×2+(−1)×1=1; P(positive)=1/(1+e⁻¹)=0.731059. A declared 0.5 threshold predicts positive. Fit weights from labelled reviews by minimizing cross-entropy; the hand-chosen weights here only illustrate the computation.

Labelled reviews → Tokenize / vectorize → Fit logistic weights → Probability → Threshold / class
","m2":false,"units":"7","marks":5,"sourcePages":[1],"historical":true},{"id":"q2a","title":"Q2a · CBOW and Skip-gram architecture","html":"
Question. Explain CBOW and Skip-gram with suitable architecture diagrams. [10 marks]

Answer

Word2Vec learns dense word embeddings by predicting words from nearby context. Its two common architectures are CBOW and skip-gram. A vocabulary ID or one-hot vector is the input representation; the learned embedding table converts that identity into a short trainable vector.

ArchitectureInputPredictionMemory cue
CBOWSurrounding context wordsCentre/target wordContext → centre
Skip-gramCentre/target wordSurrounding wordsCentre → context
CBOW: the + sat
→
Average context embeddings
→
Predict: cat
Skip-gram: cat
→
Embedding of cat
→
Predict: the and sat

For vocabulary size \"V\" and embedding dimension \"d\", an input one-hot row is \"1\\times; the input embedding matrix is \"V\\times; the hidden row is \"1\\times; the output matrix is \"d\\times; output scores are \"1\\times. Multiplying a one-hot row by the embedding matrix simply selects one row.

\"P(w_o\\mid

\"v_{w_i}\" is the input word embedding, \"u_w\" an output-context embedding, and the dot product is a compatibility score. Softmax turns all scores into probabilities that sum to one. Training increases the probability of observed context words. Negative sampling or hierarchical softmax can reduce computation in larger vocabularies; use the algorithm specified by the question.

Question. For the sentence “the cat sat on mat”, generate CBOW and skip-gram training data for centre word sat with a window of one token on each side.

Given

Tokens indexed 1 to 5: the, cat, sat, on, mat. Centre = sat.

To find

The CBOW example and the skip-gram pairs.

Formula and why it applies

CBOW predicts the centre from its neighbours; skip-gram predicts each neighbour from the centre.

Step 1. Identify neighbours

Left neighbour = cat; right neighbour = on. “the” and “mat” are outside this window.

Step 2. CBOW example

Input [cat,on] → target sat.

Step 3. Skip-gram examples

(sat,cat) and (sat,on), with first item as input and second as predicted context.

Answer. CBOW: [cat,on]→sat. Skip-gram: sat→cat and sat→on.

Question. The embeddings of context words cat and on are [1,2] and [3,0]. Compute the CBOW hidden vector using mean aggregation. Output vectors for [sat,ran] are [1,0] and [0,1]. Compute softmax probabilities.

Given

Context embeddings [1,2], [3,0]; mean aggregation; two candidate output vectors.

To find

The hidden vector, logits and probabilities.

Formula and why it applies

Mean context vector \"h=(v_1+v_2)/2\". Logit \"z_w=u_w^Th\". Softmax \"P(w)=e^{z_w}/\\sum_j.

Step 1. Add coordinates

\"[1+3,\\,2+0]=[4,2]\".

Step 2. Divide by number of words

\"h=[4/2,2/2]=[2,1]\".

Step 3. Compute logits

\"z_{sat}=1(2)+0(1)=2\". \"z_{ran}=0(2)+1(1)=1\".

Step 4. Exponentials

\"e^2\\approx7.389056\", \"e^1\\approx2.718282\"; sum \"\\approx10.107338\".

Step 5. Divide

\"P(sat)\\approx0.731059\", \"P(ran)\\approx0.268941\".

Answer. h=[2,1]; sat is more probable (0.731059).

Question. A skip-gram centre embedding is v=[1,−1]. Output embeddings for [eat,sleep,drive] are [1,1], [0,1], [1,0]. Calculate the predicted context probabilities.

Given

One centre vector and three output vectors; full softmax over these three words.

To find

Three logits and three probabilities.

Formula and why it applies

\"z_j=u_j^Tv\", then \"P(j)=e^{z_j}/\\sum_k.

Step 1. Eat dot product

\"1(1)+1(-1)=0\".

Step 2. Sleep and drive dot products

\"z_{sleep}=0(1)+1(-1)=-1\". \"z_{drive}=1(1)+0(-1)=1\".

Step 3. Exponentiate and sum

\"e^0=1\", \"e^{-1}\\approx0.367879\", \"e^1\\approx2.718282\". Sum \"\\approx4.086161\".

Step 4. Normalize

\"P(eat)\\approx0.244728\"; \"P(sleep)\\approx0.090031\"; \"P(drive)\\approx0.665241\".

Answer. drive has the highest probability in this toy learned model.

Limitations: a standard static Word2Vec vector merges a word's senses; it has limited representation of subword morphology and out-of-vocabulary words; its local prediction objective does not directly fit a full global co-occurrence table. Similarity is learned from data and is not a guarantee of factual or causal understanding.

CBOW = neighbours se missing centre. Skip-gram = centre se neighbours.

","m2":false,"units":"5","marks":10,"sourcePages":[1],"historical":true},{"id":"q2b","title":"Q2b · Seven regex constructions","html":"
Question. Define regular expressions and their uses. Write expressions for (i) all alphabetic strings, (ii) lowercase alphabetic strings ending in b, (iii) a/b strings where each a is immediately preceded and followed by b, (iv) consecutive repeated words, (v) a line starting with an integer and ending with a word, (vi) strings containing the whole words grotto and raven, (vii) capture the first English word, handling opening punctuation. [10 marks]

Answer

A regular expression specifies a pattern for matching text. NLP uses it for token extraction, cleaning, validation and rule-based entities. The following uses a Python/PCRE-like engine with lookaheads and backreferences; these extended features go beyond a strictly finite-state regular language.

TaskPatternWhy it works
All alphabetic strings^[A-Za-z]+$Only letters; + requires at least one. Use * if empty string is allowed.
Lowercase strings ending in b^[a-z]*b$The final b is mandatory.
a/b strings: every a surrounded by b^(?!a)(?!.*aa)(?!.*a$)[ab]*$Forbid starting a, adjacent aa, ending a. Then every a has a b on both sides.
Two consecutive identical words\\b([A-Za-z]+)\\s+\\1\\bCapture word, whitespace, same captured word. Case-sensitive unless ignore-case is enabled.
Start with integer; end with word^[0-9]+\\b.*\\b[A-Za-z]+$Same line; signed integers need optional [+-].
Whole grotto and whole raven anywhere^(?=.*\\bgrotto\\b)(?=.*\\braven\\b).*$Two positive lookaheads require both whole words. Dot excludes newlines unless DOTALL.
Capture first word despite opening punctuation^[^A-Za-z]*([A-Za-z]+(?:[\\x27’][A-Za-z]+)*)Group1 is first alphabetic word, with internal apostrophes allowed.

Tests: b and bab pass rule3; a, ab, ba and baab fail. “Humbert Humbert” matches rule4; “the big bug” does not. “grottos raven” fails rule6. Regex word rules are practical ASCII conventions here, not a full linguistic tokenizer.

","m2":false,"units":"3","marks":10,"sourcePages":[1,2],"historical":true},{"id":"q3a","title":"Q3a · Text preprocessing","html":"
Question. Explain text preprocessing with applications and examples. [10 marks]

Answer

Tokenization divides text into units called tokens for further processing. A token may be a word, punctuation mark, character or subword, depending on the tokenizer. Sentence segmentation identifies sentence boundaries. Tokenization is not necessarily just splitting on spaces.

MethodExampleTrade-off
Word“cats sleep” → [cats,sleep]Easy to interpret; unfamiliar words can be unseen tokens
Charactercat → [c,a,t]Handles spelling units; creates longer sequences
Subwordunhappiness → pieces such as [un,happi,ness] in an illustrative vocabularyBalances vocabulary size and rare-word representation
Sentence“Birds fly. Fish swim.” → two sentencesAbbreviations and punctuation complicate boundaries

Actual subword splits depend on the learned vocabulary; the example is illustrative, not a universal tokenizer output. Word segmentation is especially important where spaces do not reliably separate words or in compound-rich languages.

Question. Tokenize “Dr. Rao can't come today.” into words and punctuation under this rule: preserve Dr. and can't as whole tokens, and split only the final full stop.

Given

The tokenization rule is explicitly supplied.

To find

The resulting token sequence.

Formula and why it applies

Apply the stated boundary rule; do not assume every period or apostrophe is a split point.

Step 1. Handle abbreviation

Dr. remains one token.

Step 2. Handle contraction

can't remains one token.

Step 3. Write sequence

[Dr., Rao, can't, come, today, .]

Answer. 6 tokens under the specified rule. Another tokenizer may legitimately produce a different sequence.

Three difficult cases

New York may be two tokens but one named entity. 3.14 should not normally end a sentence at the decimal point. don't may remain whole or split into do + n't, depending on the tokenizer. Always state the convention when calculating counts.

Token = chosen unit. Space split ko universal rule mat samajh.

Normalization reduces superficial variation so comparable forms can be processed consistently. Common operations include lowercasing, Unicode normalization, appropriate spelling correction, contraction handling, and optional stemming or lemmatization.

Stopwords are frequent words selected for removal for a particular task, such as the or a. Removing them may reduce feature size, but removing not, no or nor can destroy sentiment meaning. Punctuation can convey sentence boundaries, emotion or mathematical structure. Lowercasing can erase differences such as US versus us.

Question. Preprocess “The CATS are not sleeping!!!” by lowercasing, removing punctuation, removing only stopwords {the,are}, and lemmatizing plural cats→cat and sleeping→sleep.

Given

Operations and stopword set are fixed. Do not remove not.

To find

The final token list.

Formula and why it applies

Apply the requested operations in order, keeping a visible intermediate result after each transformation.

Step 1. Lowercase

the cats are not sleeping!!!

Step 2. Tokenize and remove punctuation

[the,cats,are,not,sleeping]

Step 3. Remove specified stopwords

Remove the and are → [cats,not,sleeping].

Step 4. Lemmatize

cats→cat; sleeping→sleep → [cat,not,sleep].

Answer. [cat,not,sleep]. The negative meaning is retained.

Task decides preprocessing

For a simple keyword-retrieval model, removing selected function words can help. For machine translation, question answering, grammar analysis or transformer input, aggressive removal may discard necessary syntax or violate the model's expected input format. Emoji can carry sentiment; convert or retain them when relevant instead of always deleting them.

Clean karna = useful signal bachake noise hatao. Har word/punctuation delete karna cleaning nahi.

Stemming applies string-based rules, often stripping suffixes, to reduce related forms to a stem. The result need not be a valid dictionary word. Lemmatization uses vocabulary and morphological analysis, often with POS information, to return a dictionary lemma.

CriterionStemmingLemmatization
BasisSurface form/suffix rulesLexicon and morphological/POS analysis
OutputMay be nonword, e.g. studiDictionary form, e.g. study
Context/POSUsually little or noneOften important
Irregular formsMay miss went→goCan map went→go with verb analysis
CostGenerally simplerUsually more linguistic processing

Porter applies a sequence of suffix-removal rules subject to conditions; Snowball provides related language-specific stemming algorithms; Lancaster is often more aggressive; regex stemming uses explicit patterns; Lovins is another established stemming approach. Exact output depends on the algorithm and implementation.

Three worked reductions

  1. studies, studying: an English stemmer may yield studi/study depending on the rule sequence; verb lemmatization aims at study. State the exact algorithm if exact outputs are asked.
  2. better: adjective-aware lemmatization can map it to good; naïve suffix removal cannot infer that irregular relation.
  3. saw: as a past-tense verb it can become see; as a cutting-tool noun it remains saw. POS and context matter.

Over-stemming merges forms that should remain distinct; under-stemming fails to merge related forms. Normalization can increase recall in search while losing useful distinctions, so it is a task-dependent choice.

Stem = rules se cut; lemma = dictionary form, meaning/POS ka dhyan.

","m2":false,"units":"2","marks":10,"sourcePages":[2],"historical":true},{"id":"q3b","title":"Q3b · Penn Treebank tagging","html":"
Question. Using Penn Treebank tags, tag every word and ignore punctuation: (1) It is a nice night. (2) This crap game is over a garage in Fifty-second Street. (3) Nobody ever takes the newspapers she sells. (4) I am sitting in Mindy’s restaurant putting on the gefilte fish, which is a dish I am very fond of. (5) What flights do you have from Milwaukee to Tampa? [10 marks]

Answer

Read the role of each word in its sentence. Split possessive ’s into POS; punctuation has no requested tag. The original supplies a Penn-tag annex.

Sentence 1

It/PRP is/VBZ a/DT nice/JJ night/NN

Sentence 2

This/DT crap/NN game/NN is/VBZ over/IN a/DT garage/NN in/IN Fifty-second/JJ Street/NNP

Sentence 3

Nobody/NN ever/RB takes/VBZ the/DT newspapers/NNS she/PRP sells/VBZ

Sentence 4

I/PRP am/VBP sitting/VBG in/IN Mindy/NNP ’s/POS restaurant/NN putting/VBG on/RP the/DT gefilte/JJ fish/NN which/WDT is/VBZ a/DT dish/NN I/PRP am/VBP very/RB fond/JJ of/IN

Sentence 5

What/WDT flights/NNS do/VBP you/PRP have/VB from/IN Milwaukee/NNP to/IN Tampa/NNP

TagMeaningWhy here
PRPPersonal pronounIt, I, she, you
DT / WDTDeterminer / wh-determinera/the versus What before flights; which relative modifier
VB/VBP/VBZ/VBGBase/non-3sg-present/3sg-present/gerund-participlehave after do; am/do; is/takes; sitting/putting
NN/NNS/NNPCommon singular/plural/proper noungame/newspapers/Milwaukee
IN/RPPreposition or subordinator / particlein/over/from; on in the idiomatic putting on reading
JJ/RB/POSAdjective/adverb/possessive endingnice/very/’s

Annotation ambiguities: “crap” is a noun adjunct under NN; an adjectival analysis can use JJ. “Fifty-second” may be NNP within an official street name instead of JJ; “gefilte” can be a noun adjunct in some taggers. Explain the chosen contextual reading rather than claiming all taggers must agree.

Source / convention note. Context-sensitive tags and possessive tokenization are explicitly stated; reasonable corpus guidelines can differ on noun adjuncts.

","m2":false,"units":"4","marks":10,"sourcePages":[2],"historical":true},{"id":"q4a","title":"Q4a · Mention-pair coreference","html":"
Question. Explain the mention-pair model with a detailed example. [10 marks]

Answer

A mention is a span referring to an entity. A mention-pair model classifies a pair (earlier candidate antecedent, later mention) as coreferent or not. Pairwise decisions are subsequently linked into entity clusters; a positive score alone does not ensure globally consistent clusters.

Detect mentions → Generate earlier candidate pairs → Extract pair features → Score same-entity relation → Select/link antecedents → Form entity clusters
  1. Text: “Rita bought a car. She likes it.” Detect Rita, a car, She, it.
  2. For She, compare (Rita,She) and (a car,She). Features include person/number compatibility, animate type, syntactic role, sentence distance and lexical agreement.
  3. Rita–She has compatible singular human features; car–She fails the ordinary inanimate reading. Select Rita.
  4. For it, compare earlier mentions: car–it is singular/inanimate and contextually fits likes. Cluster {Rita,She} and {a car,it}.
  5. Train a classifier on labelled mention pairs; at prediction, choose a sufficiently supported antecedent or leave a new entity. Avoid reflexive/binding violations and document leakage during evaluation.

Advantages: interpretable pair features and supervised learning. Limitations: quadratic candidate growth, error propagation from mention detection, non-transitive pair predictions, and weak long-range/world-knowledge reasoning. A greedy nearest-candidate choice is not the same as a learned mention-pair model.

","m2":false,"units":"6","marks":10,"sourcePages":[2],"historical":true},{"id":"q4b","title":"Q4b · Text classification pipeline","html":"
Question. Explain the text-classification pipeline with a diagram. [10 marks]

Answer

Text classification assigns a document to one or more predefined categories. Spam detection usually uses two classes; news classification may use several topics. In single-label multiclass classification, choose one of several categories. In multilabel classification, several labels may apply to the same document.

Define labels
→
Prepare labelled examples
→
Train representation + model
→
Evaluate on unseen text
→
Predict new labels

An exam-ready pipeline

StageWhat you doWhy it matters
Task definitionDecide what counts as spam or which news topics exist.Unclear labels create inconsistent training data.
Data collection and annotationCollect representative examples and assign correct labels.The model learns from these examples.
Split dataSeparate training, validation and test data.Test examples must not influence training choices.
PreprocessTokenize and normalise where suitable; retain meaningful negation.Reduce irrelevant variation without deleting signal.
RepresentUse BoW, TF-IDF or learned embeddings.Models need numerical features.
TrainFit Naive Bayes, logistic regression or another appropriate model.Learn how features predict categories.
EvaluateMeasure relevant precision, recall and F1 on held-out examples.Accuracy alone may hide poor minority-class performance.
Deploy and monitorApply the same preprocessing and monitor changing language.Incoming data can differ from old examples.

Naive Bayes is especially useful for hand calculations. With a bag-of-words representation, its class score is the prior multiplied by the conditional probability of each token; repeated tokens repeat the factor. The “naive” assumption is conditional independence of features given the class, not that words are independent in real language.

\"s(c)=P(c)\\prod_{w\\in
\"P(w\\mid

C(w,d) counts word w in document d. C(w,c) counts it in training documents belonging to class c. Nc is the total token count for class c. Add-one smoothing adds one to every vocabulary count and therefore adds |V| to the denominator. This is the same arithmetic taught for WSD, with document classes instead of senses.

Worked classification · two-token message

A spam classifier has vocabulary {win, prize, meeting}. Priors are P(spam)=P(ham)=0.5. Spam token counts are [3,2,0]; ham counts are [0,0,5]. With add-one smoothing, classify “win prize”.

Given

|V|=3; N(spam)=5; N(ham)=5; message has win once and prize once.

To find

Class scores and predicted label.

Formula and why it applies

\"s(c)=P(c)P(\\text{win}\\mid
\"P(w\\mid

The model is multinomial Naive Bayes; multiply the two token likelihoods and the class prior.

Solution

Step 1. Compute the smoothed denominators

\"5+3=8\"

Step 2. Compute spam likelihoods

\"P(\\text{win}\\mid
\"P(\\text{prize}\\mid

Step 3. Compute ham likelihoods

\"P(\\text{win}\\mid

Step 4. Multiply the spam score

\"s(S)=\\frac12\\times\\frac12\\times\\frac38=\\frac{3}{32}=0.09375\"

Step 5. Multiply the ham score

\"s(H)=\\frac12\\times\\frac18\\times\\frac18=\\frac{1}{128}=0.0078125\"

Step 6. Compare

0.09375 > 0.0078125. These are unnormalised scores; the label is chosen by the larger score.

Predicted label: spam.

Yaad rakho: Class prior × har word ki class-likelihood; sabse bada score wins.

","m2":false,"units":"7","marks":10,"sourcePages":[2],"historical":true},{"id":"q5a","title":"Q5a · Corpora and structure","html":"
Question. Define a corpus. Explain corpus types, characteristics and structure. [10 marks]

Answer

A corpus is a collection of language data assembled for study or computational use. Its plural is corpora. A corpus can contain books, news, reviews or conversations. It need not be labelled. Annotation adds linguistic information such as POS tags, entities, parse structures or sense labels.

StructureMeaningUse
PlainCollection without category labelsWord counts, language modelling
CategorizedDocuments grouped by genre/topicClassification
Overlapping categoriesOne document has several categoriesMulti-topic news
TemporalDocuments indexed by timeLanguage change
AnnotatedLinguistic labels attachedTagging, parsing, WSD
ParallelAligned texts in different languagesTranslation

A tagset is the allowed label inventory. A treebank is a corpus annotated with syntax trees. Metadata records source, date, genre, language, annotation method and limitations. A common benchmark supports comparison under shared data and evaluation conditions.

Raw example: “Riya works in Pune.” POS annotation: Riya/PROPN works/VERB in/ADP Pune/PROPN. NER annotation: Riya/PERSON, Pune/LOCATION. Each annotation answers a different task.

A good corpus is relevant to its domain, sufficiently representative, consistently annotated and appropriately split. More data alone do not repair duplicated examples, incorrect labels or a mismatch between training and real use.

Corpus=text collection. Annotation=text ke saath labels. Metadata=collection ki information.

ResourceContentsTypical use
Gutenberg selectionLiterary/book textsLong-form language and vocabulary
Web textInformal domain-specific textStudy usage beyond edited books
Chat corpusConversational postsDialogue, informal forms
BrownEnglish texts across genresGenre comparison
ReutersNews with topic labelsText classification; overlapping topics
InauguralSpeeches indexed by timeVocabulary change across years
WordlistWords/names/stopwordsLexical filtering
Pronouncing dictionaryWords and phonetic formsSpeech, stress, rhyme

Gutenberg versus web/chat: book language does not represent every spelling or conversational pattern in messages. Brown versus Reuters: Brown emphasizes genres; Reuters news can have overlapping topic labels. Inaugural addresses: dates provide a time dimension.

Corpus sizes depend on version, subset and tokenization. Explain their purpose and structure rather than assuming a historical count is a permanent property.

Gutenberg=books; web/chat=informal; Brown=genres; Reuters=news topics; Inaugural=time.

Corpus readers expose several views. raw returns a character string, words returns token occurrences, and sents returns a list of sentences where each sentence contains tokens.

Conceptual methodReturned viewIts length counts
fileids()Document IDsFiles
raw(fileid)Character stringCharacters
words(fileid)Token listToken occurrences
sents(fileid)Sentence listsSentences
categories()Genre/topic labelsCategories
tagged_words(fileid)Word–tag pairsAnnotated tokens

Question. For sentence lists [the,cat,sleeps,.] and [the,dog,runs,.], count all tokens, sentences and lowercase alphabetic word types.

Given

Punctuation is included in token count but excluded from word types.

To find

For sentence lists [the,cat,sleeps,.] and [the,dog,runs,.], count all tokens, sentences and lowercase alphabetic word types.

Formula and why it applies

Count every token occurrence, then distinct alphabetic forms. Different counting rules give different totals.

Step 1. Token count

4+4=8.

Step 2. Sentence count

Two outer lists → 2.

Step 3. Types

{the,cat,sleeps,dog,runs} → 5.

Answer. 8 tokens, 2 sentences, 5 word types.

For “Cats run. Dogs sleep.”, a reader may return six tokens including two periods, but only four alphabetic word tokens. State the convention before calculating TTR or probabilities.

raw=characters, words=tokens, sents=sentences. Teen alag counts.

","m2":false,"units":"3","marks":10,"sourcePages":[2],"historical":true},{"id":"q5b","title":"Q5b · Multinomial Naive Bayes","html":"
Question. Explain Naive Bayes for text classification. Using the four documents below, classify “Chinese Chinese Chinese Tokyo Japan”: Chinese Beijing Chinese=yes; Chinese Chinese Shanghai=yes; Chinese Macao=yes; Tokyo Japan Chinese=no. [10 marks]

Answer

Multinomial NB assumes token draws are conditionally independent given the class; it combines the class prior and class-specific word probabilities. Repeated Chinese contributes its likelihood three times.

Given

Training textClass
Chinese Beijing Chineseyes
Chinese Chinese Shanghaiyes
Chinese Macaoyes
Tokyo Japan Chineseno

Test: Chinese Chinese Chinese Tokyo Japan

To find

Classify the test using multinomial Naive Bayes.

Formula: which one and why

Assume add-one smoothing unless the question supplies probabilities. Keep repeated words as repeated factors. Use training vocabulary only; ignore out-of-vocabulary words unless an unknown-token model is defined.

\"\\mathrm{score}(c)=P(c)\\prod_{w\\in

Step 1. Vocabulary and class priors

V = {beijing, chinese, japan, macao, shanghai, tokyo}; |V|=6.

ClassDocumentsPrior
no11/4
yes33/4

Step 2. Count every training word within each class

ClassTotal tokensbeijingchinesejapanmacaoshanghaitokyo
no3011001
yes8150110

Step 3. Count the test words

WordCount
chinese3
tokyo1
japan1

OOV words: none

Step 4. Calculate likelihoods and multiply with the prior

ClassPriorIndividual factorsExact scoreDecimal score
no1/4chinese: (1+1)/(3+1×6) = 2/9, used 3 time(s); tokyo: (1+1)/(3+1×6) = 2/9, used 1 time(s); japan: (1+1)/(3+1×6) = 2/9, used 1 time(s)8/590490.000135480702467
yes3/4chinese: (5+1)/(8+1×6) = 3/7, used 3 time(s); tokyo: (0+1)/(8+1×6) = 1/14, used 1 time(s); japan: (0+1)/(8+1×6) = 1/14, used 1 time(s)81/2689120.000301213779973

Step 5. Compare and optionally normalize

Normalized scores: no = 0.310241388, yes = 0.689758612

Division by the same total does not change the winning class.

Final answer

Predicted class: yes.

Yaad rakho: Count → prior → smoothed word chances → repeated factors → compare.

","m2":false,"units":"7","marks":10,"sourcePages":[2,3],"historical":true},{"id":"q6a","title":"Q6a · GloVe","html":"
Question. Explain GloVe word embeddings. [10 marks]

Answer

GloVe (Global Vectors) learns word embeddings by fitting weighted relationships in a global word–context co-occurrence matrix. Word2Vec uses local prediction examples; GloVe uses aggregate counts directly in its objective.

\"X_i=\\sum_k

\"X_{ik}\" is how often context word \"k\" appears around target \"i\". \"X_i\" is the total of row \"i\". \"P_{ik}\" is the resulting context probability.

The ratio \"P(k\\mid helps identify how a probe context differs between two words. For targets ice and steam, a probe such as solid should favour ice; gas should favour steam; a probe shared similarly by both gives a ratio near one. These are data tendencies, not universal numerical identities.

Question. For context solid, ice has count 12 out of row total 20; steam has count 2 out of row total 40. Compute P(solid|ice), P(solid|steam) and their ratio.

Given

X(ice,solid)=12, X(ice)=20; X(steam,solid)=2, X(steam)=40.

To find

Two conditional probabilities and the ice/steam ratio.

Formula and why it applies

\"P(k\\mid, and \"R=P(k\\mid.

Step 1. Ice probability

\"12/20=0.6\".

Step 2. Steam probability

\"2/40=0.05\".

Step 3. Ratio

\"0.6/0.05=12\".

Answer. The probe solid is 12 times as probable in ice's context under these counts.

Question. For probe water, ice and steam both have row total 50. Counts are 10 and 10. Calculate the ratio.

Given

Both probabilities use a denominator of 50.

To find

The probability ratio.

Formula and why it applies

Use row-normalized counts before taking the ratio.

Step 1. Ice probability

\"10/50=0.2\".

Step 2. Steam probability

\"10/50=0.2\".

Step 3. Divide

\"0.2/0.2=1\".

Answer. Ratio 1: this probe does not distinguish the targets in this toy data.

Question. For probe gas, counts are 1/100 around ice and 20/100 around steam. Compute the ice-to-steam ratio.

Given

Conditional counts have the same row denominator.

To find

The ratio and interpretation.

Formula and why it applies

\"R=P(gas\\mid.

Step 1. Ice probability

\"1/100=0.01\".

Step 2. Steam probability

\"20/100=0.2\".

Step 3. Ratio

\"0.01/0.2=0.05\".

Answer. Ratio 0.05, so gas is much more strongly associated with steam.

Ratio >1: pehla word favoured. <1: doosra. ≈1: probe dono mein similar.

","m2":false,"units":"5","marks":10,"sourcePages":[3],"historical":true},{"id":"q6b","title":"Q6b · BoW and TF–IDF movie corpus","html":"
Question. Find Bag of Words and TF–IDF for: “This movie is very scary and long”; “This movie is not scary and is slow”; “This movie is spooky and good”. [10 marks]

Answer

Given

DocumentOriginal text
1This movie is very scary and long
2This movie is not scary and is slow
3This movie is spooky and good

Stopwords: none supplied

To find

BoW, IDF and TF–IDF

Formula: which one and why

Use lowercase alphabetic tokens and exactly the stated stoplist. If no stoplist is supplied, retain function words. Adopt raw-count TF and natural-log IDF; the paper does not prescribe another TF/log convention.

\"\\mathrm{TFIDF}(w,d)=c(w,d)\\ln\\frac{N}{\\mathrm{df}(w)}\"

A term shared by all N documents has IDF ln(1)=0. Counts and TF–IDF weights are different representations.

Step 1. Tokenize each document

DocumentTokens
1this, movie, is, very, scary, and, long
2this, movie, is, not, scary, and, is, slow
3this, movie, is, spooky, and, good

Step 2. Fix the vocabulary order

and, good, is, long, movie, not, scary, slow, spooky, this, very

Step 3. Count occurrences (Bag of Words)

Documentandgoodislongmovienotscaryslowspookythisvery
110111010011
210201111010
311101000110

Step 4. Calculate document frequency and IDF

TermdfN / dfln(N/df)
and310.000000000
good131.098612289
is310.000000000
long131.098612289
movie310.000000000
not131.098612289
scary23/20.405465108
slow131.098612289
spooky131.098612289
this310.000000000
very131.098612289

Step 5. Multiply each TF by its IDF

Documentandgoodislongmovienotscaryslowspookythisvery
10.0000000.0000000.0000001.0986120.0000000.0000000.4054650.0000000.0000000.0000001.098612
20.0000000.0000000.0000000.0000000.0000001.0986120.4054651.0986120.0000000.0000000.000000
30.0000001.0986120.0000000.0000000.0000000.0000000.0000000.0000001.0986120.0000000.000000

Final answer

The final table is the TF–IDF matrix in the declared vocabulary order. A different explicitly stated log base rescales all IDFs; length-normalized TF changes each document scale.

","m2":false,"units":"2","marks":10,"sourcePages":[3],"historical":true},{"id":"q7a","title":"Q7a · Information extraction, chunking and NER","html":"
Question. Explain information-extraction architecture with a diagram. Explain chunking and named-entity recognition with examples. [10 marks]

Answer

Information extraction (IE) converts unstructured language into structured information such as named entities, relations and events. A structured output has explicit fields or records that can be stored, searched or analysed. IE is broader than NER: finding two names does not yet determine how those entities are related.

The three levels of extracted information

LevelWhat it identifiesExample
Entity extraction / NERA span and its entity typeAsha → person; Orbit Labs → organisation
Relation extractionA labelled connection between entitiesworks-for(Asha, Orbit Labs)
Event extractionAn occurrence with roles and attributesJoining: employee=Asha, employer=Orbit Labs, date=2025

An entity is a participant such as a person or organisation. A relation links participants. An event describes something that happened, may happen or did not happen. An event role names a participant's function: buyer, seller, employee, employer, location or time. These labels must follow the chosen extraction schema.

Raw text
→
Sentence/token processing
→
Entity spans + types
→
Reference resolution
→
Relations and event roles
→
Normalize and store records

An extraction pipeline explained

StepOperationReason
Specify schemaChoose fields and allowed relations/events.A system needs to know what records it is building.
PreprocessSegment sentences and tokenize without losing useful structure.Locate expressions and boundaries.
Recognise entitiesFind spans and assign types.Names can contain several words.
Resolve referencesLink pronouns/descriptions to the right entities.Two mentions may denote one participant.
Extract links/eventsUse patterns, syntax or learned models.Co-occurrence alone does not identify a relationship.
Handle qualifiersPreserve negation, time and uncertainty.Not hired differs from hired.
NormalizeStandardize dates, names or quantities where justified.Equivalent forms should be comparable.
Store/evaluateSave records and compare them with annotated answers.Wrong links matter even if entity names are correct.

Worked application 1 · turn a sentence into a record

Question. Extract entities and a joining event from “Asha joined Orbit Labs in Mumbai on 3 May 2025.” Use fields person, organisation, location, event and date.

Given

One affirmative sentence; Orbit Labs is an organisation and Mumbai is a location.

To find

Entity spans/types and one structured joining record.

Method and why it applies

Find the complete entity spans, then assign their roles using the verb joined and its grammatical context.

Solution

Step 1. Mark entity spans

SpanType
AshaPerson
Orbit LabsOrganisation
MumbaiLocation
3 May 2025Date

Step 2. Identify the event trigger

Joined signals a joining event. The subject Asha is the joining person; Orbit Labs is the joined organisation.

Step 3. Attach contextual fields

In Mumbai supplies a location associated with the stated joining event. On 3 May 2025 supplies its date. Do not infer the company's permanent headquarters from this sentence alone.

Step 4. Normalize the date

The unambiguous date can be stored as 2025-05-03 while preserving its original text.

Step 5. Build the record

FieldValue
eventjoining
personAsha
organisationOrbit Labs
locationMumbai
date2025-05-03

Step 6. State the information limit

The sentence supports a joining event on that date. By itself it does not prove that Asha still works there indefinitely.

Answer. A joining record for Asha and Orbit Labs at the stated location/date, with complete entity spans preserved.

Worked application 2 · direction of a relation matters

Question. From “Nova acquired Luma for ₹50 crore in 2025”, extract an acquisition event with buyer, acquired organisation, amount and year.

Given

Nova and Luma are organisation names. The active verb is acquired; no seller is named.

To find

A correctly directed acquisition record.

Method and why it applies

Use grammatical roles to identify buyer and acquired organisation. Merely returning the unordered pair {Nova,Luma} loses the event direction.

Solution

Step 1. Recognise participant entities

Nova is the subject organisation; Luma is the object organisation.

Step 2. Locate the trigger

Acquired identifies an acquisition event.

Step 3. Assign roles

RoleEntity
Buyer/acquirerNova
Acquired organisationLuma

Step 4. Extract amount and time

Amount text=₹50 crore; year=2025. Currency and magnitude belong together; do not drop crore or replace it by lakh.

Step 5. Create the record

FieldValue
eventacquisition
buyerNova
acquired_organisationLuma
amountINR 50 crore
year2025
sellerNot stated

Step 6. Reject an unsupported inference

The sentence does not name the former owner/seller. Luma is the acquired organisation, not automatically the seller.

Answer. Acquisition(buyer=Nova, acquired organisation=Luma, amount=INR 50 crore, year=2025). Seller remains unspecified.

Worked application 3 · reference, negation and time

Question. Extract employment-related events from “Ravi joined Vega in 2021. He left the company in 2024. He did not join Orion.” Keep event type, person, organisation, date and polarity.

Given

He refers to Ravi; the company refers to Vega. The last sentence explicitly negates joining Orion.

To find

A structured event list without claiming an unsupported current employer.

Method and why it applies

Resolve references, identify each event separately and preserve whether it is affirmed or negated.

Solution

Step 1. Resolve the repeated mentions

MentionReferent
He in sentence 2Ravi
the companyVega
He in sentence 3Ravi

Step 2. Extract the first event

Joining; person=Ravi; organisation=Vega; date=2021; polarity=affirmed.

Step 3. Extract the second event

Leaving; person=Ravi; organisation=Vega; date=2024; polarity=affirmed.

Step 4. Extract the third statement

Joining; person=Ravi; organisation=Orion; date=not stated; polarity=negated. This must not create an affirmative employment relation with Orion.

Step 5. Write the event table

EventPersonOrganisationDatePolarity
joiningRaviVega2021Affirmed
leavingRaviVega2024Affirmed
joiningRaviOrionNot statedNegated

Step 6. Interpret current status cautiously

The text says that Ravi left Vega and did not join Orion. It supplies no other current employer. Do not fill that field by guessing.

Answer. Three qualified event statements, including a negated Orion joining. No current employer can be established from this text.

Approaches and their trade-offs

ApproachMechanismAdvantageLimitation
Dictionary/rule basedMatch known names, trigger words and patterns such as person joined organisation.Transparent and useful in a narrow domain.New names, paraphrases and complex syntax can escape patterns.
Supervised statisticalLearn entity/relation/event labels from annotated examples and features.Combines several cues consistently.Requires representative labels and careful feature design.
Neural/contextualUse contextual representations to classify spans and links or generate a structured record.Can capture paraphrases and broader context.May still invent or misassign fields; schema and factual checks remain necessary.

Applications include extracting company events from news, conditions/treatments from clinical text, skills from resumes and attributes from product descriptions. Extraction quality is measured against annotated records: an entity match can require the correct span and type; a relation match additionally requires the correct participants and direction. Precision measures how many extracted records are correct; recall measures how many gold records were found.

For a 10-mark answer, begin with the definition and structured output, distinguish NER/relation/event extraction, draw and explain the pipeline, work through a record with roles and qualifiers, compare approaches, then state evaluation and limitations. “Extract keywords” alone is too narrow: keywords do not necessarily represent a factual relation.

Yaad rakho: NER naam pakadta hai; relation kaun-kisse jodta hai; event kya hua batata hai. Negation aur date mat girao.

Chunking, also called shallow parsing, groups adjacent tagged words into meaningful non-overlapping units, commonly noun phrases. It usually identifies local phrase boundaries without producing the complete hierarchical structure of a sentence. Chinking removes specified tokens from an existing candidate chunk, splitting or trimming it.

The/DT small/JJ cat/NN sat/VBD on/IN the/DT mat/NN
→
NP: [The small cat]
→
Outside: sat on
→
NP: [the mat]

A POS tagger labels individual words. A chunker groups words into phrases. A full parser shows how phrases relate recursively to form the entire sentence. These are related tasks with different outputs.

OperationInputOutput
POS taggingThe small catThe/DT small/JJ cat/NN
NP chunkingTagged tokens[NP The small cat]
ChinkingA broad candidate chunkRemove verbs/prepositions, keep noun-phrase segments
Full parsingSentence + grammarA nested tree with sentence, noun phrase and verb phrase nodes

Rule-based chunking uses tag patterns. Statistical or learned chunkers infer boundaries from labeled examples. A useful NP rule is an optional determiner, followed by zero or more adjectives, followed by a noun. In NLTK-style notation: NP: {<DT>?<JJ>*<NN>}. Here ? means zero or one occurrence; * means zero or more; + means one or more when used. The braces mean “form a chunk around the matched sequence”.

Question. Apply the NP rule optional DT, zero or more JJ, then one NN to “The/DT quick/JJ brown/JJ fox/NN jumps/VBZ over/IN the/DT lazy/JJ dog/NN”.

Given

Tagged input is supplied. Rule: DT? JJ* NN.

To find

All noun-phrase chunks.

Formula and why it applies

NP → DT? JJ* NN

The rule groups a noun with its preceding determiner and adjectives; a verb or preposition does not match this NP pattern.

Solution

Step 1. Start at The

The matches optional DT; quick and brown both match JJ*; fox matches NN.

Step 2. Form the first NP

[NP The quick brown fox]

Step 3. Skip nonmatching tokens

jumps/VBZ and over/IN do not satisfy the NP pattern.

Step 4. Match the second NP

the is DT, lazy is JJ, dog is NN: [NP the lazy dog].

Final answer. [NP The quick brown fox] jumps over [NP the lazy dog].

Question. Using NP → DT? JJ* NN, identify the chunks in “bright/JJ stars/NN shine/VBP and/CC a/DT moon/NN glows/VBZ”. Use the supplied tags exactly.

Given

Input tags label stars as NN for this exercise. Rule requires optional DT, zero or more JJ, then NN.

To find

The matched NP spans.

Formula and why it applies

NP → DT? JJ* NN

The optional determiner may be absent and the adjective repetition may be empty.

Solution

Step 1. Match bright stars

No determiner is present, which DT? permits. bright matches JJ; stars matches NN.

Step 2. Inspect shine and and

VBP and CC fall outside the NP rule.

Step 3. Match a moon

a matches DT; no adjective is present; moon matches NN.

Final answer. Chunks are [NP bright stars] and [NP a moon]. With ordinary Penn tagging, stars is usually NNS; broaden the noun pattern to include NNS when that tag is used.

Question. Begin with one candidate chunk containing “The/DT cat/NN sat/VBD on/IN the/DT mat/NN”. Chink out verbs and prepositions. Show the remaining chunks.

Given

Initial candidate: [The cat sat on the mat]. Remove tags VBD and IN.

To find

The chunk segments after exclusion.

Formula and why it applies

Chinking removes matched tokens from the existing chunk.

Removing the middle verb/preposition region divides the original candidate into two remaining contiguous spans.

Solution

Step 1. Locate the tokens to remove

sat is VBD and on is IN.

Step 2. Remove those tokens from chunk membership

The remaining token groups are “The cat” and “the mat”. sat and on still exist in the sentence; they are simply outside these chunks.

Step 3. Write the final chunked sentence

[The cat] sat on [the mat].

Final answer. Two chunks remain: The cat; the mat.

Yaad rakho: Chunking = andar group karo; chinking = unwanted tokens ko group se bahar karo.

Named Entity Recognition (NER) identifies text spans referring to entities and assigns their types, such as PERSON, ORGANIZATION and LOCATION. Many practical schemas also include DATE, TIME, MONEY or domain-specific types. The exact allowed labels depend on the application's annotation scheme.

NER has two parts: boundary detection finds where the entity starts and ends; classification assigns the appropriate type. A multiword name should normally be one entity span: “New York” is a location span, not two unrelated location decisions.

Text: Mary joined Google in New York on Monday.
→
Mary → PERSON
→
Google → ORGANIZATION
→
New York → LOCATION
→
Monday → DATE

Not every noun is a named entity. In “a developer works at Google”, developer is an ordinary job description while Google is a named organization. A specialized HR system may define a JOB_TITLE category, but it must state that schema explicitly. NER identifies mentions; entity linking is the separate task of connecting a mention to a unique database identity.

TaskQuestion it answersExample
POSWhat grammatical role does this token have?Google/NNP
ChunkingWhich adjacent words form a phrase?[NP a skilled developer]
NERWhich span names an entity, and of what type?[Google] ORGANIZATION
Entity linkingWhich real entity does this mention denote?Connect Paris to the intended city or person record

Applications. NER structures search results and documents, extracts organizations/people from news, routes support tickets by product or organization, supports recommendation, and extracts specialized entities from healthcare or legal text. Context is essential: “Apple released a laptop” names an organization, while “she ate an apple” describes fruit.

Yaad rakho: NER mein do kaam: naam kahan se kahan tak, aur kis category ka.

","m2":false,"units":"4,7","marks":10,"sourcePages":[3],"historical":true},{"id":"q7b","title":"Q7b · Distributional hypothesis and cosine","html":"
Question. Justify “You shall know a word by the company it keeps.” Calculate cosine similarity of w₁=[0.2,0.1,0.3,0.4] and w₂=[0.3,0,0.2,0.5]. [10 marks]

Answer

Words occurring in similar contexts often have related meanings; vectors encode their context distributions. This is a statistical tendency, not proof that every similar-context pair is synonymous—antonyms can share contexts too.

Given

u=[0.2, 0.1, 0.3, 0.4]; v=[0.3, 0, 0.2, 0.5]

To find

Cosine similarity in the given coordinate order.

Formula: which one and why

\"\\cos(u,v)=\\frac{u\\cdot

Step 1. Multiply matching coordinates

0.2×0.3 + 0.1×0 + 0.3×0.2 + 0.4×0.5 = 0.32

Step 2. Calculate both squared lengths

u: 0.2² + 0.1² + 0.3² + 0.4² = 0.30000000000000004; v: 0.3² + 0² + 0.2² + 0.5² = 0.38

Step 3. Take square roots and divide

0.32 / (√0.30000000000000004 × √0.38) = 0.947758204

Final answer

Cosine=0.947758204. This measures direction, not a probability.

","m2":false,"units":"5","marks":10,"sourcePages":[3],"historical":true}]}],"mocks":[{"id":"timed-final-2025","title":"Timed practice · Solved final paper · December 2025","unit":"PYQ","scope":"TEE · actual previous paper · 100 marks · 180 minutes","sections":[{"id":"instructions","title":"Write the whole paper under exam conditions","html":"

Use this actual previous paper as a three-hour, 100-mark practice paper. Q1 is compulsory (20 marks); choose any four full questions from Q2–Q7 (20 marks each). Each original question and its complete answer follow. Read the question, write your answer in your notebook, then compare the working below it.

Budget roughly 36 minutes per attempted 20-mark question, including its subparts. Start each full question on a fresh page, as the supplied papers instruct. This is an old paper used for timed practice, not a forecast of the next TEE.

"},{"id":"final-2025-q1a","title":"Q1A · NLU and NLG","html":"
Question. Differentiate between Natural Language Understanding (NLU) and Natural Language Generation (NLG) in NLP with suitable examples. [5 marks]

Answer

Natural Language Understanding (NLU) interprets language to identify its meaning, intent and relevant information. Natural Language Generation (NLG) produces language that expresses information in a useful form.

BasisNLUNLG
DirectionLanguage → meaning or structured informationMeaning or structured information → language
Typical inputA user utterance, document or sentenceFacts, a database record, a prompt or a representation
Typical outputIntent, entities, sentiment or a semantic representationA sentence, explanation, summary or dialogue response
Main difficultyResolving ambiguity and contextChoosing accurate content and producing coherent, fluent wording

Example: a weather assistant

User: “Will it rain in Mumbai tomorrow?” NLU recognises a weather-query intent, the location Mumbai and the date tomorrow. After a weather service supplies the forecast, NLG produces “Rain is likely in Mumbai tomorrow; carry an umbrella.”

User language → NLU: identify request → Retrieve forecast → NLG: express answer

The two functions often work together in a chatbot. Understanding the request and wording the reply are separate tasks; fluent generation alone does not prove correct understanding.

Yaad rakho: NLU mein meaning nikalo; NLG mein meaning ko sentence banao.

","m2":false,"units":"1","marks":5},{"id":"final-2025-q1b","title":"Q1B · Chunking and chinking","html":"
Question. Explain the concepts of chunking and chinking in Natural Language Processing (NLP). Discuss how Part-of-Speech (POS) tagging supports these processes, and illustrate your explanation with a suitable example. [5 marks]

Answer

Chunking groups adjacent tagged words into shallow phrases, such as noun phrases, without building a complete sentence parse. Chinking removes a specified sequence of tags or words from a chunk, splitting a larger candidate chunk when required.

POS tags provide the pattern

A POS tag describes a grammatical category: DT = determiner, JJ = adjective, NN = singular noun, VBD = past-tense verb and IN = preposition. A noun-phrase rule can be written as NP: {<DT>?<JJ>*<NN>+}: an optional determiner, zero or more adjectives, followed by one or more nouns.

Worked example

WordThesmallcatsatonthemat
TagDTJJNNVBDINDTNN

Applying the NP rule gives [The small cat] sat on [the mat]. The first chunk contains DT–JJ–NN; the second contains DT–NN. “sat on” is not included because it does not match the noun-phrase rule.

For chinking, begin with one broad chunk: [The small cat sat on the mat]. Remove the VBD and IN sequence “sat on”, for example with }<VBD|IN>+{. The result is [The small cat] sat on [the mat].

POS tagging therefore supplies grammatical labels; chunking uses labels to select phrases, while chinking uses labels to exclude unwanted material. Incorrect tags can lead to incorrect chunks.

Yaad rakho: Chunking = phrase ko jodo; chinking = beech ka unwanted part nikalo.

","m2":true,"units":"4","marks":5},{"id":"final-2025-q1c","title":"Q1C · Hyponymy, hypernymy and WSD","html":"
Question. Define hyponymy and hypernymy with suitable linguistic examples. How can hypernymy be applied in Word Sense Disambiguation (WSD) to resolve the correct meaning of polysemous words? Provide an example? [5 marks]

Answer

Hyponymy is the “is-a-kind-of” relation from a specific concept to a broader concept. Hypernymy is the inverse relation from the broader concept to the specific one. “Rose” is a hyponym of “flower”; “flower” is a hypernym of “rose”.

rose / tulip → flower → plant

The arrow points towards a broader category. Hyponymy is not a part–whole relation: a wheel is part of a car, but it is not a kind of car.

Using the hierarchy for WSD

WSD selects the intended sense of a word from its context. Obtain candidate senses from a lexical resource such as WordNet, inspect their hypernym chains, and favour the chain most compatible with nearby words.

  1. Sentence: “The mouse clicked the icon.”
  2. Candidate 1: mouse → rodent → mammal → animal. Candidate 2: mouse → pointing device → computer peripheral.
  3. The context words “clicked” and “icon” belong to the computer-use domain. They support the device hierarchy, rather than the animal hierarchy.
  4. Select the computer-pointing-device sense. In “The mouse ate cheese,” the animal hierarchy is more compatible.

Hypernym evidence supplies semantic generalisation; it is a useful signal rather than a guarantee, because short or unusual contexts may still be ambiguous.

Yaad rakho: Hyponym specific hai; hypernym uska bada category hai.

","m2":true,"units":"5","marks":5},{"id":"final-2025-q1d","title":"Q1D · Pragmatics and discourse","html":"
Question. Compare pragmatic and discourse analysis with the help of appropriate examples. [5 marks]

Answer

Pragmatic analysis interprets an utterance using the situation, speaker intention, shared knowledge and conversational expectations. Discourse analysis studies how multiple utterances form a connected text or conversation.

BasisPragmatic analysisDiscourse analysis
FocusIntended meaning in contextRelationships and coherence across utterances
Useful informationSpeaker, listener, place, time and common knowledgePrevious sentences, entities, topics and rhetorical relations
Typical tasksInterpret requests, implicatures and deixisResolve references and analyse cohesion, coherence and discourse structure
Example“It is cold here” may be a request to close a window“Riya bought a book. She read it.” links She to Riya and it to book

In “Can you pass the salt?”, literal syntax asks about ability; pragmatic interpretation usually identifies a polite request. In “The road flooded. Therefore, the school closed,” discourse analysis identifies a cause–result link between the sentences.

The levels interact: resolving “she” across a conversation may require both earlier text and real-world context. They are complementary rather than mutually exclusive.

Yaad rakho: Pragmatics: bolne ka intention. Discourse: sentences ka connection.

","m2":false,"units":"6","marks":5},{"id":"final-2025-q2a","title":"Q2A · Morpheme, lexeme and NER","html":"
Question. i. Differentiate between Morpheme and Lexeme with examples.
ii. Discuss the significance of NER in three different NLP applications. [10 marks]

Answer

i. Morpheme and lexeme

A morpheme is the smallest linguistic unit that contributes meaning or grammatical function. A lexeme is an abstract vocabulary item representing a family of inflected word forms.

BasisMorphemeLexeme
UnitA meaningful component of a wordAn abstract dictionary vocabulary item
Examplesun-, happy, -ness; cat and plural -sRUN covers run, runs, ran and running
IndependenceFree morphemes can stand alone; bound morphemes cannotA lexeme is realised through one or more word forms
NLP relevanceMorphological segmentation, stemming and analysisLemmatization and lexical meaning

“Cats” contains two morphemes: cat + plural -s. “cat” and “cats” are forms of the lexeme CAT. “Unhappiness” can be segmented into un + happy + ness, although its spelling changes at the boundary. A morpheme and a syllable are different: syllables concern pronunciation, while morphemes concern meaning or grammar.

ii. Named Entity Recognition in applications

NER locates spans that name entities and assigns types such as PERSON, ORGANIZATION, LOCATION, DATE or domain-specific categories.

ApplicationEntities and exampleWhy NER matters
News information extraction“Asha joined Infosys in Pune.” → Asha/PERSON, Infosys/ORG, Pune/LOCATIONCreates searchable people–organisation–location records and supports later relation extraction
Healthcare document processingFind disease, drug and dosage spans in a clinical noteSupports record indexing and medication extraction; specialist labels and validation are required
Customer support and commerceExtract product names, dates and order identifiers from “My Pixel arrived damaged on Monday.”Routes complaints, links the correct product and fills structured support fields

NER answers “Which entity spans occur here, and what type are they?” It does not by itself establish every relation between those entities or resolve all names to unique real-world identities.

Yaad rakho: Morpheme = word ka meaningful piece; lexeme = word family. NER = naam dhoondho aur type do.

","m2":false,"units":"1,4","marks":10},{"id":"final-2025-q2b","title":"Q2B · Laplace-smoothed bigram probability","html":"
Question. Consider the following training corpus:
1. artists paint murals in city
2. musicians play songs in city
3. painters paint murals on walls
4. signs are near city
Using a bigram language model with Laplace smoothing, evaluate the probability of the test sentence: “artists paint murals near city”. Assume the <s> and </s> for each sentence. [10 marks]

Answer

Given

Test: artists paint murals near city. Add one start token <s> and one end token </s> to every training and test sentence.

To find

The Laplace-smoothed bigram probability of the complete test sentence, including its end token.

Formula: which one and why

\"P_{+1}(w_i\\mid

C(previous,current) counts that adjacent pair in the training corpus. C(previous) counts its occurrences as a history. V is the vocabulary size. Add-one smoothing is necessary because an unseen pair must not force the whole sentence probability to zero.

\"P(w_1,\\ldots,w_n,\\langle/s\\rangle\\mid\\langle

Solution

Step 1. Fix the token and vocabulary convention

Lowercase words and remove the final punctuation. The supplied course key counts both boundary tokens in V. Following that convention: are, artists, city, in, murals, musicians, near, on, paint, painters, play, signs, songs, walls, <s>, </s> gives V = 16. The start marker supplies history; it is not an extra probability factor before the first word.

Step 2. List every required transition

The complete test chain is <s> → artists → paint → murals → near → city → </s>.

Step 3. Count first, then add one

Required bigramHistory countPair countSmoothed probability
<s> → artists41(1+1)/(4+16) = 1/10
artists → paint11(1+1)/(1+16) = 2/17
paint → murals22(2+1)/(2+16) = 1/6
murals → near20(0+1)/(2+16) = 1/18
near → city11(1+1)/(1+16) = 2/17
city → </s>33(3+1)/(3+16) = 4/19

Count only adjacent words, never words that merely appear in the same sentence. The unseen pairs have count zero, so their numerator becomes one.

Step 4. Multiply all six probabilities

\"P=\\frac{2}{20}\\times\\frac{2}{17}\\times\\frac{3}{18}\\times\\frac{1}{18}\\times\\frac{2}{17}\\times\\frac{4}{19}\"

Start with subtotal 1. At each row, multiply the previous subtotal by exactly one next factor. Simplify the fraction after multiplication, but do not round it.

Factor being includedPrevious subtotal × next factor = new subtotal
1. <s> → artists\"1\\times\\frac{1}{10}=\\frac{1}{10}\"
2. artists → paint\"\\frac{1}{10}\\times\\frac{2}{17}=\\frac{1}{85}\"
3. paint → murals\"\\frac{1}{85}\\times\\frac{1}{6}=\\frac{1}{510}\"
4. murals → near\"\\frac{1}{510}\\times\\frac{1}{18}=\\frac{1}{9180}\"
5. near → city\"\\frac{1}{9180}\\times\\frac{2}{17}=\\frac{1}{78030}\"
6. city → </s>\"\\frac{1}{78030}\\times\\frac{4}{19}=\\frac{2}{741285}\"

The last subtotal is the complete product. Only now convert the fraction to a decimal.

\"P=\\frac{2}{741285}\\approx

Step 5. Interpret the result

This is a joint sentence probability, so a small value is normal. Do not add transition probabilities; each word must occur in sequence, so the chain multiplies them.

Final answer

P ≈ 2.69801763 × 10-6 with the stated course vocabulary convention.

Yaad rakho: Pair count mein 1 jodo, previous-word count mein V; phir saare fractions multiply karo.

Convention note: Some implementations exclude <s> from the predicted vocabulary. Their V is one smaller and their numerical probability differs. In an exam, state V explicitly and follow the convention specified in the question or class solution.

","m2":true,"units":"3","marks":10},{"id":"final-2025-q3a","title":"Q3A · Ambiguity and bottom-up parsing","html":"
Question. i. Explain the various types of ambiguities encountered in Natural Language Processing (NLP). Illustrate each type with suitable examples.
ii. Compare top-down and bottom-up parsing and apply bottom-up syntax parsing approach to the sentence “A cat chased the mouse” and draw a parse tree using the grammar rules:
S → NP VP; NP → Pronoun | Proper-Noun | Det Nominal; Nominal → Noun | Nominal Noun | Nominal PP; VP → Verb | Verb NP | VP PP; PP → Prep NP.
Lexicon: Det → {A, the}; Noun → {cat, mouse}; Verb → {chased}. [10 marks]

Answer

i. Types of ambiguity

Ambiguity exists when the same language expression permits more than one interpretation. The correct interpretation usually requires grammatical, semantic or contextual evidence.

TypeMeaningExample and readings
LexicalOne word has multiple meanings“She visited the bank”: financial institution or riverbank
SyntacticThe sentence permits different structural attachments“I saw the man with a telescope”: I used the telescope, or the man had it
Semantic / scopeA structured sentence permits different interpretations“Every student read a book”: one shared book, or possibly a different book per student
ReferentialA referring expression has more than one candidate referent“Asha told Riya that she won”: she could refer to Asha or Riya
PragmaticIntention depends on the situation“Can you open the door?” can ask about ability or request an action

ii. Parsing approaches

AspectTop-downBottom-up
StartThe start symbol SThe input words
OperationExpand a nonterminal using grammar productionsCombine words and constituents using productions in reverse
SuccessLeaves match the complete inputAll input is reduced to S
Possible difficultyNaive recursive descent can loop on left recursion or need backtrackingA naive shift-reduce parser may face shift/reduce conflicts

Given

The five tokens are A / cat / chased / the / mouse. The supplied grammar permits determiner + nominal noun phrases and a verb + noun phrase verb phrase.

To find

A bottom-up sequence of reductions and a parse tree whose root is S and whose leaves preserve the input order.

Rule: which one and why

A production A → B C permits B C to be reduced to A. Bottom-up parsing begins with the words; lexical rules assign their categories, and phrase rules combine those categories.

Step 1. Assign lexical categories

A → Det; cat → Noun; chased → Verb; the → Det; mouse → Noun.

Step 2. Build the first noun phrase

cat is Noun. Reduce Noun to Nominal using Nominal → Noun. Then reduce Det Nominal, representing “A cat”, to NP.

Step 3. Build the object noun phrase

mouse is Noun. Reduce it to Nominal. Reduce “the” + “mouse”, now Det Nominal, to NP.

Step 4. Build the verb phrase

“chased” is Verb and “the mouse” is NP. Reduce Verb NP to VP.

Step 5. Build the sentence

The remaining constituents are NP (“A cat”) and VP (“chased the mouse”). Apply S → NP VP in reverse to obtain S. Every input token is used exactly once.

SNPVPDetNominalVerbNPNounDetNominalNounAcatchasedthemouse
Bottom-up parse tree for A cat chased the mouse.

Final answer: [S [NP [Det A] [Nominal [Noun cat]]] [VP [Verb chased] [NP [Det the] [Nominal [Noun mouse]]]]].

Yaad rakho: Bottom-up: words se phrases, phrases se S. Top-down: S se words.

","m2":false,"units":"1,4","marks":10},{"id":"final-2025-q3b","title":"Q3B · Co-occurrence matrix and cosine","html":"
Question. Consider the given paragraph: The teacher wrote on the board. The students listened to the teacher. The board was full of notes. The student asked a question. The teacher answered from the book.
Target words: teacher, student, board, book.
Context words: wrote, listened, notes, question.
i. Construct a co-occurrence matrix for the target and context words. Take the window size as one whole sentence.
ii. Evaluate the similarity between the vectors of the target words “teacher-student” and “teacher-board” using the cosine similarity.
iii. Plot the target word vectors by considering only two dimensions: “wrote” and “notes”. [10 marks]

Answer

Given

Five sentences, four target rows and four context columns. Treat students and student as the same target lemma, and count each target–context pair once when both occur in a sentence.

To find

The 4 × 4 sentence-window co-occurrence matrix, two cosine similarities and the requested 2-D plot.

Formula: which one and why

\"M_{ij}=\\sum_{s=1}^{5}\\mathbf{1}[\\text{target}_i\\text{

The indicator is 1 when both words appear in that sentence and 0 otherwise. Each matrix row is the target word’s vector: its list of four context counts.

\"\\cos(\\mathbf

The dot product multiplies corresponding coordinates and adds them. The length is the square root of the sum of squared coordinates. Cosine compares directions while allowing different vector lengths.

Solution

Step 1. Record the contribution of each sentence

SentenceTargets presentRelevant contextCounts added
1teacher, boardwroteteacher–wrote +1; board–wrote +1
2student, teacherlistenedstudent–listened +1; teacher–listened +1
3boardnotesboard–notes +1
4studentquestionstudent–question +1
5teacher, booknoneNo requested context count

Step 2. Construct every requested column

Targetwrotelistenednotesquestion
teacher1100
student0101
board1010
book0000

Step 3. Calculate teacher–student

\"\\mathbf
\"\\|\\mathbf
\"\\|\\mathbf
\"\\cos(\\mathbf

Step 4. Calculate teacher–board

\"\\mathbf
\"\\|\\mathbf
\"\\cos(\\mathbf

Step 5. Select only the two requested plot coordinates

Wordx = wrotey = notes
teacher10
student00
board11
book00
001122teacherstudent, bookboardwrotenotes
The plot uses only the requested two coordinates; cosine uses all context columns.

Final answer

Both requested cosine similarities are 0.5. Student and book overlap at the origin of the 2-D projection, but their complete four-dimensional vectors differ. The book row is zero, so a cosine involving book would be undefined without an explicit convention.

Yaad rakho: Har sentence mein target aur context saath? Ek count. Cosine ke liye poori row lo, plot ke liye sirf do columns.

","m2":false,"units":"5","marks":10},{"id":"final-2025-q4a","title":"Q4A · Three preprocessing techniques","html":"
Question. Explain any three basic text preprocessing techniques used in Natural Language Processing (NLP). Discuss the purpose of each technique and illustrate them with suitable examples. [10 marks]

Answer

Text preprocessing transforms raw text into a representation suitable for a chosen NLP task. The best steps depend on the task; removing information indiscriminately can reduce performance.

1. Tokenization

Tokenization divides text into units called tokens, such as words, subwords or sentences. These units are required for counting words and building many model inputs. “The cats are sleeping.” can become [The, cats, are, sleeping, .] under a punctuation-preserving word tokenizer. A different tokenizer may split contractions or punctuation differently, so the convention must be consistent during training and prediction.

2. Lowercasing

Lowercasing maps uppercase letters to lowercase so that variants such as “Book”, “BOOK” and “book” can share one feature. This reduces vocabulary duplication in a case-insensitive search system. However, “US” and “us” are different, and capitals help NER recognise names. Therefore lowercasing should be chosen according to the application.

3. Lemmatization

Lemmatization maps an inflected word to its dictionary base form, often using its POS and context. Examples: cats → cat; running/VBG → run; better/JJR → good. It allows a search for “run” to match relevant inflections. A stemmer instead uses simpler truncation rules and can produce a non-word stem; lemmatization is linguistically informed and can need more processing.

Combined example

StageRepresentation
RawThe cats are running.
TokenizedThe | cats | are | running | .
Lowercasedthe | cats | are | running | .
Lemmatized with correct POSthe | cat | be | run | .

These steps do not automatically remove stop words or punctuation. Those are separate choices. For sentiment analysis, preserve negation such as “not”; for syntax parsing, function words and word order are often essential.

Yaad rakho: Preprocess task dekh ke karo: har word ko blindly delete mat karo.

","m2":false,"units":"2","marks":10},{"id":"final-2025-q4b","title":"Q4B · HMM emission, transition and sequence","html":"
Question. Consider the following POS tagged corpus. Apply Hidden Markov Model (HMM) approach to:
1. Cat/Noun chase/Verb mice/Noun
2. The/Det dog/Noun runs/Verb
3. A/Det cat/Noun eats/Verb cheese/Noun
i. Construct the emission probability matrix.
ii. Construct the transmission probability matrix.
iii. Calculate the probability of sequence of events: The/Det cat/Noun eats/Verb mice/Noun.
Note: Include start and end characters for all the sentences. [10 marks]

Answer

Given

SentenceTagged sequence
1cat/Noun chase/Verb mice/Noun
2the/Det dog/Noun runs/Verb
3a/Det cat/Noun eats/Verb cheese/Noun

Test words: the cat eats mice. Given tag sequence: Det → Noun → Verb → Noun. Include Start and End for all sentences. Lowercase the corpus so sentence-initial Cat and cat are one word.

To find

Emission probabilities, transition probabilities and the joint probability of the specified tag-and-word sequence.

Formula: which one and why

\"P(w\\mid

An emission probability asks: among tokens carrying this tag t, what fraction are the word w?

\"P(t_j\\mid

A transition probability asks: among transitions leaving tag tᵢ, what fraction go to tⱼ? Include End in the outgoing counts. The paper says “transmission”; the standard HMM term is transition.

\"P(\\mathbf

Solution

Step 1. Count tokens for each tag

Det has 2 tokens; Noun has 5 tokens; Verb has 3 tokens. The repeated noun cat occurs twice with Noun; every other displayed noun occurs once. The three verb types each occur once.

Step 2. Build the emission matrix

Tag / wordacatchasecheesedogeatsmicerunsthe
Det1/200000001/2
Noun02/501/51/501/500
Verb001/3001/301/30

Each row sums to one. A zero means no occurrence under that tag in this unsmoothed training corpus; no add-one smoothing was requested.

Step 3. Write the three tag chains

Start → Noun → Verb → Noun → End
Start → Det → Noun → Verb → End
Start → Det → Noun → Verb → Noun → End

Step 4. Count outgoing transitions

FromTo DetTo NounTo VerbTo EndOutgoing total
Start21003
Det02002
Noun00325
Verb02013

Step 5. Divide each row by its outgoing total

From / ToDetNounVerbEnd
Start2/31/300
Det0100
Noun003/52/5
Verb02/301/3

End is terminal, so it has no estimated outgoing row. The four nonterminal rows each sum to one.

Step 6. Choose the factors for the given sequence

Event in orderProbability
Start → Det2/3
Det emits the1/2
Det → Noun1
Noun emits cat2/5
Noun → Verb3/5
Verb emits eats1/3
Verb → Noun2/3
Noun emits mice1/5
Noun → End2/5

Step 7. Multiply, including the end transition

\"P=\\frac23\\times\\frac12\\times1\\times\\frac25\\times\\frac35\\times\\frac13\\times\\frac23\\times\\frac15\\times\\frac25\"

Start with subtotal 1. At each row, multiply the previous subtotal by exactly one next factor. Simplify the fraction after multiplication, but do not round it.

Factor being includedPrevious subtotal × next factor = new subtotal
1. Start → Det\"1\\times\\frac{2}{3}=\\frac{2}{3}\"
2. Det emits the\"\\frac{2}{3}\\times\\frac{1}{2}=\\frac{1}{3}\"
3. Det → Noun\"\\frac{1}{3}\\times1=\\frac{1}{3}\"
4. Noun emits cat\"\\frac{1}{3}\\times\\frac{2}{5}=\\frac{2}{15}\"
5. Noun → Verb\"\\frac{2}{15}\\times\\frac{3}{5}=\\frac{2}{25}\"
6. Verb emits eats\"\\frac{2}{25}\\times\\frac{1}{3}=\\frac{2}{75}\"
7. Verb → Noun\"\\frac{2}{75}\\times\\frac{2}{3}=\\frac{4}{225}\"
8. Noun emits mice\"\\frac{4}{225}\\times\\frac{1}{5}=\\frac{4}{1125}\"
9. Noun → End\"\\frac{4}{1125}\\times\\frac{2}{5}=\\frac{8}{5625}\"

The last subtotal is the complete product. Only now convert the fraction to a decimal.

\"P=\\frac8{5625}\\approx0.0014222222\"

Final answer

Joint sequence probability = 8/5625 ≈ 0.0014222222. This is the probability of the specified words and tags together. It is not the probability of the word sequence summed over every possible tag path.

Yaad rakho: Tag se tag = transition; tag se word = emission. Dono alternate multiply, end ko mat bhoolo.

Boundary reminder: The initial factor is P(Det | Start) = 2/3, based on three sentence starts, and the final factor is P(End | Noun) = 2/5. The overall fraction of Det tokens, 2/10, is not the start probability. Using it and omitting End, as in the handwritten key’s last line, computes a different quantity.

","m2":true,"units":"4","marks":10},{"id":"final-2025-q5a","title":"Q5A · TF–IDF matrix","html":"
Question. For the following two sentences:
Sentence A: “The student read a science book in the library”
Sentence B: “The teacher read a history book in the classroom”
1. Construct the TF-IDF matrix for these sentences after removing the stop words {The, a, in}.
2. Identify and comment on the important terms in each sentence based on their TF-IDF scores. [10 marks]

Answer

Given

Two documents, N = 2. Remove the listed stop words case-insensitively, so both “The” and “the” are removed. Use raw count TF and natural-log IDF, the convention used in the course key.

To find

The term-by-document TF–IDF matrix and the terms that distinguish each sentence.

Formula: which one and why

\"\\mathrm{TF}(t,d)=C(t,d),\\qquad
\"\\mathrm{TFIDF}(t,d)=\\mathrm{TF}(t,d)\\,\\mathrm{IDF}(t)\"

C(t,d) is the number of occurrences of term t in document d. df(t) is the number of documents containing t, not its total token count. N is the number of documents. IDF downweights terms shared by many documents.

Solution

Step 1. Clean and list the tokens

A: student, read, science, book, library.
B: teacher, read, history, book, classroom. Each retained token appears once in its sentence.

Step 2. Fix one common vocabulary order

student, read, science, book, library, teacher, history, classroom. Each row must refer to the same term in both documents.

Step 3. Count document frequency and calculate IDF

Term typedfIDF calculation
read, book2ln(2/2) = ln(1) = 0
Every other term1ln(2/1) = ln(2) ≈ 0.693147

A term absent from a document has TF = 0. A present term has TF = 1 here.

Step 4. Multiply TF by IDF

TermTF in ATF in BIDFTF–IDF ATF–IDF B
student100.6931470.6931470
read11000
science100.6931470.6931470
book11000
library100.6931470.6931470
teacher010.69314700.693147
history010.69314700.693147
classroom010.69314700.693147

Step 5. Interpret the weights

A is distinguished by student, science and library. B is distinguished by teacher, history and classroom. “read” and “book” have zero IDF in this tiny unsmoothed two-document collection because they appear in every document; that does not mean those words are meaningless in English.

Final answer

All six document-specific terms receive 0.693147 in their own document and 0 in the other. Shared read and book receive 0 in both.

Yaad rakho: TF: is document mein kitni baar? IDF: kitne documents mein common? Rare-across-documents term ko zyada weight.

If normalized TF is explicitly requested, divide each present-token count by five; each nonzero weight becomes 0.693147/5 = 0.1386294. Do not silently switch conventions.

","m2":false,"units":"2","marks":10},{"id":"final-2025-q5b","title":"Q5B · Naïve Bayes word-sense classification","html":"
Question. Consider the sentence: “The bat hit the ball hard on the cricket field.” Two possible senses of “bat”: Sense 1: Bat (Sports equipment – cricket/baseball); Sense 2: Bat (Nocturnal flying mammal).
Sentence IDContext wordsSense
1ball, cricket, score, stadiumSports
2night, cave, wings, flyAnimal
3player, run, match, batSports
4nocturnal, vampire, insectAnimal
5team, field, bowl, batSports
6dark, flying, cave, nightAnimal
a) Using the Naïve Bayes algorithm, classify the sense of “bat” in the given sentence. Apply Laplace Smoothing wherever applicable.
b) Discuss the significance of Laplace Smoothing in Naïve Bayes Classification. [10 marks]

Answer

Given

There are three Sports examples and three Animal examples. Treat training context lists as bag-of-words token counts. Exclude the ambiguous target “bat” from the test context. Use the fixed training vocabulary; ignore out-of-vocabulary test words hit and hard. The usable test features are ball, cricket and field.

To find

The posterior-score comparison for Sports and Animal, and the role of smoothing.

Formula: which one and why

\"\\mathrm{score}(s)=P(s)\\prod_{w\\in\\mathrm{context}}P(w\\mid
\"P(w\\mid

P(s) is the class prior, C(w,s) counts occurrences of context word w in class s, Nₛ is the total number of context tokens in that class and V is the shared vocabulary size. Conditional independence lets us multiply word probabilities. The common denominator P(context) is unnecessary when only comparing classes.

Solution

Step 1. Calculate the priors

\"P(S)=\\frac36=0.5,\\qquad

Step 2. Count class tokens and vocabulary

Sports has 4 + 4 + 4 = 12 tokens. Animal has 4 + 3 + 4 = 11 tokens. The shared training vocabulary has 20 distinct words: ball, cricket, score, stadium, player, run, match, bat, team, field, bowl, night, cave, wings, fly, nocturnal, vampire, insect, dark, flying. “bat”, “night” and “cave” repeat but count only once each towards V.

Step 3. Look up each usable context word

FeatureSports countAnimal countP(word | Sports)P(word | Animal)
ball10(1+1)/(12+20) = 2/32(0+1)/(11+20) = 1/31
cricket102/321/31
field102/321/31

Step 4. Multiply the Sports score

\"\\mathrm{score}(S)=0.5\\times\\frac2{32}\\times\\frac2{32}\\times\\frac2{32}=0.0001220703125\"

Step 5. Multiply the Animal score

\"\\mathrm{score}(A)=0.5\\times\\frac1{31}\\times\\frac1{31}\\times\\frac1{31}\\approx0.00001678359236\"

Step 6. Compare the scores

0.0001220703125 is greater than 0.00001678359236. Therefore select Sports. Optional normalization gives P(Sports | retained features) ≈ 0.87913; normalization is not needed for the decision.

Final answer

The intended sense is bat as sports equipment. Ball, cricket and field favour that sense.

Yaad rakho: Pehle class prior, phir har context-word likelihood multiply; bigger score wali sense choose karo.

Why Laplace smoothing matters

Without smoothing, all three test features have zero count in Animal. One zero factor makes the entire class score zero, even if other evidence could support that class. Add-one smoothing assigns nonzero probability to every word in the defined vocabulary and redistributes probability mass. It does not mean every unseen word outside the vocabulary can be inserted without changing the vocabulary model.

If following the supplied paper key exactly

The key also multiplies probabilities for hit and hard using the same denominators 32 and 31. That convention produces Sports = 1.19209290 × 10⁻⁷ and Animal ≈ 1.74647163 × 10⁻⁸, again selecting Sports. Its treatment of out-of-vocabulary words is not a normalized fixed-vocabulary model. State your feature convention clearly; the complete calculation above uses a consistent fixed training vocabulary.

","m2":true,"units":"5","marks":10},{"id":"final-2025-q6a","title":"Q6A · Coreference, pronouns and Word2Vec","html":"
Question. i. Differentiate between co-reference resolution and pronoun resolution with suitable examples.
ii. Explain the working principle of the Continuous Bag of Words (CBOW) model and Skipgram model in Word2Vec. Illustrate how it predicts the target word from its surrounding context. [10 marks]

Answer

i. Coreference and pronoun resolution

Coreference resolution groups expressions that refer to the same entity, including names, descriptions and pronouns. Pronoun resolution identifies the referent of a pronoun; it is narrower in scope.

Expression / taskExample
Coreference cluster“Dr Rao arrived. The surgeon opened her bag.” → {Dr Rao, the surgeon, her}
Pronoun resolutionFor the same text, resolve “her” to Dr Rao.
DifferenceResolving the description “the surgeon” is coreference work even when no pronoun is involved.

ii. Word2Vec

Word2Vec learns dense word embeddings by predicting words from local context. A dense embedding is a short learned list of real numbers for a word; words used in similar contexts can obtain related vectors. CBOW and Skip-gram use opposite prediction directions.

FeatureCBOWSkip-gram
InputSurrounding context wordsOne centre word
Prediction targetThe centre wordSurrounding context words
Training example[the, cat, on, the] → satsat → the; sat → cat; sat → on; sat → the
RepresentationCombine context embeddings, often by averagingUse the centre-word embedding for each context prediction

Example sentence: “The cat sat on the mat.” With a radius-two window around sat, the context is [The, cat, on, the]. In CBOW, obtain the four context embeddings, average or sum them, calculate scores for vocabulary words, and train the model to assign high probability to sat. In Skip-gram, use sat as input and create four centre–context training pairs.

CBOW: nearby words → combine embeddings → predict centre word
Skip-gram: centre word → its embedding → predict nearby words

Training adjusts embedding parameters to improve prediction; the final word vectors are the learned representation. Thus the question’s phrase “predicts the target word from surrounding context” describes CBOW specifically; Skip-gram reverses that direction.

Yaad rakho: CBOW: neighbours se centre. Skip-gram: centre se neighbours.

","m2":false,"units":"5,6","marks":10},{"id":"final-2025-q6b","title":"Q6B · Hobbs: Ravi and Suresh","html":"
Question. i. Explain Hobb’s Algorithm. Discuss the steps with help of example and state its limitation.
ii. Sentence: Ravi met Suresh at the station after he arrived from Delhi. Using Hobbs’ algorithm, determine the most likely antecedent of the pronoun “he”. Show the steps of traversal according to the algorithm. [10 marks]

Answer

i. Method

Hobbs’ algorithm is a syntax-based pronoun-resolution procedure that searches constituency parse trees in a specified order. It proposes noun phrases as antecedents and filters incompatible candidates, for example by number and gender agreement.

Read the tree labels first: NP means noun phrase; S means sentence/clause; VP means verb phrase. X is the current NP or S search node, and p is the upward path used to reach it. Breadth-first means shallower nodes before deeper nodes; left-to-right fixes the order at the same depth. An N-bar (N′) node is the noun-centred part of an NP in a parse using that label.

Complete ordered procedure

  1. Locate the pronoun. Start at the NP immediately dominating the pronoun: the smallest NP containing it.
  2. Make the first climb. Move upward to the first NP or S above the starting NP. Call it X and record the path p.
  3. Search initially to the left. Traverse branches below X to the left of p, breadth-first and left-to-right. Propose an NP only if another NP or S lies between that candidate and X. Check agreement and other applicable constraints for each proposed candidate; accept the first eligible compatible one.
  4. Check the highest sentence. If X is the highest S and no candidate has been accepted, search previous sentence trees, most recent first. In each, traverse breadth-first and left-to-right and propose NPs. Accept the first eligible compatible candidate; if the available context contains none, report no resolved antecedent.
  5. Otherwise climb again. Move from X to the next NP or S above it. Make that node the new X and record the new upward path p.
  6. Test a higher NP. If the new X is an NP, propose X only when p did not pass through its nominal N-bar child. Apply the same compatibility checks. Use the labels in the supplied parse; do not invent a missing N-bar branch.
  7. Search left again. Traverse the new X’s branches to the left of p, breadth-first and left-to-right, proposing NPs. The extra intervening-NP/S requirement in Step 3 belongs to the initial search.
  8. Search permitted right branches. If X is an S, traverse branches to the right of p, breadth-first and left-to-right, but do not descend below any NP or S encountered. Propose the encountered eligible NPs and test their compatibility.
  9. Repeat or finish. Return to Step 4 until a compatible candidate is accepted or the available sentence trees are exhausted. Record the candidate order and why each candidate was accepted or rejected.

Why the initial restriction matters: in “John saw him”, the subject NP John is directly under the clause S, with no additional NP or S between them. Step 3 therefore does not propose John as the ordinary local antecedent of him. A reflexive such as himself is a different grammatical case. The worked question below shows the specific parse and traversal; selecting an antecedent is not simply choosing the nearest name.

ii. Apply the method

Given: the pronoun he occurs as the subject of the subordinate clause “after he arrived from Delhi”. Assume that this clause is attached to the main verb phrase. The relevant main-clause NPs are Ravi, Suresh and the station.

To find: the first eligible singular-male antecedent in Hobbs’ syntactic search order.

S (main)\n├── NP: Ravi                    ← first higher-left candidate\n└── VP\n    ├── VP: met [NP Suresh] at [NP the station]\n    └── SBAR: after\n        └── S (embedded)        ← first X\n            ├── NP: he          ← start\n            └── VP: arrived from Delhi

Step 1. Start at the pronoun NP

Locate NP “he” inside the after-clause. Move to that clause’s S node. No earlier noun phrase in this small clause provides a candidate.

Step 2. Move to the higher sentence

The embedded S is not the highest S, so Step 5 climbs through SBAR and VP to the main S. Step 7 now searches to the left of that new path. This is the later left search, so the extra initial-search restriction from Step 3 is not imposed again.

Step 3. Examine candidates in structural order

The main subject NP Ravi is encountered before the deeper object NP Suresh in the relevant breadth-first search. Ravi is singular and compatible with “he”. “the station” would fail the ordinary person constraint.

Step 4. Choose the first compatible candidate

Select Ravi under the assumed parse and agreement settings. Suresh remains a possible interpretation in ordinary language, but the syntactic preference chooses Ravi first.

Limitations

Final answer: Hobbs’ syntactic preference is Ravi for this analysis. This is an algorithmic selection, not proof of the speaker’s intended referent.

Yaad rakho: Nearest naam blindly nahi: tree ki search order aur agreement follow karo.

","m2":false,"units":"6","marks":10},{"id":"final-2025-q7a","title":"Q7A · Text classification pipeline","html":"
Question. Explain the working of text classification in Natural Language Processing (NLP) and discuss the key steps involved in building a text classification system in context of real-world applications such as spam detection or news categorization. [10 marks]

Answer

Text classification assigns one or more predefined labels to a document or message based on its content. In spam detection the labels may be spam and not-spam; in news categorization they may be sports, business and politics.

Labelled text → Prepare and split data → Represent text → Train classifier → Evaluate → Classify new text
  1. Define the labels and task. Decide whether each text gets one label or multiple labels. Establish what counts as spam so annotators use a consistent rule.
  2. Collect and label representative examples. Gather messages from the intended application and assign reliable labels. Include enough examples of each class and protect private information.
  3. Split training, validation and test data. Use training data to learn, validation data to choose settings and an untouched test set for final evaluation. Keep duplicate or closely related texts from leaking between splits.
  4. Preprocess consistently. Tokenize and normalize only as appropriate. In spam detection, URLs, currency symbols and capitalisation may contain useful information.
  5. Create features. A classical baseline uses BoW or TF–IDF. A neural system may use embeddings or a pretrained contextual model. Fit the vocabulary and IDF on training data only.
  6. Train a classifier. For example, multinomial Naïve Bayes estimates class priors and word likelihoods; logistic regression learns a decision boundary. Training uses the labelled examples.
  7. Evaluate. Inspect a confusion matrix, precision, recall and F1 as well as accuracy. False spam flags may hide legitimate mail, so the costs of errors matter.
  8. Deploy and monitor. Apply the same preprocessing and feature mapping to new texts. Monitor changing vocabulary and error patterns, and retrain when suitable new labelled data becomes available.

Example prediction

A message such as “Claim your free prize now” is transformed into its feature representation. The classifier compares the evidence for the two labels and outputs spam if its decision score crosses the chosen threshold. The label comes from learned patterns; the mere presence of one word should not be treated as a universally correct rule.

Yaad rakho: Labelled examples → features → model → evaluation → new text ka label.

","m2":false,"units":"7","marks":10},{"id":"final-2025-q7b","title":"Q7B · Corpora and Jaccard similarity","html":"
Question. i. Differentiate between a flat text corpus and an annotated corpus, and discuss how each can be applied in various NLP tasks with suitable examples.
ii. Explain a real-world scenario where the Jaccard Similarity measure is more appropriate than Euclidean or Manhattan distance, and justify why Jaccard is the best choice in that case. [10 marks]

Answer

i. Flat and annotated corpora

A corpus is a collection of language data. A flat text corpus stores text without the linguistic labels needed by a particular task. An annotated corpus adds labels or structure such as POS tags, entity spans, parse trees or sentence sentiment.

BasisFlat text corpusAnnotated corpus
ExampleA collection of plain news articlesThe same articles with PERSON/ORG spans marked
InformationThe original text and possibly simple metadataText plus explicit task-related linguistic labels
ApplicationsWord frequency, collocations and unsupervised language-model trainingSupervised POS tagging, NER or parser training and evaluation
PreparationRelatively inexpensive to gather, but still needs cleaningRequires annotation guidelines, labour and quality checks

“Flat” describes the lack of the annotations being discussed, not an absence of all file organization. Both corpus types may still have document boundaries and metadata.

ii. Jaccard in a set-based application

Consider recommending articles using their topic-tag sets. Article A has {AI, NLP, ethics}; article B has {AI, NLP, healthcare}. We care about overlap among present tags, not how many irrelevant tags are absent from both.

Given

A = {AI, NLP, ethics}; B = {AI, NLP, healthcare}.

To find

The fraction of distinct present tags shared by the articles.

Formula and why

\"J(A,B)=\\frac{|A\\cap

The intersection counts common items; the union counts all distinct items present in either set.

Step 1. Find common tags

A ∩ B = {AI, NLP}, so the intersection size is 2.

Step 2. Find all distinct present tags

A ∪ B = {AI, NLP, ethics, healthcare}, so the union size is 4.

Step 3. Divide

\"J(A,B)=\\frac24=0.5\"

Jaccard directly measures proportional overlap and ignores joint absences. Euclidean or Manhattan distance on binary tag vectors counts mismatches but does not by itself normalize by the union size. If frequency or magnitude were meaningful, those distance measures could be more appropriate. Here, membership-only tags and sparse sets make Jaccard a suitable choice.

Yaad rakho: Jaccard: common present items / total distinct present items.

","m2":false,"units":"2,3","marks":10}]},{"id":"timed-reexam-2026","title":"Timed practice · Solved re-exam · February 2026","unit":"PYQ","scope":"TEE · actual previous paper · 100 marks · 180 minutes","sections":[{"id":"instructions","title":"Write the whole paper under exam conditions","html":"

Use this actual previous paper as a three-hour, 100-mark practice paper. Q1 is compulsory (20 marks); choose any four full questions from Q2–Q7 (20 marks each). Each original question and its complete answer follow. Read the question, write your answer in your notebook, then compare the working below it.

Budget roughly 36 minutes per attempted 20-mark question, including its subparts. Start each full question on a fresh page, as the supplied papers instruct. This is an old paper used for timed practice, not a forecast of the next TEE.

"},{"id":"reexam-2026-q1a","title":"Q1A · Generic NLP pipeline","html":"
Question. Draw block diagram of generic NLP pipeline and explain the task associated with each block. [5 marks]

Answer

An NLP pipeline is an ordered workflow that transforms language data into a task-specific result. The exact components depend on the application; a generic supervised pipeline is shown below.

Collect text → Clean and preprocess → Represent text → Train / apply model → Evaluate and deliver output
BlockTaskExample for sentiment analysis
Data acquisitionCollect representative text and any required labelsReviews labelled positive or negative
Cleaning and preprocessingHandle noise, tokenize and normalize appropriatelyRemove HTML while preserving “not”
Text representationConvert tokens into usable model featuresTF–IDF values or contextual embeddings
ModelingLearn patterns from training data and predict new labelsTrain a classifier on labelled review features
EvaluationMeasure performance on unseen held-out data and inspect errorsPrecision, recall, F1 and confusion matrix
Deployment and feedbackDeliver predictions and monitor changing dataShow review sentiment and monitor new product vocabulary

Training/validation/test separation must precede fitting the learned preprocessing and feature statistics. New inputs must use the same feature mapping as training. Not every task uses every linguistic level explicitly: a modern end-to-end model may combine several operations internally.

Yaad rakho: Text lo → prepare karo → numbers banao → model lagao → result check aur use karo.

","m2":false,"units":"1","marks":5},{"id":"reexam-2026-q1b","title":"Q1B · Content and function words","html":"
Question. i. Define function words and content words.
ii. Which category (function words and content words) belong to open class and closed class? Justify your answer.
iii. For the sentence “The small cat is sleeping under the tree”, identify the function words and content words. [5 marks]

Answer

Content words carry much of a sentence’s lexical meaning, usually including nouns, main verbs, adjectives and many adverbs. Function words primarily express grammatical relationships, including determiners, prepositions, conjunctions, pronouns and auxiliary verbs.

CategoryTypical classReason
Content wordsOpen classNew nouns, verbs and adjectives enter the language readily, such as new technology names
Function wordsClosed classThe grammatical inventory is relatively stable; new determiners or pronouns are much less frequently added
WordRole in this sentenceCategory
TheDeterminerFunction
smallAdjectiveContent
catNounContent
isAuxiliary supporting progressive tenseFunction
sleepingMain verbContent
underPrepositionFunction
theDeterminerFunction
treeNounContent

Final answer: content words = small, cat, sleeping, tree; function words = The, is, under, the. “is” is auxiliary here; a word’s role must be judged in context. “Closed” means relatively resistant to new members, not that change is logically impossible.

Yaad rakho: Content words scene batate hain; function words grammar ka connection banate hain.

","m2":true,"units":"4","marks":5},{"id":"reexam-2026-q1c","title":"Q1C · Word2Vec and GloVe","html":"
Question. Differentiate between word2vec and GloVe in terms of their learning strategies. Which type of corpus would be more suitable for each method, and why? [5 marks]

Answer

Both Word2Vec and GloVe learn dense word vectors from distributional information: words occurring in related contexts tend to receive related representations. They differ mainly in the training objective and how corpus co-occurrence evidence is used.

BasisWord2VecGloVe
Learning strategyPredictive: CBOW predicts a centre word; Skip-gram predicts nearby wordsCount-based objective fitted to global word–context co-occurrence statistics
Training evidenceLocal context windows converted into prediction examplesA corpus-wide co-occurrence matrix and relationships among counts
Parameters learnedWord/context embeddings that make context predictions accurateWord/context embeddings and biases that fit weighted log co-occurrence counts
Corpus considerationsLarge token sequences with useful local context, including domain-specific textA sufficiently large corpus from which reliable global co-occurrence statistics can be collected

For a domain such as medicine, either method can benefit from a representative medical corpus. Word2Vec is convenient when generating local training windows directly from a stream of tokens. GloVe is natural when a global sparse co-occurrence matrix can be constructed and stored. Corpus size, domain match, preprocessing and available compute matter; there is no rule that Word2Vec requires informal text while GloVe requires formal text.

Both standard forms produce one static vector per vocabulary word, so multiple senses are blended. They do not automatically provide a different vector for the same word in each sentence.

Yaad rakho: Word2Vec predicts local neighbours; GloVe fits global co-occurrence counts.

","m2":false,"units":"5","marks":5},{"id":"reexam-2026-q1d","title":"Q1D · Anaphora, cataphora and coreference","html":"
Question. Differentiate between cataphora and anaphora in terms of direction of reference. How coreference resolution is different from anaphora reference? [5 marks]

Answer

Anaphora points backward to an expression or information introduced earlier. Cataphora points forward to an expression introduced later.

RelationExampleDirection
Anaphora“Riya arrived. She smiled.” She refers to the earlier Riya.Backward
Cataphora“Before she spoke, Riya checked the microphone.” She refers to the later Riya.Forward

Coreference means two or more expressions refer to the same entity. Coreference resolution groups such expressions, regardless of whether they are names, descriptions or pronouns and regardless of which is mentioned first.

For “Dr Sen arrived. The surgeon smiled. She waved,” {Dr Sen, the surgeon, She} is a coreference cluster. The latter two mentions are also backward-looking references. Coreference is an identity relation; anaphora describes a dependency on prior discourse and is not always identity. For example, “I bought a house. The roof leaks” contains a bridging reference: the roof depends on the house for interpretation, but roof and house are not the same entity.

Yaad rakho: Ana = pehle ki taraf; cata = aage ki taraf; coreference = same entity.

","m2":false,"units":"6","marks":5},{"id":"reexam-2026-q2a","title":"Q2A · NLP tasks and syntax versus semantics","html":"
Question. i. Discuss five different tasks associated with Natural Language Processing.
ii. Compare and contrast syntactic analysis and semantic analysis in NLP. Support your explanation with examples. [10 marks]

Answer

i. Five NLP tasks

TaskWhat it doesExample
Machine translationExpresses source-language content in a target languageEnglish “Good morning” → Hindi greeting
Sentiment analysisIdentifies the opinion or attitude expressed“The service was excellent” → positive sentiment
Named Entity RecognitionFinds and types entity spans“Asha joined Google” → PERSON and ORGANIZATION
Question answeringReturns an answer supported by a question’s relevant informationFrom a passage, identify who discovered a described result
Text classificationAssigns predefined category labelsClassify an email as spam or legitimate

ii. Syntactic and semantic analysis

BasisSyntactic analysisSemantic analysis
Main questionHow are words grammatically organised?What do the words and sentence mean?
Typical outputPOS tags, constituency or dependency structureWord senses, semantic relations or a meaning representation
Knowledge usedGrammar rules and structural patternsLexical meaning, context and semantic constraints
ExampleIdentify “The boy” as subject NP and “ate an apple” as VPInterpret the boy as eater and apple as the thing eaten

“Colourless green ideas sleep furiously” can have a plausible English grammatical structure while remaining semantically unusual. Syntax alone therefore does not guarantee ordinary meaning. Conversely, “boy eat apple yesterday” conveys a possible event despite grammatical errors.

The analyses support each other. In “I saw the man with a telescope,” syntactic PP attachment determines which meaning is available, and semantic knowledge can help choose a plausible attachment. A full NLP system often needs both.

Yaad rakho: Syntax = sentence ka structure; semantics = us structure ka meaning.

","m2":false,"units":"1,4,5","marks":10},{"id":"reexam-2026-q2b","title":"Q2B · Laplace smoothing on a second corpus","html":"
Question. Consider the following training corpus:
1. cats chase mice in the garden
2. dogs bark loudly at night
3. birds sing in the morning
4. children play games in the park
Using a bigram language model with Laplace smoothing, evaluate the probability of the test sentence: “cats bark in the park.” Assume <s> and </s> as start and end of the sentence. [10 marks]

Answer

Given

Test: cats bark in the park. Add one start token <s> and one end token </s> to every training and test sentence.

To find

The Laplace-smoothed bigram probability of the complete test sentence, including its end token.

Formula: which one and why

\"P_{+1}(w_i\\mid

C(previous,current) counts that adjacent pair in the training corpus. C(previous) counts its occurrences as a history. V is the vocabulary size. Add-one smoothing is necessary because an unseen pair must not force the whole sentence probability to zero.

\"P(w_1,\\ldots,w_n,\\langle/s\\rangle\\mid\\langle

Solution

Step 1. Fix the token and vocabulary convention

Lowercase words and remove the final punctuation. The supplied course key counts both boundary tokens in V. Following that convention: at, bark, birds, cats, chase, children, dogs, games, garden, in, loudly, mice, morning, night, park, play, sing, the, <s>, </s> gives V = 20. The start marker supplies history; it is not an extra probability factor before the first word.

Step 2. List every required transition

The complete test chain is <s> → cats → bark → in → the → park → </s>.

Step 3. Count first, then add one

Required bigramHistory countPair countSmoothed probability
<s> → cats41(1+1)/(4+20) = 1/12
cats → bark10(0+1)/(1+20) = 1/21
bark → in10(0+1)/(1+20) = 1/21
in → the33(3+1)/(3+20) = 4/23
the → park31(1+1)/(3+20) = 2/23
park → </s>11(1+1)/(1+20) = 2/21

Count only adjacent words, never words that merely appear in the same sentence. The unseen pairs have count zero, so their numerator becomes one.

Step 4. Multiply all six probabilities

\"P=\\frac{2}{24}\\times\\frac{1}{21}\\times\\frac{1}{21}\\times\\frac{4}{23}\\times\\frac{2}{23}\\times\\frac{2}{21}\"

Start with subtotal 1. At each row, multiply the previous subtotal by exactly one next factor. Simplify the fraction after multiplication, but do not round it.

Factor being includedPrevious subtotal × next factor = new subtotal
1. <s> → cats\"1\\times\\frac{1}{12}=\\frac{1}{12}\"
2. cats → bark\"\\frac{1}{12}\\times\\frac{1}{21}=\\frac{1}{252}\"
3. bark → in\"\\frac{1}{252}\\times\\frac{1}{21}=\\frac{1}{5292}\"
4. in → the\"\\frac{1}{5292}\\times\\frac{4}{23}=\\frac{1}{30429}\"
5. the → park\"\\frac{1}{30429}\\times\\frac{2}{23}=\\frac{2}{699867}\"
6. park → </s>\"\\frac{2}{699867}\\times\\frac{2}{21}=\\frac{4}{14697207}\"

The last subtotal is the complete product. Only now convert the fraction to a decimal.

\"P=\\frac{4}{14697207}\\approx

Step 5. Interpret the result

This is a joint sentence probability, so a small value is normal. Do not add transition probabilities; each word must occur in sequence, so the chain multiplies them.

Final answer

P ≈ 2.72160554 × 10-7 with the stated course vocabulary convention.

Yaad rakho: Pair count mein 1 jodo, previous-word count mein V; phir saare fractions multiply karo.

Convention note: Some implementations exclude <s> from the predicted vocabulary. Their V is one smaller and their numerical probability differs. In an exam, state V explicitly and follow the convention specified in the question or class solution.

","m2":true,"units":"3","marks":10},{"id":"reexam-2026-q3a","title":"Q3A · Top-down parsing","html":"
Question. In the context of natural language processing, explain the concept of top-down and bottom-up parsing approaches. Apply top-down syntax parsing approach to the sentence “The tall boy saw a girl in the park.” and draw a parse tree using the CFG grammar rules:
S → NP VP; NP → Det Adj N | Det N; VP → V NP | VP PP; PP → P NP.
Lexicon: Det → {the | a}; Adj → {tall}; N → {boy | girl | park}; V → {saw}; P → {in}. [10 marks]

Answer

AspectTop-downBottom-up
StartThe start symbol SThe input words
OperationExpand a nonterminal using grammar productionsCombine words and constituents using productions in reverse
SuccessLeaves match the complete inputAll input is reduced to S
Possible difficultyNaive recursive descent can loop on left recursion or need backtrackingA naive shift-reduce parser may face shift/reduce conflicts

Given

The sentence contains the tokens the / tall / boy / saw / a / girl / in / the / park. Lowercase the initial “The” to match the lexicon. The full stop is punctuation outside the listed grammar.

To find

A derivation beginning at S and a parse tree matching the whole sentence.

Rule: which one and why

A context-free production replaces one nonterminal with its right-hand sequence. A top-down derivation begins at S and expands the desired structure. The supplied grammar attaches the PP through VP → VP PP; it does not provide NP → NP PP.

Step 1. Expand the sentence

S ⇒ NP VP. Choose NP → Det Adj N for “the tall boy”.

Step 2. Match the subject words

Det ⇒ the, Adj ⇒ tall, N ⇒ boy. The remaining material must be generated by VP.

Step 3. Allow a verb phrase followed by a PP

Expand the outer VP using VP → VP PP. Then expand the inner VP using VP → V NP. This finite derivation selects the recursive rule once.

Step 4. Match the verb and object

V ⇒ saw. Choose NP → Det N, then Det ⇒ a and N ⇒ girl.

Step 5. Expand the prepositional phrase

PP ⇒ P NP. P ⇒ in. NP ⇒ Det N, with Det ⇒ the and N ⇒ park.

Step 6. Check the complete leaf order

The leaves read “the tall boy saw a girl in the park”, exactly the normalized input. The PP “in the park” attaches to the VP under this supplied grammar.

SNPVPDetAdjNVPPPVNPPNPDetNDetNThetallboysawagirlinthepark
Top-down parse tree showing the PP attached to VP.

Complete derivation

S ⇒ NP VP ⇒ Det Adj N VP ⇒ the tall boy VP ⇒ the tall boy VP PP ⇒ the tall boy V NP PP ⇒ the tall boy saw Det N PP ⇒ the tall boy saw a girl P NP ⇒ the tall boy saw a girl in Det N ⇒ the tall boy saw a girl in the park.

Implementation caution: VP → VP PP is left-recursive. A naive recursive-descent implementation may loop; it must transform or otherwise handle the grammar. A valid finite top-down derivation, as shown above, still exists.

Yaad rakho: Rule jo diya hai wahi use karo: yahan “in the park” VP ka part hai.

","m2":true,"units":"4","marks":10},{"id":"reexam-2026-q3b","title":"Q3B · Lesk word-sense disambiguation","html":"
Question. Consider the sentence: “The bank can ensure your deposits are safe.” The word “bank” has the following two possible senses (WordNet-style glosses):
Bank₁: Sloping land especially beside a body of water.
Bank₂: A financial institution that accepts deposits and channels the money into lending activities.
i. Apply the LESK algorithm to determine the correct sense of the word “bank” in the given sentence. Show the steps clearly.
ii. Briefly discuss limitations of the LESK algorithm in handling ambiguous words in real-world contexts. [10 marks]

Answer

Given

Context sentence and the two supplied dictionary glosses. Use a simplified Lesk method: lowercase, remove function words, exclude the target bank and compare distinct content-word overlap. Do not add synonyms that the question does not supply.

To find

Which candidate gloss has the greatest overlap with the local context.

Formula: which one and why

\"\\mathrm{score}(s)=|C\\cap

C is the set of context content words. Gₛ is the content-word set of the gloss for sense s. The intersection contains words appearing in both; its size is the score. This is an overlap count, not a probability.

Solution

Step 1. Prepare the context

Remove The, can, your, are and the target bank. A reasonable content set is C = {ensure, deposits, safe}. If “can” is retained under another fixed stop-list it matches neither gloss and does not change the result.

Step 2. Prepare Bank₁’s gloss

G₁ = {sloping, land, especially, beside, body, water}. There is no shared word with {ensure, deposits, safe}.

Step 3. Calculate Bank₁’s score

\"C\\cap

Step 4. Prepare Bank₂’s gloss and count overlap

G₂ = {financial, institution, accepts, deposits, channels, money, lending, activities}. The shared word is deposits.

Step 5. Calculate Bank₂’s score and select

\"C\\cap

1 > 0, so choose the financial-institution sense.

Final answer

Bank₂: financial institution. The explicit overlap “deposits” supports this choice.

Yaad rakho: Lesk: context aur har meaning ki definition mein common words count karo; highest overlap choose karo.

ii. Limitations

Lemmatization, richer glosses and semantic similarity can improve variants, but the exact variant and tie-breaking rule should be stated.

","m2":true,"units":"5","marks":10},{"id":"reexam-2026-q4a","title":"Q4A · Bag of Words and three cosine similarities","html":"
Question. Consider the following three documents:
Document A: “The dog chased the cat.”
Document B: “The cat chased the mouse.”
Document C: “The dog ran after the cat.”
Using the Bag of Words (BoW) model, represent each document as a vector after removing the stop words {The, by, was}.
i. Construct the term-document matrix.
ii. Compute the cosine similarity between each pair of documents (A–B, A–C, and B–C).
iii. Interpret and comment on the results obtained in terms of document similarity. [10 marks]

Answer

Given

Three documents. Lowercase and remove punctuation. Remove the specified stop words case-insensitively; “the” is removed everywhere. Keep “after” because it is not in the supplied stop list. Use raw word counts.

To find

A common-vocabulary count matrix, three cosine values and their interpretation.

Formula: which one and why

\"\\mathrm{BoW}_j(d)=C(v_j,d)\"

vⱼ is vocabulary term j. The coordinate stores its count in document d; order in the sentence is not retained.

\"\\cos(\\mathbf

Solution

Step 1. Clean each document

A: dog chased cat.
B: cat chased mouse.
C: dog ran after cat.

Step 2. Fix vocabulary and matrix

Vocabulary: dog, chased, cat, mouse, ran, after.

TermABC
dog101
chased110
cat111
mouse010
ran001
after001

A = [1,1,1,0,0,0]; B = [0,1,1,1,0,0]; C = [1,0,1,0,1,1].

Step 3. Calculate vector lengths

\"\\|A\\|=\\sqrt{1^2+1^2+1^2}=\\sqrt3\"
\"\\|B\\|=\\sqrt{1^2+1^2+1^2}=\\sqrt3\"
\"\\|C\\|=\\sqrt{1^2+1^2+1^2+1^2}=2\"

Step 4. Calculate A–B

\"A\\cdot
\"\\cos(A,B)=\\frac2{\\sqrt3\\sqrt3}=\\frac23\\approx0.666667\"

Step 5. Calculate A–C

\"A\\cdot
\"\\cos(A,C)=\\frac2{\\sqrt3\\cdot2}=\\frac1{\\sqrt3}\\approx0.577350\"

Step 6. Calculate B–C

\"B\\cdot
\"\\cos(B,C)=\\frac1{\\sqrt3\\cdot2}\\approx0.288675\"

Final answer

A–B = 0.666667; A–C = 0.577350; B–C = 0.288675. A and B have the highest lexical overlap under this representation. B and C share only cat. BoW similarity reflects token overlap, not full semantic equivalence or word order.

Yaad rakho: Same vocabulary order rakho. Corresponding counts multiply-add; phir dono lengths se divide.

","m2":false,"units":"2","marks":10},{"id":"reexam-2026-q4b","title":"Q4B · A second HMM sequence","html":"
Question. Consider the following POS tagged corpus. Apply Hidden Markov Model (HMM) approach to:
1. Dog/Noun eats/Verb bread/Noun
2. The/Det cat/Noun sleeps/Verb
3. A/Det dog/Noun eats/Verb fish/Noun
i. Construct the emission probability matrix.
ii. Construct the transmission probability matrix.
iii. Calculate the probability of the given sequence of events: The/Det dog/Noun eats/Verb bread/Noun.
Note: Include start and end characters for all the sentences. [10 marks]

Answer

The repeated verb eats has a different emission count from the final paper; recalculate it rather than copying the previous numerical result.

Given

SentenceTagged sequence
1dog/Noun eats/Verb bread/Noun
2the/Det cat/Noun sleeps/Verb
3a/Det dog/Noun eats/Verb fish/Noun

Lowercase the corpus. Test sequence: the/Det dog/Noun eats/Verb bread/Noun. Add Start and End markers.

To find

Emission and transition matrices, then the probability of the supplied tagged sequence.

Formula: which one and why

\"P(w\\mid
\"P(t_j\\mid

Emission divides the word’s tag-specific count by the total tokens with that tag. Transition divides the tag-pair count by all transitions leaving the previous tag, including End.

\"P(\\mathbf

Solution

Step 1. Count tag totals and word frequencies

Det: the=1, a=1, total2. Noun: dog=2, bread=1, cat=1, fish=1, total5. Verb: eats=2, sleeps=1, total3.

Step 2. Construct emission matrix

Tag / wordabreadcatdogeatsfishsleepsthe
Det1/20000001/2
Noun01/51/52/501/500
Verb00002/301/30

Step 3. Count transitions from the three chains

Start → Noun → Verb → Noun → End
Start → Det → Noun → Verb → End
Start → Det → Noun → Verb → Noun → End

FromDet countNoun countVerb countEnd countTotal
Start21003
Det02002
Noun00325
Verb02013

Step 4. Divide outgoing counts

From / toDetNounVerbEnd
Start2/31/300
Det0100
Noun003/52/5
Verb02/301/3

All nonterminal rows sum to one. No smoothing is requested.

Step 5. Write every sequence factor

EventProbability
Start → Det2/3
Det emits the1/2
Det → Noun1
Noun emits dog2/5
Noun → Verb3/5
Verb emits eats2/3
Verb → Noun2/3
Noun emits bread1/5
Noun → End2/5

Step 6. Multiply all factors

\"P=\\frac23\\times\\frac12\\times1\\times\\frac25\\times\\frac35\\times\\frac23\\times\\frac23\\times\\frac15\\times\\frac25\"

Start with subtotal 1. At each row, multiply the previous subtotal by exactly one next factor. Simplify the fraction after multiplication, but do not round it.

Factor being includedPrevious subtotal × next factor = new subtotal
1. Start → Det\"1\\times\\frac{2}{3}=\\frac{2}{3}\"
2. Det emits the\"\\frac{2}{3}\\times\\frac{1}{2}=\\frac{1}{3}\"
3. Det → Noun\"\\frac{1}{3}\\times1=\\frac{1}{3}\"
4. Noun emits dog\"\\frac{1}{3}\\times\\frac{2}{5}=\\frac{2}{15}\"
5. Noun → Verb\"\\frac{2}{15}\\times\\frac{3}{5}=\\frac{2}{25}\"
6. Verb emits eats\"\\frac{2}{25}\\times\\frac{2}{3}=\\frac{4}{75}\"
7. Verb → Noun\"\\frac{4}{75}\\times\\frac{2}{3}=\\frac{8}{225}\"
8. Noun emits bread\"\\frac{8}{225}\\times\\frac{1}{5}=\\frac{8}{1125}\"
9. Noun → End\"\\frac{8}{1125}\\times\\frac{2}{5}=\\frac{16}{5625}\"

The last subtotal is the complete product. Only now convert the fraction to a decimal.

\"P=\\frac{16}{5625}\\approx0.0028444444\"

Final answer

P = 16/5625 ≈ 0.0028444444 for these words and specified tags jointly. The two occurrences of eats give P(eats | Verb) = 2/3.

Yaad rakho: Same tag structure ho sakta hai, lekin word emission counts dobara count karna.

Boundary reminder: Include P(Det | Start) = 2/3 and P(End | Noun) = 2/5. The handwritten key’s final line instead inserts the overall Det fraction 2/10 and omits End; the complete boundary-aware probability requested here is 0.0028444444.

","m2":true,"units":"4","marks":10},{"id":"reexam-2026-q5a","title":"Q5A · Why TF–IDF is useful","html":"
Question. Justify with an appropriate example the importance of TF-IDF technique in the document representation and information retrieval task. [10 marks]

Answer

TF–IDF represents each document using weights that combine within-document term frequency with how unusual a term is across the collection. It supports information retrieval by giving larger weight to terms that are frequent in a document but not present in every document.

Given

Use three short documents: D₁ = “solar energy solar”; D₂ = “wind energy”; D₃ = “energy policy”. Use raw-count TF and natural-log unsmoothed IDF. N = 3.

To find

A TF–IDF representation illustrating why distinctive terms help retrieve relevant documents.

Formula: which one and why

\"\\mathrm{TFIDF}(t,d)=C(t,d)\\ln\\frac{N}{\\mathrm{df}(t)}\"

C counts occurrences within a document; df counts documents containing the term. The logarithm reduces the scale of very large rarity ratios.

Solution

Step 1. Count term occurrences

TermD₁D₂D₃df
solar2001
energy1113
wind0101
policy0011

Step 2. Calculate IDF

\"\\mathrm{IDF}(solar)=\\mathrm{IDF}(wind)=\\mathrm{IDF}(policy)=\\ln3\\approx1.098612\"
\"\\mathrm{IDF}(energy)=\\ln(3/3)=0\"

Step 3. Multiply each count by its IDF

TermD₁ weightD₂ weightD₃ weight
solar2 × 1.098612 = 2.19722400
energy000
wind01.0986120
policy001.098612

Step 4. Connect the representation to retrieval

A query containing “solar” matches D₁’s high solar coordinate. A query containing “wind” matches D₂. “energy” alone cannot distinguish these documents under this unsmoothed collection because it occurs in all three. A retrieval system can compare query and document vectors, commonly by cosine similarity.

Final answer

TF–IDF preserves distinctive lexical evidence and reduces dominance by collection-wide frequent terms. D₁ is most relevant to a solar query under these features.

Yaad rakho: Document mein frequent + collection mein rare = high TF–IDF.

Strengths and limits

The representation is sparse, interpretable and useful as a fast baseline. However, basic TF–IDF ignores word order and does not recognise synonyms or multiple word senses automatically. Its weights depend on the collection and on the chosen TF/IDF convention. Fit IDF on training or indexed documents, not on a hidden test set.

","m2":false,"units":"2","marks":10},{"id":"reexam-2026-q5b","title":"Q5B · Sentence-window vectors and cosine","html":"
Question. Consider the paragraph: The farmer planted seeds in the soil. The rain watered the soil. The sun helped the plants grow. The farmer harvested the plants. The soil became rich and fertile.
Target words: farmer, soil, sun, plant.
Context words: plant, water, grow, harvest.
i. Construct a co-occurrence matrix for the target and context words. Take the window size as one whole sentence.
ii. Evaluate the similarity between the vectors of the target words “farmer-soil”, “farmer-sun”, and “farmer-plant” using the cosine similarity.
iii. Plot the target word vectors by considering only two dimensions: “planted” and “harvested”. [10 marks]

Answer

Given

Five sentence windows. For alignment with the supplied matrix, target plant means the noun plants, occurring in sentences 3 and 4. Context plant covers planted and plants; watered → water; harvested → harvest. Count a target–context pair once per sentence in which both are present, including the plant–plant sentence-presence diagonal.

To find

The target–context matrix, three cosine similarities and a two-coordinate plot.

Formula: which one and why

\"M_{ij}=\\sum_s\\mathbf1[\\text{target}_i\\text{
\"\\cos(\\mathbf

This is a sentence-presence matrix, not a position-based sliding window that excludes a token from its own context. Stating the convention is necessary because those alternatives give a different diagonal.

Solution

Step 1. Normalize and identify each sentence’s contribution

SentenceTargets presentContext lemmas present
1farmer, soilplant
2soilwater
3sun, plantplant, grow
4farmer, plantplant, harvest
5soilnone

Step 2. Fill all matrix rows

Targetplantwatergrowharvest
farmer2001
soil1100
sun1010
plant2011

Farmer meets context plant in sentences 1 and 4. Soil meets plant in sentence 1 and water in sentence 2. Sun meets plant and grow in sentence 3. The noun plant occurs in sentences 3 and 4, producing its last row.

Step 3. Compute farmer–soil

\"\\mathbf
\"\\|\\mathbf
\"\\cos(\\mathbf

Step 4. Compute farmer–sun

\"\\mathbf
\"\\|\\mathbf

Step 5. Compute farmer–plant

\"\\mathbf
\"\\|\\mathbf
\"\\cos(\\mathbf

Step 6. Plot the requested two coordinates

Interpret the axis names planted and harvested as their normalized context features plant and harvest. Farmer=(2,1), soil=(1,0), sun=(1,0), plant=(2,1). Farmer and plant overlap; soil and sun overlap.

001122farmer, plantsoil, sunplant (planted)harvest (harvested)
The plot uses only the requested two coordinates; cosine uses all context columns.

Final answer

Using the stated noun-target convention: farmer–soil = 0.632456, farmer–sun = 0.632456, farmer–plant = 0.912871.

Yaad rakho: Plant noun aur plant verb ko same maana ya separate? Pehle convention likho, phir count karo.

Two details that prevent an incorrect answer

Arithmetic: 5/√30 = 0.912871; the source key’s printed approximation 0.9444 does not equal its own fraction. Keep the exact fraction until the last step.

Normalization: If the question instead requires all noun and verb forms of plant to be merged as targets, sentence 1 also contains the plant target. Its row becomes [3,0,1,1], and farmer–plant becomes 7/√55 ≈ 0.943880. The paper leaves this choice unstated; the main worked solution explicitly uses the noun-target interpretation that reproduces its supplied matrix.

","m2":false,"units":"5","marks":10},{"id":"reexam-2026-q6a","title":"Q6A · Pronoun approaches and lexical relations","html":"
Question. i. Briefly describe any two approaches to pronoun resolution and mention one advantage of each.
ii. Describe the difference between Homonymy and Synonymy with suitable examples. Discuss how these two relations differently affect natural language processing applications? [10 marks]

Answer

i. Two approaches to pronoun resolution

ApproachHow it worksAdvantage
Rule-based / syntacticUses parse structure, agreement, recency and linguistic constraints. Hobbs’ algorithm searches parse trees for eligible antecedents.Interpretable: a proposed antecedent can be traced to explicit rules.
Statistical or neuralLearns candidate–pronoun compatibility or whole-document coreference patterns from labelled examples, using context and linguistic features or contextual embeddings.Can learn interactions and contextual cues that are difficult to enumerate as hand-written rules.

For “Asha dropped the glass because it was slippery,” a system should evaluate candidate referents and context rather than merely choose the most recent noun. Rules can constrain number and syntax; learned models can use compatibility patterns, though neither approach is guaranteed to resolve every ambiguity.

ii. Homonymy and synonymy

RelationDefinitionExample
HomonymySame spelling or sound, unrelated meaningsbank as financial institution versus bank as river edge
SynonymyDifferent words with the same or closely related meanings in a contextcar and automobile; big and large

Homonymy creates an ambiguity problem: a search for bank may retrieve finance or rivers. WSD uses context to choose the intended sense. Synonymy creates a vocabulary-matching problem: a document saying automobile may be relevant to a query saying car. Query expansion or semantic representations can improve recall by connecting related terms.

The remedies differ. Do not merge unrelated homonym senses simply because their spelling matches. Do not insist on exact surface-form matching when different words express the same concept. Synonyms are also context-sensitive: “big mistake” and “large mistake” are not always equally natural.

Yaad rakho: Homonym: same word-form, different meaning. Synonym: different words, similar meaning.

","m2":false,"units":"5,6","marks":10},{"id":"reexam-2026-q6b","title":"Q6B · Hobbs: Ramesh and Shyam","html":"
Question. i. Explain Hobbs’ Algorithm for pronoun resolution. State its main steps and limitations.
ii. Apply Hobbs’ Algorithm to the sentence: “When Ramesh gave Shyam a book, he smiled.” Identify the antecedent of “he” by showing traversal steps. [10 marks]

Answer

Hobbs’ algorithm is a syntax-based pronoun-resolution procedure that searches constituency parse trees in a specified order. It proposes noun phrases as antecedents and filters incompatible candidates, for example by number and gender agreement.

Read the tree labels first: NP means noun phrase; S means sentence/clause; VP means verb phrase. X is the current NP or S search node, and p is the upward path used to reach it. Breadth-first means shallower nodes before deeper nodes; left-to-right fixes the order at the same depth. An N-bar (N′) node is the noun-centred part of an NP in a parse using that label.

Complete ordered procedure

  1. Locate the pronoun. Start at the NP immediately dominating the pronoun: the smallest NP containing it.
  2. Make the first climb. Move upward to the first NP or S above the starting NP. Call it X and record the path p.
  3. Search initially to the left. Traverse branches below X to the left of p, breadth-first and left-to-right. Propose an NP only if another NP or S lies between that candidate and X. Check agreement and other applicable constraints for each proposed candidate; accept the first eligible compatible one.
  4. Check the highest sentence. If X is the highest S and no candidate has been accepted, search previous sentence trees, most recent first. In each, traverse breadth-first and left-to-right and propose NPs. Accept the first eligible compatible candidate; if the available context contains none, report no resolved antecedent.
  5. Otherwise climb again. Move from X to the next NP or S above it. Make that node the new X and record the new upward path p.
  6. Test a higher NP. If the new X is an NP, propose X only when p did not pass through its nominal N-bar child. Apply the same compatibility checks. Use the labels in the supplied parse; do not invent a missing N-bar branch.
  7. Search left again. Traverse the new X’s branches to the left of p, breadth-first and left-to-right, proposing NPs. The extra intervening-NP/S requirement in Step 3 belongs to the initial search.
  8. Search permitted right branches. If X is an S, traverse branches to the right of p, breadth-first and left-to-right, but do not descend below any NP or S encountered. Propose the encountered eligible NPs and test their compatibility.
  9. Repeat or finish. Return to Step 4 until a compatible candidate is accepted or the available sentence trees are exhausted. Record the candidate order and why each candidate was accepted or rejected.

Why the initial restriction matters: in “John saw him”, the subject NP John is directly under the clause S, with no additional NP or S between them. Step 3 therefore does not propose John as the ordinary local antecedent of him. A reflexive such as himself is a different grammatical case. The worked question below shows the specific parse and traversal; selecting an antecedent is not simply choosing the nearest name.

Given

The pronoun he is the subject of the main clause. The preceding subordinate clause is “When Ramesh gave Shyam a book”. Assume the temporal clause attaches to the whole main sentence.

To find

The first suitable antecedent in the left-to-right breadth-first search of the preceding clause.

S (main)                        ← first X\n├── SBAR: When\n│   └── S (embedded)             ← intervening S\n│       ├── NP: Ramesh           ← first candidate\n│       └── VP\n│           ├── V: gave\n│           ├── NP: Shyam\n│           └── NP: a book\n├── NP: he                      ← start\n└── VP: smiled

Step 1. Locate the pronoun NP

Start at NP “he” and move upward to its S ancestor. Record the path from he.

Step 2. Search branches before the pronoun

The initial when-clause lies to the left of the path. Apply Step 3: its embedded S lies between each of its noun phrases and the main S, so these NPs satisfy the initial intervening-NP/S requirement. Traverse left-to-right and breadth-first.

Step 3. Propose noun phrases

The subject NP Ramesh is encountered before the deeper object NP Shyam and object NP a book. Ramesh agrees with the assumed singular-male pronoun.

Step 4. Accept the first compatible antecedent

Select Ramesh. Shyam is also a semantically possible referent in ordinary discourse, but the algorithm’s stated syntactic preference selects Ramesh first.

Limitations

Final answer: Ramesh, under the shown parse and ordinary agreement assumptions. A syntactic preference is not a guarantee about the author’s intended meaning.

Yaad rakho: Left branches ko breadth-first dekho; pehla eligible matching NP choose karo.

","m2":false,"units":"6","marks":10},{"id":"reexam-2026-q7a","title":"Q7A · Information retrieval","html":"
Question. i. Explain the concept of Information Retrieval, highlighting its role in Recommender Systems and Search Engines.
ii. Discuss the various steps involved in the Information Retrieval process. [10 marks]

Answer

i. Concept and roles

Information Retrieval (IR) finds and ranks documents or other information objects relevant to a user’s information need. A query is the user’s expression of that need. IR typically returns a ranked set of items; information extraction instead identifies structured facts or fields inside those items.

ApplicationRole of IRExample
Search engineMatches a submitted query against a large indexed collection and ranks candidate pagesA search for “symptoms of dehydration” returns relevant health pages
Recommender systemRetrieves candidate items using a user profile, recent activity or a content representation; a later model may personalize rankingRetrieve articles similar to topics the user reads

Retrieval is a component of many recommenders; recommendation also involves preferences, feedback, novelty and business constraints. A recommender need not wait for an explicit textual query.

ii. Retrieval process

Collect → Preprocess → Index → Process query → Retrieve candidates → Rank → Evaluate
  1. Collect and define documents. Crawl or ingest documents, record metadata and remove unwanted duplicates.
  2. Preprocess. Tokenize and normalize using rules suitable for both documents and queries. Preserve task-relevant information.
  3. Build representations and an index. An inverted index maps each term to the documents containing it; a dense retrieval system stores vector representations.
  4. Process the query. Apply compatible tokenization, spelling handling or expansion, then obtain query features.
  5. Retrieve candidates. Use the index to identify potentially relevant documents efficiently.
  6. Score and rank. A lexical method may use TF–IDF/BM25-style evidence; a semantic method compares representations. Rank scores estimate relevance rather than proving correctness.
  7. Present results. Return titles, snippets and links, possibly with filtering or diversification.
  8. Evaluate and update. Use relevance judgments and measures such as precision, recall or ranking metrics. Refresh the index as documents change.

Example: for “solar energy storage”, an inverted index retrieves documents containing those terms; a ranking function rewards useful matches. A semantic system may also retrieve relevant material using related terms such as batteries.

Yaad rakho: IR mein relevant document dhoondho aur rank karo; extraction mein document ke andar facts nikalo.

","m2":false,"units":"7","marks":10},{"id":"reexam-2026-q7b","title":"Q7B · Gutenberg corpus and normalization","html":"
Question. i. Discuss how the Gutenberg Corpus can be utilized for various text analysis tasks in Natural Language Processing?
ii. Explain the importance of text normalization in Natural Language Processing (NLP). Illustrate your answer with a suitable example of an application where text normalization plays a crucial role. [10 marks]

Answer

i. Using the Gutenberg corpus

The Gutenberg corpus provides literary texts, commonly used through tools such as NLTK’s Gutenberg collection. It supplies naturally occurring sentences and longer documents for exploring language patterns. The available subset and licensing should be checked before reuse in a new distribution.

TaskUse of the corpus
Frequency and vocabulary studyCount tokens and word types, inspect common words and compare lexical diversity across texts
Concordances and collocationsView a word in context and discover words that occur near each other
N-gram language modelingEstimate unigram or bigram counts and compare sequence probabilities
Stylistic or authorship analysisCompare sentence lengths, vocabulary and function-word patterns across authors
Text preprocessing experimentsStudy the effect of tokenization, case normalization and lemmatization on counts

A sound workflow selects files, inspects encoding and boundaries, tokenizes consistently, computes the relevant statistics and interprets them against corpus composition. Literary text is not automatically representative of chat, medical records or present-day search queries; domain mismatch matters.

ii. Text normalization

Normalization converts selected surface variants into a consistent form so that equivalent expressions do not unnecessarily produce different features. Possible steps include case folding, whitespace cleanup, Unicode normalization, contraction expansion and lemmatization. Choose these according to the task.

Example: product search

  1. Raw query: “ RUNNING shoes ”. Product description: “Comfortable run shoes”.
  2. Trim extra whitespace and apply consistent case: “running shoes”.
  3. If the application chooses lemmatization, running → run. The query and the indexed description now share the intended run and shoes features.
  4. Apply the same normalization to stored documents and future queries, otherwise matching remains inconsistent.

Normalization reduces avoidable vocabulary variation and can improve recall. It must preserve distinctions important to the task: lowercasing US to us may lose meaning, and removing “not” can reverse sentiment. Aggressive stemming or synonym replacement can create incorrect matches.

Yaad rakho: Same meaning ke surface variants ko consistent banao; useful meaning delete mat karo.

","m2":false,"units":"2,3","marks":10}]}],"revision":[{"id":"tee-revision","title":"TEE · full-course revision","unit":0,"sections":[{"id":"all-units","title":"All seven units: final revision","html":"

TEE is 100 marks in 3 hours, scaled to 50 in the course total. Q1 is compulsory; choose any four of Q2–Q7. Each full question carries 20 marks. The two supplied 2025–26 papers follow this rule. Do not use the short M2 route as complete TEE preparation.

UnitMust be able to explainMust be able to work
1NLP levels, building blocks, NLU/NLG, approaches, pipeline and challengesClassify ambiguity and identify linguistic units in examples.
2Tokenization, case, stopwords, stemming/lemmatization, encoding, BoW/TF-IDFConstruct vectors; compute TF-IDF, cosine/Jaccard/Euclidean/Manhattan/Minkowski.
3Corpora, annotation, dictionaries, regex, N-gram models and smoothingList n-grams; count probabilities; smooth; compute sentence probability/perplexity.
4POS approaches, HMM, chunking/chinking, CFG parsing, NERBuild emission/transition matrices; score paths; draw a tree; trace parser operations.
5Lexical relations, WordNet, WSD, distributional/vector semantics, Word2Vec, GloVeLesk and NB; co-occurrence vectors; similarity; CBOW/Skip-gram examples.
6Pragmatics/discourse, reference, coreference, pronouns, Hobbs and other approachesIdentify chains; show a tree traversal and state parse assumptions.
7Classification, sentiment, spelling, translation, QA, IE, IRDesign an application pipeline; calculate relevant model/evaluation examples.

TEE revision sequence

First revise the formula sheet and one full example per numerical family. Next read each unit’s definitions and comparisons. Then work one complete previous paper using its original choice rule. Finally revisit only the steps you got wrong, and read the other paper’s new question forms. Keep examples and diagrams beside definitions when revising; a term you cannot illustrate is not yet ready to explain.

For a 10-mark numerical, show the count table or matrix and the intermediate probabilities. For a 10-mark theory answer, organise definition, mechanism/steps, diagram/example, advantages and limitations around the actual question. Do not add unrelated paragraphs just to fill space.

"},{"id":"numerical-method","title":"Numerical answer framework","html":"
Question clueUse this methodFirst thing to write
“next word”, “sentence probability”N-gram modelList the needed word transitions.
“unseen pair”, “Laplace”Add-one smoothingState V and count each history.
“perplexity”Reciprocal Nth rootState P(test sequence) and N.
“transition/emission matrix”HMM count estimationSeparate tag-to-tag counts from tag-to-word counts.
“given tag sequence”HMM joint pathInitial × emission × transition × emission …
“best tag sequence”Viterbi or enumerate tiny pathsInitialise one score per tag; retain predecessor maxima.
“gloss overlap”Simplified LeskPreprocess context and glosses consistently.
“classify word sense with counts”Naive Bayes WSDPriors from examples, likelihoods from tokens.
“parse tree”Given CFG + chosen parserShow grammar expansions or stack reductions.

Errors that change the answer

1. Dividing a conditional count by the whole corpus instead of its history count. 2. Adding 1 to the numerator without adding |V| to the denominator. 3. Forgetting the end transition. 4. Counting repeated words as new vocabulary types. 5. Using P(tag|word) in place of P(word|tag). 6. Calling an unnormalised score a posterior. 7. Silently mixing case, lemmatisation or boundary conventions.

Short theory answer format

For 2 marks: give the exact definition and one specific example or distinction. For 3 marks: add the mechanism or two ordered steps; include a small diagram/table where it carries information. The actual allocation is examiner-dependent, so these are writing guides, not an official marking scheme.

"}],"scope":"TEE · revision, not a replacement for full lessons"}],"about":[{"id":"pyq-analysis","title":"All recovered TEE papers · unit, topic and marks map","unit":"PYQ","sections":[{"id":"all-papers","title":"Latest and older papers: exact question map","html":"

How to use this question map

Start with the newest final and re-exam, then use older papers to practise variations. The table links each original lettered part to its complete worked answer. Repeated questions are history, not a prediction. A syllabus topic stays required even when no recovered paper asks it.

Marks: 100 marks are attempted: compulsory Q1 (20), then any four other 20-mark questions. Most recovered papers offer140 marks; NLP December2023 offers120 because the complete original ends at Q6. A mixed question retains one mark allocation; its units are not counted as separate questions. Blank individual allocations remain unknown.

2026-02-09 · re-exam · R26

140 marks offered; 100 attempted.

Question [marks]Current unit and topicQuestion formAnswer / scope
Q1a [5]Unit 1 · NLP pipelinetheoryRead question + answer
Q1b [5]Unit 1, 4 · Content and function wordsmixedRead question + answer
Q1c [5]Unit 5 · Word2Vec and GloVetheoryRead question + answer
Q1d [5]Unit 6 · Cataphora, anaphora and coreferencetheoryRead question + answer
Q2a [10]Unit 1 · Five NLP tasks; syntax and semanticstheoryRead question + answer
Q2b [10]Unit 3 · Add-one bigram sentence probabilitynumericalRead question + answer
Q3a [10]Unit 4 · Top-down CFG parsingcaseRead question + answer
Q3b [10]Unit 5 · Lesk with supplied glossesnumericalRead question + answer
Q4a [10]Unit 2 · Bag of words and three cosine similaritiesnumericalRead question + answer
Q4b [10]Unit 4 · HMM estimation and given tag-path joint probabilitynumericalRead question + answer
Q5a [10]Unit 2 · Explain and illustrate TF-IDFmixedRead question + answer
Q5b [10]Unit 5 · Co-occurrence, cosine and normalized plotnumericalRead question + answer · Target/context normalization and sentence-presence convention stated; alternative target lemmatization has a separately calculated result.
Q6a [10]Unit 5, 6 · Pronoun-resolution methods; homonymy and synonymytheoryRead question + answer
Q6b [10]Unit 6 · Hobbs antecedent traversalcaseRead question + answer · Hobbs requires a parse and compatibility assumptions. The assumed parse and traversal order are disclosed.
Q7a [10]Unit 7 · Information-retrieval stagestheoryRead question + answer
Q7b [10]Unit 2, 3 · Gutenberg and text normalizationtheoryRead question + answer

Original offered marks by mapped topic, grouped without double counting: Unit 1: 2 parts, 15 known marks; Unit 1, 4: 1 parts, 5 known marks; Unit 5: 3 parts, 25 known marks; Unit 6: 2 parts, 15 known marks; Unit 3: 1 parts, 10 known marks; Unit 4: 2 parts, 20 known marks; Unit 2: 2 parts, 20 known marks; Unit 5, 6: 1 parts, 10 known marks; Unit 7: 1 parts, 10 known marks; Unit 2, 3: 1 parts, 10 known marks. Joint-unit buckets remain joint; no invented split is assigned. These original-paper totals include any legacy tasks explicitly marked above; they are not a current-syllabus marks blueprint.

2025-12-02 · final · F25

140 marks offered; 100 attempted.

Question [marks]Current unit and topicQuestion formAnswer / scope
Q1a [5]Unit 1 · NLU versus NLGtheoryRead question + answer
Q1b [5]Unit 4 · Chunking, chinking and POStheoryRead question + answer
Q1c [5]Unit 5 · Hyponymy/hypernymy and WSDtheoryRead question + answer
Q1d [5]Unit 6 · Pragmatic and discourse analysistheoryRead question + answer
Q2a [10]Unit 1, 4 · Morpheme/lexeme and NER applicationstheoryRead question + answer
Q2b [10]Unit 3 · Add-one bigram sentence probabilitynumericalRead question + answer
Q3a [10]Unit 1, 4 · Ambiguities and bottom-up CFG parsemixedRead question + answer
Q3b [10]Unit 5 · Co-occurrence, cosine and vector plotnumericalRead question + answer
Q4a [10]Unit 2 · Three preprocessing techniquestheoryRead question + answer
Q4b [10]Unit 4 · HMM estimation and given tag-path joint probabilitynumericalRead question + answer
Q5a [10]Unit 2 · TF-IDF matrixnumericalRead question + answer
Q5b [10]Unit 5 · Naive Bayes word-sense disambiguationnumericalRead question + answer · Fixed vocabulary/OOV treatment is stated; source-key OOV smoothing differs. Both conventions and their consequence are explained.
Q6a [10]Unit 5, 6 · Coreference/pronouns; CBOW and Skip-gramtheoryRead question + answer
Q6b [10]Unit 6 · Hobbs antecedent traversalcaseRead question + answer · Hobbs requires a parse and compatibility assumptions. The assumed parse and traversal order are disclosed.
Q7a [10]Unit 7 · Document-classification pipelinetheoryRead question + answer
Q7b [10]Unit 2, 3 · Corpus structures and Jaccard examplemixedRead question + answer

Original offered marks by mapped topic, grouped without double counting: Unit 1: 1 parts, 5 known marks; Unit 4: 2 parts, 15 known marks; Unit 5: 3 parts, 25 known marks; Unit 6: 2 parts, 15 known marks; Unit 1, 4: 2 parts, 20 known marks; Unit 3: 1 parts, 10 known marks; Unit 2: 2 parts, 20 known marks; Unit 5, 6: 1 parts, 10 known marks; Unit 7: 1 parts, 10 known marks; Unit 2, 3: 1 parts, 10 known marks. Joint-unit buckets remain joint; no invented split is assigned. These original-paper totals include any legacy tasks explicitly marked above; they are not a current-syllabus marks blueprint.

2025-02-21 · re-exam · R25

140 marks offered; 100 attempted. Q1 total20 is printed, but the four individual part allocations are blank. Do not infer5marks as if printed. Sixth PDF page is blank.

Question [marks]Current unit and topicQuestion formAnswer / scope
Q1a [not printed individually]Unit 2 · BoW and one-hot encodingtheoryRead question + answer
Q1b [not printed individually]Unit 5 · Lesk WSDtheoryRead question + answer
Q1c [not printed individually]Unit 4 · Syntax parsing approachestheoryRead question + answer
Q1d [not printed individually]Unit 1, 7 · NLU versus NLG and applicationstheoryRead question + answer
Q2a [10]Unit 2 · TF–IDF cosine after a fixed stoplistnumericalRead question + answer
Q2b [10]Unit 7 · Weather Naive BayesnumericalRead question + answer
Q3a [5]Unit 6 · Coreference and anaphoratheoryRead question + answer
Q3b [5]Unit 6 · Cohesion, coherence and discourse markerstheoryRead question + answer
Q3c [10]Unit 7 · Question answering systemtheoryRead question + answer
Q4a [5]Unit 1 · NLP pipelinetheoryRead question + answer
Q4b [5]Unit 1 · Morphology and syntaxtheoryRead question + answer
Q4c [10]Unit 3 · Most probable bigram continuationsnumericalRead question + answer
Q5a [6]Unit 2 · Preprocess the NLP passagecaseRead question + answer
Q5b [4]Unit 2 · Stemming and lemmatization toolstheoryRead question + answer
Q5c [10]Unit 6 · Hobbs on John and MarycaseRead question + answer
Q6a [10]Unit 3 · Corpora and acquisitiontheoryRead question + answer
Q6b [10]Unit 4 · HMM components and estimatesmixedRead question + answer
Q7a [5]Unit 4 · Word classes, chunking and chinkingtheoryRead question + answer
Q7b [5]Unit 4 · Named entity recognitiontheoryRead question + answer
Q7c [10]Unit 5 · Co-occurrence with inconsistent requested comparisonsnumericalRead question + answer · Source defect: follow-up names belong to another corpus. Exact supported matrix is complete; unsupported parts are explicitly unresolved from the original data.

Original offered marks by mapped topic, grouped without double counting: Unit 2: 4 parts, 20 known marks + 1 parts with no individual mark printed; Unit 5: 2 parts, 10 known marks + 1 parts with no individual mark printed; Unit 4: 4 parts, 20 known marks + 1 parts with no individual mark printed; Unit 1, 7: 1 parts, 0 known marks + 1 parts with no individual mark printed; Unit 7: 2 parts, 20 known marks; Unit 6: 3 parts, 20 known marks; Unit 1: 2 parts, 10 known marks; Unit 3: 2 parts, 20 known marks. Joint-unit buckets remain joint; no invented split is assigned. These original-paper totals include any legacy tasks explicitly marked above; they are not a current-syllabus marks blueprint.

2024-12-03 · final · F24

140 marks offered; 100 attempted. Four Q1 part allocations are blank; their group totals20. Other printed lettered allocations determine each 20-mark main question.

Question [marks]Current unit and topicQuestion formAnswer / scope
Q1a [not printed individually]Unit 2, 7 · TF–IDF in retrievaltheoryRead question + answer
Q1b [not printed individually]Unit 4 · NER and approachestheoryRead question + answer
Q1c [not printed individually]Unit 5 · Skip-gram and CBOWtheoryRead question + answer
Q1d [not printed individually]Unit 6 · Pragmatics and discoursetheoryRead question + answer
Q2a [6]Unit 2, 4 · Six preprocessing operationstheoryRead question + answer
Q2b [4]Unit 2, 5 · Lexical similarity and cosinemixedRead question + answer
Q2c [10]Unit 7 · Animal categorical Naive BayesnumericalRead question + answer
Q3a [10]Unit 6 · Hobbs and ambiguous professor–student referencecaseRead question + answer · Pronoun referent is ambiguous without extra gender/discourse information; structural algorithm preference is separated from semantic interpretation.
Q3b [10]Unit 7 · Classification and sentiment in industrytheoryRead question + answer
Q4a [10]Unit 4 · Top-down syntax treecaseRead question + answer
Q4b [10]Unit 3 · Data acquisition and four augmentation methodstheoryRead question + answer
Q5a [10]Unit 2 · Feature engineering, BoW and EuclideannumericalRead question + answer
Q5b [5]Unit 5 · WSD approaches and usestheoryRead question + answer
Q5c [5]Unit 6 · Discourse analysis of a conversationcaseRead question + answer
Q6a [10]Unit 3 · Henry bigram continuationsnumericalRead question + answer · Case is retained as printed: Do and do are distinct. The corpus is not silently grammatically corrected.
Q6b [5]Unit 1 · Five linguistic levelstheoryRead question + answer
Q6c [5]Unit 1 · NLU and NLG working togethertheoryRead question + answer
Q7a [10]Unit 4 · POS, chunks and boundary-aware HMMmixedRead question + answer · Printed tags are used as training data; START and END are explicitly required and included.
Q7b [10]Unit 5 · Co-occurrence, dot products and plotnumericalRead question + answer

Original offered marks by mapped topic, grouped without double counting: Unit 2, 7: 1 parts, 0 known marks + 1 parts with no individual mark printed; Unit 4: 3 parts, 20 known marks + 1 parts with no individual mark printed; Unit 5: 3 parts, 15 known marks + 1 parts with no individual mark printed; Unit 6: 3 parts, 15 known marks + 1 parts with no individual mark printed; Unit 2, 4: 1 parts, 6 known marks; Unit 2, 5: 1 parts, 4 known marks; Unit 7: 2 parts, 20 known marks; Unit 3: 2 parts, 20 known marks; Unit 2: 1 parts, 10 known marks; Unit 1: 2 parts, 10 known marks. Joint-unit buckets remain joint; no invented split is assigned. These original-paper totals include any legacy tasks explicitly marked above; they are not a current-syllabus marks blueprint.

2024-02-12 · re-exam · R24

140 marks offered; 100 attempted.

Question [marks]Current unit and topicQuestion formAnswer / scope
Q1a [5]Unit 6 · Anaphora versus cataphoratheoryRead question + answer
Q1b [5]Unit 3 · Five corporatheoryRead question + answer
Q1c [5]Unit 1 · Linguistic foundationstheoryRead question + answer
Q1d [5]Unit 3 · Five regex operatorstheoryRead question + answer
Q2a [10]Unit 3 · Wordlist puzzle and lexical resourcemixedRead question + answer · The dictionary is unspecified; an exact number of valid words would be fabricated.
Q2b [10]Unit 1, 7 · NLP applications and limitationstheoryRead question + answer
Q3a [10]Unit 4 · Top-down restaurant parsecaseRead question + answer · Original CFG is incomplete for the given bare-plural object. Exact failure and explicit repaired derivation are both shown.
Q3b [10]Unit 2 · Preprocessing pipelinetheoryRead question + answer
Q4a [10]Unit 5 · Embeddings and Skip-gramtheoryRead question + answer
Q4b [10]Unit 2, 4 · POS, NER, chunking and TTRtheoryRead question + answer
Q5a [10]Unit 5 · Lexical and vector semanticstheoryRead question + answer
Q5b [10]Unit 7 · Classification label structurestheoryRead question + answer
Q6a [10]Unit 2, 7 · TF–IDF in document representationmixedRead question + answer · The prompt requests an example but supplies no corpus. Numbers in the explanation are authored illustrations, not original numerical givens.
Q6b [10]Unit 4 · HMM tables and pathnumericalRead question + answer
Q7a [10]Unit 7 · Healthcare and finance applicationstheoryRead question + answer
Q7b [10]Unit 7 · Naive Bayes sentimentnumericalRead question + answer

Original offered marks by mapped topic, grouped without double counting: Unit 6: 1 parts, 5 known marks; Unit 3: 3 parts, 20 known marks; Unit 1: 1 parts, 5 known marks; Unit 1, 7: 1 parts, 10 known marks; Unit 4: 2 parts, 20 known marks; Unit 2: 1 parts, 10 known marks; Unit 5: 2 parts, 20 known marks; Unit 2, 4: 1 parts, 10 known marks; Unit 7: 3 parts, 30 known marks; Unit 2, 7: 1 parts, 10 known marks. Joint-unit buckets remain joint; no invented split is assigned. These original-paper totals include any legacy tasks explicitly marked above; they are not a current-syllabus marks blueprint.

2023-12-01 · final · F23

120 marks offered; 100 attempted. Original has six main questions on three pages: Q1 compulsory and any four of Q2–Q6. Offered total 120, attempted 100.

Question [marks]Current unit and topicQuestion formAnswer / scope
Q1a [5]Unit 5, 6 · WSD and pronoun resolutiontheoryRead question + answer
Q1b [5]Unit 5 · WordNet and its hierarchytheoryRead question + answer
Q1c [5]Unit 2, 7 · Sentiment feature extractionnumericalRead question + answer
Q1d [5]Unit 7 · Binary, multiclass and multilabeltheoryRead question + answer
Q2a [10]Unit 3 · Text corpus structure and applicationstheoryRead question + answer
Q2b [10]Unit 5 · Lesk versus Walker and bank disambiguationmixedRead question + answer · The original gives a sentence but no thesaurus/category mapping, so a unique numerical score cannot be recovered.
Q3a [10]Unit 6 · Hobbs algorithm and limitationstheoryRead question + answer
Q3b [10]Unit 2 · Stemming and lemmatizationtheoryRead question + answer
Q4a [10]Unit 5 · Lexical and vector semanticstheoryRead question + answer
Q4b [10]Unit 2 · Three-document TF–IDFnumericalRead question + answer
Q5a [10]Unit 5 · Word embeddings and CBOWtheoryRead question + answer
Q5b [10]Unit 4 · HMM tables and given pathnumericalRead question + answer
Q6a [10]Unit 7 · Text classification pipelinetheoryRead question + answer
Q6b [10]Unit 2, 7 · VSM, TF and IDF in retrievaltheoryRead question + answer

Original offered marks by mapped topic, grouped without double counting: Unit 5, 6: 1 parts, 5 known marks; Unit 5: 4 parts, 35 known marks; Unit 2, 7: 2 parts, 15 known marks; Unit 7: 2 parts, 15 known marks; Unit 3: 1 parts, 10 known marks; Unit 6: 1 parts, 10 known marks; Unit 2: 2 parts, 20 known marks; Unit 4: 1 parts, 10 known marks. Joint-unit buckets remain joint; no invented split is assigned. These original-paper totals include any legacy tasks explicitly marked above; they are not a current-syllabus marks blueprint.

2023-01-30 · re-exam · R23

140 marks offered; 100 attempted. Q1 header is unclear/conflicting [14/20], but all four parts print 5 marks each; sum=20. Main paper is 100 attempted marks.

Question [marks]Current unit and topicQuestion formAnswer / scope
Q1a [5]Unit 2 · Normalization changes vocabularytheoryRead question + answer
Q1b [5]Unit 3 · Good corpus characteristicstheoryRead question + answer
Q1c [5]Unit 5 · Word2Vec versus Doc2VectheoryRead question + answer · legacy named-method extension; not separately named in current syllabus. Optional historical extension; not a separately prescribed current-syllabus method.
Q1d [5]Unit 1, 4 · Ambiguity in NLPtheoryRead question + answer
Q2a [10]Unit 2 · Normalize the Tanisha–Simran passagecaseRead question + answer · Stoplist and stemming algorithm are choices, not supplied numerical givens; both are stated.
Q2b [10]Unit 2, 5 · Text vectorization alternativestheoryRead question + answer
Q3a [10]Unit 1, 5, 6 · Semantics, pragmatics and two ambiguitiestheoryRead question + answer
Q3b [10]Unit 7 · Naive Bayes text classifiermixedRead question + answer · Chinese corpus is an authored/reused teaching illustration here; it is actual given data in F22 Q5b, not this prompt.
Q4a [10]Unit 2 · Jaccard, cosine and Euclidean without preprocessingnumericalRead question + answer
Q4b [10]Unit 5 · CBOW and Skip-gramtheoryRead question + answer
Q5a [10]Unit 3 · Password regexcaseRead question + answer
Q5b [10]Unit 7 · Two NLP applicationstheoryRead question + answer
Q6a [10]Unit 6 · Mention-pair modeltheoryRead question + answer
Q6b [10]Unit 7 · Classification evaluation including ROC/AUCmixedRead question + answer
Q7a [10]Unit 4 · Organization and facility mentionscaseRead question + answer · Boundary and generic-mention decisions depend on the provided ACE annotation convention; the answer distinguishes them from a named-only interpretation.
Q7b [10]Unit 4 · Tagging and two complete parse treescaseRead question + answer

Original offered marks by mapped topic, grouped without double counting: Unit 2: 3 parts, 25 known marks; Unit 3: 2 parts, 15 known marks; Unit 5: 2 parts, 15 known marks; Unit 1, 4: 1 parts, 5 known marks; Unit 2, 5: 1 parts, 10 known marks; Unit 1, 5, 6: 1 parts, 10 known marks; Unit 7: 3 parts, 30 known marks; Unit 6: 1 parts, 10 known marks; Unit 4: 2 parts, 20 known marks. Joint-unit buckets remain joint; no invented split is assigned. These original-paper totals include any legacy tasks explicitly marked above; they are not a current-syllabus marks blueprint.

Current-syllabus qualification: Q1c: historical named-method extension, excluded from current-only priority totals. The original whole-part marks remain visible so the paper is faithfully represented.

2022-11-22 · final · F22

140 marks offered; 100 attempted.

Question [marks]Current unit and topicQuestion formAnswer / scope
Q1a [5]Unit 1 · Need and difficulties of NLPtheoryRead question + answer
Q1b [5]Unit 1 · Levels of language analysistheoryRead question + answer
Q1c [5]Unit 7 · Medical chatbottheoryRead question + answer
Q1d [5]Unit 7 · Sentiment with logistic regressionmixedRead question + answer
Q2a [10]Unit 5 · CBOW and Skip-gram architecturetheoryRead question + answer
Q2b [10]Unit 3 · Seven regex constructionscaseRead question + answer
Q3a [10]Unit 2 · Text preprocessingtheoryRead question + answer
Q3b [10]Unit 4 · Penn Treebank taggingcaseRead question + answer · Context-sensitive tags and possessive tokenization are explicitly stated; reasonable corpus guidelines can differ on noun adjuncts.
Q4a [10]Unit 6 · Mention-pair coreferencetheoryRead question + answer
Q4b [10]Unit 7 · Text classification pipelinetheoryRead question + answer
Q5a [10]Unit 3 · Corpora and structuretheoryRead question + answer
Q5b [10]Unit 7 · Multinomial Naive BayesnumericalRead question + answer
Q6a [10]Unit 5 · GloVetheoryRead question + answer
Q6b [10]Unit 2 · BoW and TF–IDF movie corpusnumericalRead question + answer
Q7a [10]Unit 4, 7 · Information extraction, chunking and NERtheoryRead question + answer
Q7b [10]Unit 5 · Distributional hypothesis and cosinemixedRead question + answer

Original offered marks by mapped topic, grouped without double counting: Unit 1: 2 parts, 10 known marks; Unit 7: 4 parts, 30 known marks; Unit 5: 3 parts, 30 known marks; Unit 3: 2 parts, 20 known marks; Unit 2: 2 parts, 20 known marks; Unit 4: 1 parts, 10 known marks; Unit 6: 1 parts, 10 known marks; Unit 4, 7: 1 parts, 10 known marks. Joint-unit buckets remain joint; no invented split is assigned. These original-paper totals include any legacy tasks explicitly marked above; they are not a current-syllabus marks blueprint.

Priority without skipping the syllabus

The latest papers show the form and depth of questions, while the official current syllabus sets the learning scope. Authored drills, explanatory numbers and optional extensions are not counted as past questions. Source assumptions and ambiguous data are stated in their answers.

"}]},{"id":"scope","title":"Portion, paper pattern & sources","unit":0,"sections":[{"id":"sources","title":"What this NLP guide covers","html":"

M2 portion confirmed by you: Unit 3 from PDF slide 62 to 160; Unit 4 complete (128 slides); Unit 5 slides 1–57. Slide 62 of Unit 3 starts N-grams. Slide 57 of Unit 5 completes the Naive Bayes WSD example. Vector-space models start on slide 58 and belong to TEE here.

M2 pattern confirmed by you: both 2-mark questions compulsory; any two of the three 3-mark questions. Total attempted 10. Course policy gives one hour. Three newly authored, fully solved M2 practice papers use that pattern.

TEE: all seven units of the official AY 2026–27 syllabus. The new ZIP supplies faculty slide packs for Units 1–5 plus a regular-expressions handout. Units 6–7 are syllabus-based explanations supplemented with the standard references below; their faculty slide packs were not supplied. The official syllabus includes GloVe and spelling correction, and both are retained even though the policy’s lecture table abbreviates them.

Previous papers found: final exam dated 2 December 2025 and re-exam dated 9 February 2026, with answer keys, inside Semester V.zip. Programme copies were deduplicated. Every supplied question subpart is mapped to the current syllabus; no question was excluded as out of syllabus. Only wholly M2-eligible subparts appear in the M2 paper tab; mixed questions remain in the complete TEE papers.

The current syllabus lists course code 702AI0C037. The supplied policy and old papers use 702AI0C009; the policy also retains an older programme/semester header. Topic coverage follows the current official syllabus and your exact M2 cutoff, rather than treating the old header as a different course.

Conventions that must stay visible

Course bigram answer keys count both start and end symbols in vocabulary size. Worked PYQs state and follow that convention; other exercises state their own vocabulary explicitly. A start symbol is conditioning context, so alternative textbook conventions may exclude it from prediction vocabulary and produce a different smoothed number. HMM path scores and Naive Bayes class scores are labelled as unnormalised when applicable. Any normalisation, case-folding or lemmatisation assumption is stated before counting.

Reading references

Examples are teaching problems unless explicitly identified by their paper and question number. Practice-paper marks indicate intended answer length, not an official forecast. Reading alone cannot guarantee a score; writing the solutions independently is what makes the procedures usable under exam time.

TEE depth review · 10 October 2026

The core TEE route retains the faculty topics CYK/CKY, grammar conversion, interpolation and Stupid Backoff. Add-k and HMM forward/posterior calculations are available separately under Optional extensions; they are not part of the core reading or formula sequence. The two TEE timed-practice entries reproduce the supplied actual papers and are not additional unseen mock papers.

Method references: Jurafsky and Martin: Hidden Markov Models; Jurafsky and Martin: Constituency Parsing. New numerical inputs and worked traces are teaching examples.

"},{"id":"videos","title":"Video companions in your NotebookLM","html":"

Your M2 notebook contains the exact three slide ranges and checked numerical teaching examples. The requested video topics are N-grams and Laplace smoothing, HMM/Viterbi, and Naive Bayes WSD. Use the written solutions here for the complete calculations.

Open your NLP M2 notebook →

The notebook is in your signed-in Google account. Video generation and availability are shown in its Studio panel. Generated narration is a supplementary explanation; the numerical results in this guide have been independently checked.

"}],"scope":"NLP · M2 and TEE · edition 3 October 2026"},{"id":"optional-extensions","title":"Optional extensions — after the TEE questions","unit":"Reference","scope":"Additional explanation; no corresponding question in the two supplied TEE papers. No exam marks are implied.","sections":[{"id":"add-k","title":"Add-k: preserve a normalized vocabulary row","html":"

Optional extension. Add-k is a supporting generalization of Laplace smoothing. The supplied TEE papers ask add-one bigram sentence probabilities, not an add-k calculation. Complete the actual solved papers first.

Add-k smoothing generalizes add-one smoothing. Add the same positive pseudo-count k to each allowed outcome. There are V outcomes, so the total pseudo-count is kV. Setting k=1 recovers Laplace smoothing. A smaller k usually changes the original counts less; it is not automatically the best choice for every dataset.

\"P_k(w\\mid

Use one fixed vocabulary, including an explicit unknown-word policy if needed. Choose k from training/validation evidence, not by fitting the final test results. Unigram smoothing uses N, the total token count, in place of C(h).

Question. After history h, counts for the complete next-token vocabulary [a,b,c] are [3,1,0]. Calculate add-k probabilities for k=0.5 and check the row sum. [Authored worked example]

Answer

Given

C(h)=4; V=3; k=1/2; the zero-count word c is an allowed outcome.

To find

All three probabilities and their interpretation.

Formula and why it applies

\"P_k(w\\mid

Smoothing must adjust the denominator for every pseudo-count added across the row.

Solution

Step 1. Form the new total
\"4+(0.5)(3)=5.5.\"
Step 2. Calculate a
\"P(a\\mid
Step 3. Calculate b and c separately
\"P(b\\mid
\"P(c\\mid
Step 4. Check normalization
\"\\frac7{11}+\\frac3{11}+\\frac1{11}=1.\"
Final answer. The row is [7/11,3/11,1/11]. The unseen allowed outcome c receives positive probability, and the row remains a probability distribution.
"},{"id":"forward-probability","title":"Forward HMM: sum paths and normalize posteriors","html":"

Optional extension. Forward totals and tag-path posteriors are optional HMM extensions here. Both supplied TEE papers ask emission and transition matrices, then the joint probability of a given tag sequence. Complete the actual solved papers first.

The forward algorithm finds the total probability of observations when the hidden path is unknown. Its cell αₜ(j) is the joint probability of the observed prefix o₁…oₜ and ending in hidden state j, summed over every earlier hidden path. It stores two totals per time step in the two-state example; it does not choose one winning predecessor.

\"\\alpha_1(j)=\\pi_jb_j(o_1)\"
\"\\alpha_t(j)=b_j(o_t)\\sum_i\\alpha_{t-1}(i)a_{ij}\"
\"P(O)=\\sum_j\\alpha_T(j).\"

πⱼ is the first-state probability; aᵢⱼ is transition i→j; bⱼ(oₜ) is the probability that state j emits the current observation. Multiply each earlier total by its transition, add all incoming alternatives, then multiply by the current emission. With explicit END transitions, terminate using Σⱼ αT(j)aⱼ,END instead. Do not invent END factors in a fixed-length model that has none.

QuestionOperationOutput
Given one hidden path and the wordsMultiply its start, transitions and emissionsP(path, observations)
Find the best hidden pathViterbi: maximum, with backpointersBest path and its joint score
Find observation probabilityForward: sum incoming pathsP(observations)
Find a specified path posteriorDivide its joint score by P(observations)P(path | observations)

Question. Use the HMM tables below to compute P(1,3,1) with the forward algorithm. Compare with the Viterbi path H,H,C whose joint score is 0.00672, then obtain that path’s posterior. [Authored worked example]

Answer

Given

Use H,C state order and the tables shown. Three observations are 1,3,1. There is no END state.

QuantityHC
Initial π0.80.2
From H → next state0.70.3
From C → next state0.40.6
StateEmit 1Emit 2Emit 3
H0.20.40.4
C0.50.40.1

To find

Every forward cell, total observation probability, and P(H,H,C | 1,3,1).

Formula and why it applies

\"\\alpha_t(j)=b_j(o_t)\\sum_i\\alpha_{t-1}(i)a_{ij}\"

All hidden paths are mutually exclusive alternatives, so their scores must be added. Viterbi’s maximum answers a different question.

Solution

Step 1. Initialize observation 1
\"\\alpha_1(H)=0.8(0.2)=0.16,\\qquad\\alpha_1(C)=0.2(0.5)=0.10.\"
Step 2. Sum incoming paths to H at observation 3
\"\\alpha_2(H)=[0.16(0.7)+0.10(0.4)](0.4)\"
\"=[0.112+0.040](0.4)=0.0608.\"
Step 3. Sum incoming paths to C at observation 3
\"\\alpha_2(C)=[0.16(0.3)+0.10(0.6)](0.1)\"
\"=[0.048+0.060](0.1)=0.0108.\"
Step 4. Calculate final H at observation 1
\"\\alpha_3(H)=[0.0608(0.7)+0.0108(0.4)](0.2)\"
\"=[0.04256+0.00432](0.2)=0.009376.\"
Step 5. Calculate final C at observation 1
\"\\alpha_3(C)=[0.0608(0.3)+0.0108(0.6)](0.5)\"
\"=[0.01824+0.00648](0.5)=0.01236.\"
Step 6. Add the final-state alternatives
\"P(1,3,1)=0.009376+0.01236=0.021736.\"
Step 7. Read the table as a trace
State / time1: observe 12: observe 33: observe 1
H0.160.06080.009376
C0.100.01080.01236
Column total0.260.07160.021736

For each new cell, draw arrows from both cells in the previous column. Along each arrow multiply by its transition; add the two contributions and multiply by the destination’s emission.

Step 8. Normalize the specified best path
\"P(H,H,C\\mid1,3,1)=\\frac{0.00672}{0.021736}=\\frac{840}{2717}\\approx0.309164.\"

The best single path has only part of the total mass. The remaining paths account for the rest.

Final answer. P(observations)=0.021736. Viterbi selects H,H,C with joint score 0.00672; its posterior is about 0.309164, not 0.00672.

Question. With the same explicitly listed HMM, find the probability of observations 3,1 by forward calculation and by enumerating all four hidden paths. [Authored worked example]

Answer

Given

π=(0.8,0.2); transition rows H=(0.7,0.3), C=(0.4,0.6); emissions H=(0.2,0.4,0.4), C=(0.5,0.4,0.1) for labels 1,2,3; no END factor.

To find

P(3,1), and an independent path-sum check.

Formula and why it applies

Forward reuses prefix totals. Exhaustive enumeration is practical for this two-position example and must give the same total.

Solution

Step 1. Initialize observation 3
\"\\alpha_1(H)=0.8(0.4)=0.32,\\quad\\alpha_1(C)=0.2(0.1)=0.02.\"
Step 2. Calculate the second H cell
\"\\alpha_2(H)=[0.32(0.7)+0.02(0.4)](0.2)=0.0464.\"
Step 3. Calculate the second C cell
\"\\alpha_2(C)=[0.32(0.3)+0.02(0.6)](0.5)=0.054.\"
Step 4. Sum final cells
\"P(3,1)=0.0464+0.054=0.1004.\"
Step 5. Enumerate each complete path
PathFull multiplicationJoint score
HH0.8 × 0.4 × 0.7 × 0.20.0448
HC0.8 × 0.4 × 0.3 × 0.50.048
CH0.2 × 0.1 × 0.4 × 0.20.0016
CC0.2 × 0.1 × 0.6 × 0.50.006
\"0.0448+0.048+0.0016+0.006=0.1004.\"
Final answer. Both methods give 0.1004. The largest single-path score is 0.048, which is smaller than the sum.

For K states and T observations, both dense forward and Viterbi recurrences take O(TK²) work. Forward needs only the preceding column for the probability total. Viterbi also stores backpointers to recover the path. Very long sequences need scaling or log-sum-exp for forward; ordinary logarithms alone do not turn a sum into a maximum.

Yaad rakho: Viterbi = best raasta. Forward = saare raaston ka total. This is a supporting HMM extension, not an assertion that a separate forward-algorithm question appeared in the supplied papers.

"}]}]}},"routeAliases":{"tee/read/u3-smoothing-extensions/add-k":"tee/about/optional-extensions/add-k","tee/formulas/f-deep-u3-smoothing-extensions/add-k":"tee/about/optional-extensions/add-k","tee/read/u4-forward-cky/forward-probability":"tee/about/optional-extensions/forward-probability","tee/formulas/f-deep-u4-forward-cky/forward-probability":"tee/about/optional-extensions/forward-probability"}}