# Natural Language Processing — all public teaching content Exact source JSON: https://deep-learning-study-room.vercel.app/exports/nlp.json --- ## M2 · What NLP is doing Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/read/foundations--language/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/read/foundations/language Prerequisites for M2 · no earlier NLP knowledge assumed **Natural Language Processing (NLP)** is the study of computational methods for analysing, interpreting and producing human language. An NLP system receives text or speech and produces an output such as a tag, a meaning, a translation or a probability. For M2, the central question is often: **how do counts from a training corpus become probabilities, and how do those probabilities help choose a sentence, a tag or a word sense?** You will learn the vocabulary first, then do the arithmetic with complete working. Training textexamples already observed → Countshow often things occur → Probabilitiescounts divided by totals → Decisioncompare possible answers | Term | Meaning | Concrete example | | --- | --- | --- | | Corpus | A collection of language data. The plural is corpora. | Three sentences saved together form a tiny corpus. | | Document | One text item within a collection. | One review or one article. | | Token | One occurrence produced by tokenization. | “cats like cats” has 3 word tokens. | | Type | A distinct token value. | The same text has 2 types: cats and like. | | Vocabulary | The set of token types recognised by the model. | V = {cats, like}; vocabulary size is 2. | | Training | Estimating rules or numbers using examples. | Count how often like follows cats. | | Test | Applying the learned model to a new example. | Calculate a probability for a sentence not used to build the counts. | ### Three levels you will repeatedly see | Level | Question it answers | Example | | --- | --- | --- | | Word sequence | Which word is likely next? | After “drink”, is “water” likely? | | Syntax | What role or structure does the word have? | “book” is a noun in “a book” and a verb in “book tickets”. | | Semantics | Which meaning is intended? | “bank” near “deposit” means a financial institution. | **Yaad rakho:** Word ka order = language model; word ka role = POS; word ka matlab = sense. ### Your M2 route Read the probability refresher below, then **N-grams and smoothing → POS/HMM → parsing/chunking/NER → word relations and WSD**. Answers sit directly below their questions. The separate Formulas tab is for revision after learning the methods. Both 2-mark questions are compulsory; attempt any two of the three 3-mark questions. Total attempted = 10 marks in one hour. M2 scope: Unit 3 slides 62–160; Unit 4 slides 1–128; Unit 5 slides 1–57. Earlier-unit material in this short opening chapter is included only to supply prerequisites. ### M2 01 — NLP basics, counts and conditional probability id : nlp-m2-01-foundations-counts-probability title : M2 01 — NLP basics, counts and conditional probability title\_confirmed : false topic\_title : NLP basics, counts and conditional probability mode : m2 kind : focused status : queued duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — Probability and Bayes | Reviewed step-by-step numericals id : overview-probability-bayes title : NLP M2 — Probability and Bayes | Reviewed step-by-step numericals title\_confirmed : true topic\_title : NLP M2 — Probability and Bayes | Reviewed step-by-step numericals mode : m2 kind : supplemental status : available duration : 8:53 reviewed : true notebook\_url : youtube\_url corrected\_visuals : true upload\_pending\_correction : true correction\_note : Complete HMM and Naive Bayes calculations; clarified joint scores versus model posteriors. Full transcript and detected slides checked; short voice corrections included. The corrected YouTube upload is still processing. related\_topic\_ids : - nlp-m2-01-foundations-counts-probability - nlp-m2-02-bayes-generative-discriminative These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## M2 · Tokens, counts and symbols Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/read/foundations--notation/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/read/foundations/notation Prerequisites for M2 · no earlier NLP knowledge assumed A formula is a short way of stating a calculation. First name the quantities; only then insert the numbers. In this guide, C means a count and P means a probability. | Notation | Read it as | What to do | | --- | --- | --- | | $$ C(\text{cat})=3 $$ | Count of cat is three | Count every occurrence, including repeats. | | $$ P(\text{cat}) $$ | Probability of cat | A number from 0 to 1. | | $$ P(\text{cat}\mid\text{the}) $$ | Probability of cat given the | Restrict attention to occasions where the history is “the”. | | $$ \sum_i x_i $$ | Sum of the x values | Add the terms: x₁ + x₂ + … | | $$ \prod_i x_i $$ | Product of the x values | Multiply the terms: x₁ × x₂ × … | | $$ \arg\max_c s(c) $$ | The choice c with the largest score | Return a label, not the numeric score. | | $$ \max_c s(c) $$ | Largest score | Return a number. | ### Example 1 · tokens and types For the corpus “we study nlp” and “we like nlp”, find the token count, vocabulary and count of “we”. **Given** Two lowercase sentences. Ignore punctuation; do not add boundary markers. **To find** Token count N, vocabulary V, vocabulary size |V| and C(we). ### Formula and why it applies A token is an occurrence; a type is a distinct word. Add sentence lengths to count tokens; remove duplicates only when finding the vocabulary. ### Solution #### Step 1. List every occurrence we, study, nlp, we, like, nlp. #### Step 2. Count tokens $$ N=3+3=6 $$ #### Step 3. Keep one copy of each type V = {we, study, nlp, like}; |V| = 4. #### Step 4. Count the requested word “we” appears once in each sentence, hence C(we) = 2. N = 6; |V| = 4; C(we) = 2. **Yaad rakho:** Token mein repeat gin; type mein duplicate hata. ### Example 2 · sum and product Given probabilities 0.5, 0.2 and 0.1, compute their sum and their product. **Given** Three numbers: 0.5, 0.2, 0.1. **To find** Sum and product; recognise that these are different operations. ### Formula and why it applies $$ \sum_{i=1}^{3}p_i=p_1+p_2+p_3 $$ $$ \prod_{i=1}^{3}p_i=p_1p_2p_3 $$ Add for a sum. Multiply for a product. A sentence-chain calculation normally uses the product of its conditional probabilities. ### Solution #### Step 1. Add the first two numbers $$ 0.5+0.2=0.7 $$ #### Step 2. Complete the sum $$ 0.7+0.1=0.8 $$ #### Step 3. Multiply the first two numbers $$ 0.5\times0.2=0.1 $$ #### Step 4. Complete the product $$ 0.1\times0.1=0.01 $$ Sum = 0.8; product = 0.01. ### M2 01 — NLP basics, counts and conditional probability id : nlp-m2-01-foundations-counts-probability title : M2 01 — NLP basics, counts and conditional probability title\_confirmed : false topic\_title : NLP basics, counts and conditional probability mode : m2 kind : focused status : queued duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — Probability and Bayes | Reviewed step-by-step numericals id : overview-probability-bayes title : NLP M2 — Probability and Bayes | Reviewed step-by-step numericals title\_confirmed : true topic\_title : NLP M2 — Probability and Bayes | Reviewed step-by-step numericals mode : m2 kind : supplemental status : available duration : 8:53 reviewed : true notebook\_url : youtube\_url corrected\_visuals : true upload\_pending\_correction : true correction\_note : Complete HMM and Naive Bayes calculations; clarified joint scores versus model posteriors. Full transcript and detected slides checked; short voice corrections included. The corrected YouTube upload is still processing. related\_topic\_ids : - nlp-m2-01-foundations-counts-probability - nlp-m2-02-bayes-generative-discriminative These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## M2 · Probability, without skipped steps Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/read/foundations--probability/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/read/foundations/probability Prerequisites for M2 · no earlier NLP knowledge assumed **Empirical probability** estimates how likely an event is from its observed frequency. Divide the number of relevant observations by the total number of observations in the selected group. For a conditional probability, the denominator is the count of the condition, not the size of the entire corpus. $$ P(A)=\frac{C(A)}{N} $$ $$ P(B\mid A)=\frac{C(A,B)}{C(A)} $$ Here A is the conditioning event, B is the event being predicted, C(A,B) counts observations containing both in the required relationship, and N is the total number of observations. “Given A” tells you which denominator to use. Select the condition“given A” → Count matching outcomesC(A,B) → Divide by condition countC(A) ### Example 1 · word probability A corpus contains 20 tokens; “rain” occurs 5 times. Find the unigram probability of “rain”. **Given** N = 20 and C(rain) = 5. **To find** P(rain). ### Formula and why it applies $$ P(w)=\frac{C(w)}{N} $$ Unigram probability ignores neighbouring words. Every token in the corpus is one observation. ### Solution #### Step 1. Insert the count and total $$ P(\text{rain})=\frac{5}{20} $$ #### Step 2. Divide $$ \frac{5}{20}=0.25 $$ P(rain) = 0.25, or 25%. ### Example 2 · conditional probability In a corpus, “drink” is followed by another word 8 times. It is followed by “water” 6 times. Find P(water | drink). **Given** C(drink as a history) = 8; C(drink, water) = 6. **To find** Probability that the next word is water, given drink. ### Formula and why it applies $$ P(\text{water}\mid\text{drink})=\frac{C(\text{drink water})}{C(\text{drink})} $$ The condition is drink, so use the 8 drink-histories as the denominator. ### Solution #### Step 1. Identify the favourable continuations Six of the eight continuations are water. #### Step 2. Divide $$ P(\text{water}\mid\text{drink})=\frac68=0.75 $$ P(water | drink) = 0.75. **Yaad rakho:** Bar ke right mein jo hai, denominator uska count. ### Example 3 · multiply a sequence A model gives P(a | start) = 0.4, P(cat | a) = 0.5 and P(end | cat) = 0.2. Find the probability of the complete sentence “a cat”. **Given** Start → a = 0.4; a → cat = 0.5; cat → end = 0.2. **To find** Probability of the three-transition path. ### Formula and why it applies $$ P(\text{a cat, end})=P(\text{a}\mid\text{start})P(\text{cat}\mid\text{a})P(\text{end}\mid\text{cat}) $$ Every transition must happen in sequence, so multiply the model’s conditional probabilities. ### Solution #### Step 1. Write the three factors in order $$ P=0.4\times0.5\times0.2 $$ #### Step 2. Multiply the first pair $$ 0.4\times0.5=0.2 $$ #### Step 3. Multiply by the last factor $$ 0.2\times0.2=0.04 $$ Complete-sentence probability = 0.04. ### Example 4 · compare labels Two tag candidates have scores noun = 0.036 and verb = 0.048. Which tag does argmax return? If these are the only candidates, find the normalised noun probability. **Given** s(noun) = 0.036 and s(verb) = 0.048. **To find** Winning label and normalised noun probability. ### Formula and why it applies $$ \hat c=\arg\max_c s(c) $$ $$ P(c)=\frac{s(c)}{\sum_j s(j)} $$ A score need not sum to 1. Dividing each score by the sum gives a distribution over the candidates. ### Solution #### Step 1. Compare scores 0.048 > 0.036, so verb wins. #### Step 2. Find their total $$ 0.036+0.048=0.084 $$ #### Step 3. Divide the noun score by the total $$ P(\text{noun})=\frac{0.036}{0.084}=\frac37\approx0.428571 $$ argmax returns “verb”. The normalised noun probability is approximately 0.428571. ### M2 01 — NLP basics, counts and conditional probability id : nlp-m2-01-foundations-counts-probability title : M2 01 — NLP basics, counts and conditional probability title\_confirmed : false topic\_title : NLP basics, counts and conditional probability mode : m2 kind : focused status : queued duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — Probability and Bayes | Reviewed step-by-step numericals id : overview-probability-bayes title : NLP M2 — Probability and Bayes | Reviewed step-by-step numericals title\_confirmed : true topic\_title : NLP M2 — Probability and Bayes | Reviewed step-by-step numericals mode : m2 kind : supplemental status : available duration : 8:53 reviewed : true notebook\_url : youtube\_url corrected\_visuals : true upload\_pending\_correction : true correction\_note : Complete HMM and Naive Bayes calculations; clarified joint scores versus model posteriors. Full transcript and detected slides checked; short voice corrections included. The corrected YouTube upload is still processing. related\_topic\_ids : - nlp-m2-01-foundations-counts-probability - nlp-m2-02-bayes-generative-discriminative These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## M2 · Bayes’ rule: prior, evidence and answer Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/read/foundations--bayes/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/read/foundations/bayes Prerequisites for M2 · no earlier NLP knowledge assumed **Bayes’ rule** reverses a conditional probability. A word-sense classifier asks “which sense is most likely given these context words?” Training data more easily supplies “how likely are these words when this sense is used?” Bayes connects the two. $$ P(c\mid x)=\frac{P(x\mid c)P(c)}{P(x)} $$ | Symbol | Meaning | | --- | --- | | c | A candidate class, tag or word sense. | | x | The observed evidence, such as a context word. | | P(c) | Prior: how common the class is before seeing the evidence. | | P(x | c) | Likelihood: how compatible the evidence is with that class. | | P(c | x) | Posterior: probability of the class after observing the evidence. | | P(x) | Evidence normaliser, shared by all classes for the same observation. | For ranking classes, calculate **prior × likelihood**. The common denominator does not affect which class is largest. For an actual posterior probability, divide by the sum of all class scores. ### A complete Bayes calculation A word has two senses: financial bank and river bank. Priors are 0.6 and 0.4. The probability of observing “loan” is 0.5 under the financial sense and 0.1 under the river sense. Find the most likely sense and its posterior. **Given** P(F) = 0.6; P(R) = 0.4; P(loan | F) = 0.5; P(loan | R) = 0.1. These two senses are exhaustive. **To find** Scores and P(F | loan). ### Formula and why it applies $$ s(c)=P(c)P(\text{loan}\mid c) $$ $$ P(F\mid\text{loan})=\frac{s(F)}{s(F)+s(R)} $$ Multiply prior and likelihood to reverse the conditioning. Then normalise because a posterior was requested. ### Solution #### Step 1. Score the financial sense $$ s(F)=0.6\times0.5=0.30 $$ #### Step 2. Score the river sense $$ s(R)=0.4\times0.1=0.04 $$ #### Step 3. Add the scores $$ P(\text{loan})=0.30+0.04=0.34 $$ #### Step 4. Normalise the financial score $$ P(F\mid\text{loan})=\frac{0.30}{0.34}=\frac{15}{17}\approx0.882353 $$ Financial bank wins; its posterior probability is about 0.882353. **Yaad rakho:** Pehle se kitna common × evidence kitna suit karta = score. ### Bayes Rule & Models id : nlp-m2-02-bayes-generative-discriminative title : Bayes Rule & Models title\_confirmed : true topic\_title : Bayes rule and generative versus discriminative models mode : m2 kind : focused status : available duration : 10:51 reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — Probability and Bayes | Reviewed step-by-step numericals id : overview-probability-bayes title : NLP M2 — Probability and Bayes | Reviewed step-by-step numericals title\_confirmed : true topic\_title : NLP M2 — Probability and Bayes | Reviewed step-by-step numericals mode : m2 kind : supplemental status : available duration : 8:53 reviewed : true notebook\_url : youtube\_url corrected\_visuals : true upload\_pending\_correction : true correction\_note : Complete HMM and Naive Bayes calculations; clarified joint scores versus model posteriors. Full transcript and detected slides checked; short voice corrections included. The corrected YouTube upload is still processing. related\_topic\_ids : - nlp-m2-01-foundations-counts-probability - nlp-m2-02-bayes-generative-discriminative These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## M2 · Small probabilities, logs and calculator use Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/read/foundations--calculator/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/read/foundations/calculator Prerequisites for M2 · no earlier NLP knowledge assumed Repeated multiplication of probabilities produces small numbers. A calculator may show 0.0000125 as **1.25 × 10⁻⁵**; they are the same value. A logarithm turns a product into a sum. Log probabilities are often negative because probabilities between 0 and 1 have negative logs. $$ \log(ab)=\log a+\log b $$ $$ \log_b x=y\quad\Longleftrightarrow\quad b^y=x $$ Use the log base specified in the question. Natural log is ln (base e); ordinary calculator log is usually base 10. Perplexity is simplest to evaluate using the direct power formula. Do not mix bases in the same calculation. ### Calculator example · roots in perplexity A model assigns probability 1/16 to a test sequence with 4 predicted tokens. Calculate perplexity. **Given** P = 1/16; N = 4 predicted tokens. **To find** Perplexity PP. ### Formula and why it applies $$ PP=P^{-1/N}=\left(\frac1P\right)^{1/N} $$ Perplexity takes the reciprocal probability and then its Nth root. N must match the number of predicted events in the product. ### Solution #### Step 1. Take the reciprocal $$ \frac{1}{1/16}=16 $$ #### Step 2. Take the fourth root $$ PP=16^{1/4}=2 $$ #### Step 3. Interpret A perplexity of 2 means an effective average uncertainty comparable to two equally likely alternatives. It is not 2% error. PP = 2. ### Calculator entry order Enter the full product with parentheses. For the Nth root enter **(1 ÷ P)^(1 ÷ N)**. Keep at least 6 significant digits while working; round the final answer. If your calculator offers Ans, reuse the stored value instead of typing a heavily rounded result. **Yaad rakho:** Probability ke factors multiply; log mein add; perplexity mein ulta karke root. ### M2 06 — Perplexity, logarithms and calculator working id : nlp-m2-06-perplexity-logs-calculator title : M2 06 — Perplexity, logarithms and calculator working title\_confirmed : false topic\_title : Perplexity, logarithms and calculator working mode : m2 kind : focused status : queued duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — Perplexity | Reviewed formulas and solved numericals id : overview-perplexity-logs title : NLP M2 — Perplexity | Reviewed formulas and solved numericals title\_confirmed : true topic\_title : NLP M2 — Perplexity | Reviewed formulas and solved numericals mode : m2 kind : supplemental status : available duration : 9:35 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Readable formulas and all worked examples; corrected comparison conditions and context claims. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids : - nlp-m2-06-perplexity-logs-calculator These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## M2 · Words, counts and probabilities: the notation you need Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/read/m2-language-models--words-counts-probability/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/read/m2-language-models/words-counts-probability Unit 3, pages 62–108; essential probability revision **A token** is one occurrence of a word. A **type** is one distinct word. A **corpus** is a collection of texts used for analysis or training. In “cats chase cats”, there are three word tokens but only two word types: cats and chase. A language model learns regularities from such a corpus to assign probabilities to word sequences. It does not simply count whether a sentence is grammatically correct. | Symbol | Read it as | Concrete meaning | | --- | --- | --- | | C(cat) | count of cat | Number of times cat occurs. | | C(the, cat) | count of the followed by cat | Count adjacent ordered pairs; cat the is different. | | P(cat) | probability of cat | A number from 0 to 1. | | P(cat | the) | probability of cat given the | Among contexts ending in the, how often cat follows. | | V | vocabulary size | Number of distinct outcomes the model may predict. | | N | number of tokens or predictions | Its exact meaning must be stated for the calculation. | The vertical bar **| means “given”**. The condition is on its right. In P(cat | the), the model has already seen “the” and is predicting “cat”. This direction matters: P(cat | the) and P(the | cat) usually have different denominators. **Question. A corpus contains 20 word tokens. The word cat occurs 5 times. Find the maximum-likelihood unigram probability of cat.** #### Given Total tokens N = 20; count C(cat) = 5. #### To find P(cat). #### Formula and why it applies $$ P(w)=\frac{C(w)}{N} $$ Relative frequency estimates the probability of drawing this word from the corpus. The denominator includes every token, not just distinct words. #### Solution ##### Step 1. Select the matching count The required outcome is cat, which occurs 5 times. ##### Step 2. Divide by the total number of trials $$ P(\text{cat})=\frac{5}{20}=0.25 $$ ##### Step 3. Interpret the result Under this unigram model, cat receives probability 0.25, or 25%. **Final answer.** P(cat) = 0.25. **Yaad rakho:** Jo chahiye uska count upar; total possible trials neeche. **Question. The word the has 10 outgoing occurrences in a corpus. It is followed by cat on 4 occasions and dog on 6. Find P(cat | the) and P(dog | the).** #### Given C(the, cat) = 4; C(the, dog) = 6; C(the) = 10 outgoing contexts. #### To find Both next-word probabilities. #### Formula and why it applies $$ P(w\mid h)=\frac{C(h,w)}{C(h)} $$ The condition h = the restricts attention to its 10 continuations. We do not divide by the total corpus size. #### Solution ##### Step 1. Use the cat continuation $$ P(\text{cat}\mid\text{the})=\frac{4}{10}=0.4 $$ ##### Step 2. Use the dog continuation $$ P(\text{dog}\mid\text{the})=\frac{6}{10}=0.6 $$ ##### Step 3. Check the complete row $$ 0.4+0.6=1 $$ **Final answer.** P(cat | the) = 0.4 and P(dog | the) = 0.6. **Yaad rakho:** Given waala word denominator choose karata hai. **Question. A collection has 8 sentences; 6 start with I and 2 start with we. Estimate each starting-word probability.** #### Given Each sentence receives a start marker . C() = 8; C(, I) = 6; C(, we) = 2. #### To find P(I | ) and P(we | ). #### Formula and why it applies $$ P(w_1\mid\langle s\rangle)=\frac{C(\langle s\rangle,w_1)}{C(\langle s\rangle)} $$ The start marker supplies the context for the first word; one start context is available per sentence. #### Solution ##### Step 1. Calculate the first option $$ P(I\mid\langle s\rangle)=6/8=0.75 $$ ##### Step 2. Calculate the second option $$ P(we\mid\langle s\rangle)=2/8=0.25 $$ **Final answer.** The estimated probabilities are 0.75 and 0.25. ### M2 01 — NLP basics, counts and conditional probability id : nlp-m2-01-foundations-counts-probability title : M2 01 — NLP basics, counts and conditional probability title\_confirmed : false topic\_title : NLP basics, counts and conditional probability mode : m2 kind : focused status : queued duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — Probability and Bayes | Reviewed step-by-step numericals id : overview-probability-bayes title : NLP M2 — Probability and Bayes | Reviewed step-by-step numericals title\_confirmed : true topic\_title : NLP M2 — Probability and Bayes | Reviewed step-by-step numericals mode : m2 kind : supplemental status : available duration : 8:53 reviewed : true notebook\_url : youtube\_url corrected\_visuals : true upload\_pending\_correction : true correction\_note : Complete HMM and Naive Bayes calculations; clarified joint scores versus model posteriors. Full transcript and detected slides checked; short voice corrections included. The corrected YouTube upload is still processing. related\_topic\_ids : - nlp-m2-01-foundations-counts-probability - nlp-m2-02-bayes-generative-discriminative These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## M2 · Unigrams, bigrams and trigrams: construct them correctly Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/read/m2-language-models--n-grams-construction/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/read/m2-language-models/n-grams-construction Unit 3, pages 62–108; essential probability revision **Definition.** An n-gram is a contiguous sequence of n items. For word n-grams, the items are consecutive word tokens. A unigram contains one word; a bigram contains two; a trigram contains three. Character n-grams use characters instead, so always identify the unit being counted. Tokens: we | study | natural | language → Move a window one token at a time → Bigrams: we study / study natural / natural language For m tokens without padding, the number of n-gram *positions* is max(0, m−n+1). Repeated positions may contain the same n-gram type. A bag-of-n-grams representation counts each n-gram type and loses the overall sentence order beyond the local order inside each n-gram. $$ \text{Number of unpadded n-gram positions}=\max(0,m-n+1) $$ **Question. List all word unigrams, bigrams and trigrams for “the prime minister of our country”. Do not add boundary tokens.** #### Given Six tokens: the, prime, minister, of, our, country. n = 1, 2 and 3. #### To find All three lists and their counts. #### Formula and why it applies $$ \#\text{positions}=m-n+1 $$ Each window must contain consecutive words and move one position. No wraparound or invented final word is allowed. #### Solution ##### Step 1. Take one word at a time Unigrams: (the), (prime), (minister), (of), (our), (country). Count = 6−1+1 = 6. ##### Step 2. Take two consecutive words Bigrams: (the, prime), (prime, minister), (minister, of), (of, our), (our, country). Count = 6−2+1 = 5. ##### Step 3. Take three consecutive words Trigrams: (the, prime, minister), (prime, minister, of), (minister, of, our), (of, our, country). Count = 6−3+1 = 4. **Final answer.** 6 unigrams, 5 bigrams and 4 trigrams. **Question. Construct bigrams for “I like tea” using one start and one end marker.** #### Given Original tokens: I, like, tea. Add and once each. #### To find The boundary-aware bigram sequence. #### Formula and why it applies $$ \#\text{bigrams}=(m+2)-1=m+1 $$ The start context predicts the first word, and the last word predicts that the sentence ends. #### Solution ##### Step 1. Write the padded sequence I like tea ##### Step 2. Move a two-token window (, I), (I, like), (like, tea), (tea, ). ##### Step 3. Count the windows $$ 3+1=4 $$ **Final answer.** There are four boundary-aware bigrams. **Question. Construct trigrams for “I like tea” using two start markers and one end marker.** #### Given Original sentence has three words. For a trigram model each predicted token needs two prior context tokens. #### To find All trigram prediction events. #### Formula and why it applies $$ P(w_i\mid w_{i-2},w_{i-1}) $$ Two start markers provide a complete context even for the first word. #### Solution ##### Step 1. Pad with two starts I like tea ##### Step 2. Slide a three-token window (, , I), (, I, like), (I, like, tea), (like, tea, ). ##### Step 3. Count predicted events The model predicts I, like, tea and the end marker: four predictions. **Final answer.** Four trigrams. Start markers are context; they are not ordinary predicted words. **Question. Give the bag-of-bigrams counts for “a b a b” without padding.** #### Given Four tokens in their given order. #### To find Bigram positions, distinct bigram types and their counts. #### Formula and why it applies $$ \#\text{positions}=4-2+1=3 $$ A bag stores frequencies; repeated bigrams must not be discarded. #### Solution ##### Step 1. Enumerate all positions (a,b), (b,a), (a,b). ##### Step 2. Group identical pairs | Bigram type | Count | | --- | --- | | a b | 2 | | b a | 1 | **Final answer.** Three bigram positions but two distinct bigram types. **Yaad rakho:** n-gram mein n words; context mein n−1 words. ### NLP M2 03: N-grams & Unigrams id : nlp-m2-03-ngrams-language-models title : NLP M2 03: N-grams & Unigrams title\_confirmed : true topic\_title : N-grams, chain rule and unigram models mode : m2 kind : focused status : available duration : 9:52 reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — N-grams and Laplace | Reviewed worked numericals id : overview-ngrams-laplace title : NLP M2 — N-grams and Laplace | Reviewed worked numericals title\_confirmed : true topic\_title : NLP M2: N-Grams & Laplace mode : m2 kind : supplemental status : available duration : 7:41 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Complete counts and four Laplace factors; corrected formulas, conditional-factor wording and rounding. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids : - nlp-m2-03-ngrams-language-models - nlp-m2-04-bigram-trigram-tables - nlp-m2-05-laplace-smoothing-discount These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## M2 · Collocations and the purpose of a language model Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/read/m2-language-models--collocations-and-lm/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/read/m2-language-models/collocations-and-lm Unit 3, pages 62–108; essential probability revision **A collocation** is a conventional word combination whose words occur together more strongly than ordinary chance or independent frequencies would suggest. Examples include “strong tea”, “heavy rain” and “red wine”. A bigram is merely two consecutive items; it need not be a collocation. “of the” can be frequent because both words are frequent, while a collocation expresses a particularly strong association. | Term | What it represents | Example | | --- | --- | --- | | Bigram | Any adjacent two-token sequence | the book | | Collocation | A characteristic strongly associated expression | heavy rain | | Language model | A distribution used to score or predict sequences | P(rain | heavy) | **Definition.** A statistical language model assigns a probability to a sequence of words, or equivalently predicts the next word from preceding context. It is useful for autocomplete, speech recognition, machine translation, spelling correction, assistive communication and ranking candidate text. High probability reflects patterns learned from its training distribution; it is not a guarantee of truth or grammatical correctness. Previous words: please turn your homework → Candidate: in / over / elephant → Language model gives each candidate a probability → Application selects or ranks candidates **Noisy-channel connection.** Suppose x is the observed noisy text or speech and y is a candidate intended message. The noisy-channel approach chooses the y that maximizes P(x | y)P(y). The channel model P(x | y) asks how y could produce x; the language model P(y) asks whether y is a plausible message. P(x) is the same for all candidate y and therefore can be omitted when ranking. $$ \hat y=\arg\max_y P(y\mid x)=\arg\max_y P(x\mid y)P(y) $$ **Question. A spelling-correction system considers “flowers” and “flours”. For the observed input, the channel likelihoods are 0.2 and 0.4, and the respective language-model probabilities in context are 0.08 and 0.01. Which candidate does the noisy-channel rule choose?** #### Given Candidate flowers: P(x|flowers)=0.2, P(flowers)=0.08. Candidate flours: P(x|flours)=0.4, P(flours)=0.01. #### To find The larger unnormalized candidate score. #### Formula and why it applies $$ \operatorname{score}(y)=P(x\mid y)P(y) $$ Both the error mechanism and the language context matter. The same denominator would normalize both candidates. #### Solution ##### Step 1. Score flowers $$ 0.2\times0.08=0.016 $$ ##### Step 2. Score flours $$ 0.4\times0.01=0.004 $$ ##### Step 3. Compare $$ 0.016>0.004 $$ **Final answer.** Choose flowers. The scores are not yet posterior probabilities. **Yaad rakho:** Har word-pair bigram hai; har frequent pair meaningful collocation nahi. ### NLP M2 03: N-grams & Unigrams id : nlp-m2-03-ngrams-language-models title : NLP M2 03: N-grams & Unigrams title\_confirmed : true topic\_title : N-grams, chain rule and unigram models mode : m2 kind : focused status : available duration : 9:52 reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — N-grams and Laplace | Reviewed worked numericals id : overview-ngrams-laplace title : NLP M2 — N-grams and Laplace | Reviewed worked numericals title\_confirmed : true topic\_title : NLP M2: N-Grams & Laplace mode : m2 kind : supplemental status : available duration : 7:41 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Complete counts and four Laplace factors; corrected formulas, conditional-factor wording and rounding. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids : - nlp-m2-03-ngrams-language-models - nlp-m2-04-bigram-trigram-tables - nlp-m2-05-laplace-smoothing-discount These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## M2 · Probability chain rule and the Markov assumption Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/read/m2-language-models--chain-rule-markov/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/read/m2-language-models/chain-rule-markov Unit 3, pages 62–108; essential probability revision **The probability chain rule** decomposes the probability of a complete sequence into successive conditional probabilities. It is different from the derivative chain rule used in deep learning. Here we multiply probabilities, not derivatives. $$ P(w_1,w_2,w_3)=P(w_1)P(w_2\mid w_1)P(w_3\mid w_1,w_2) $$ Read left to right: probability of the first word; then probability of the second after the first; then probability of the third after both previous words. For a longer sentence, continue the same pattern. The product symbol ∏ simply means “multiply all of these terms”. $$ P(w_1,\ldots,w_m)=\prod_{i=1}^{m}P(w_i\mid w_1,\ldots,w_{i-1}) $$ **The Markov assumption** approximates long history by a limited recent context. A unigram model ignores preceding words. A bigram model keeps the previous word. A trigram model keeps the previous two words. This reduces the number of counts needed, but loses longer-range dependencies. | Model | Prediction rule | For the next word country after the prime minister of our | | --- | --- | --- | | Unigram | P(wᵢ) | P(country) | | Bigram | P(wᵢ | wᵢ₋₁) | P(country | our) | | Trigram | P(wᵢ | wᵢ₋₂,wᵢ₋₁) | P(country | of, our) | | Full chain rule | P(wᵢ | all previous words) | P(country | the, prime, minister, of, our) | **Question. For the three-word sentence “I like tea”, P(I)=0.2, P(like|I)=0.5 and P(tea|I,like)=0.4. Find its probability using the full chain rule.** #### Given Three conditional factors are given, including the full two-word history for tea. No end probability is requested. #### To find P(I, like, tea). #### Formula and why it applies $$ P(I,like,tea)=P(I)P(like\mid I)P(tea\mid I,like) $$ The chain rule supplies one factor per word in the specified sequence. #### Solution ##### Step 1. Insert the three supplied factors $$ P=0.2\times0.5\times0.4 $$ ##### Step 2. Multiply the first two $$ 0.2\times0.5=0.1 $$ ##### Step 3. Multiply by the third $$ 0.1\times0.4=0.04 $$ **Final answer.** The sentence probability is 0.04. **Question. A bigram model gives P(I|)=0.5, P(like|I)=0.6, P(tea|like)=0.25 and P(|tea)=0.8. Find the full sentence probability.** #### Given All four conditional probabilities, including the end event, are supplied. #### To find P(I like tea | ). #### Formula and why it applies $$ P=P(I\mid\langle s\rangle)P(like\mid I)P(tea\mid like)P(\langle/s\rangle\mid tea) $$ Under the bigram assumption, every factor uses one preceding token. #### Solution ##### Step 1. Insert the factors in sentence order $$ P=0.5\times0.6\times0.25\times0.8 $$ ##### Step 2. Start and second word $$ 0.5\times0.6=0.3 $$ ##### Step 3. Third word $$ 0.3\times0.25=0.075 $$ ##### Step 4. End the sentence $$ 0.075\times0.8=0.06 $$ **Final answer.** The complete sentence probability is 0.06. **Yaad rakho:** Sequence probability = saare required transitions ka product. **Typical 2-mark answer.** An n-gram language model predicts a word from its previous n−1 words using a Markov approximation. For example, a bigram model uses P(wᵢ | wᵢ₋₁). It estimates these probabilities from corpus counts and multiplies the appropriate factors to score a sentence. ### NLP M2 03: N-grams & Unigrams id : nlp-m2-03-ngrams-language-models title : NLP M2 03: N-grams & Unigrams title\_confirmed : true topic\_title : N-grams, chain rule and unigram models mode : m2 kind : focused status : available duration : 9:52 reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — N-grams and Laplace | Reviewed worked numericals id : overview-ngrams-laplace title : NLP M2 — N-grams and Laplace | Reviewed worked numericals title\_confirmed : true topic\_title : NLP M2: N-Grams & Laplace mode : m2 kind : supplemental status : available duration : 7:41 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Complete counts and four Laplace factors; corrected formulas, conditional-factor wording and rounding. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids : - nlp-m2-03-ngrams-language-models - nlp-m2-04-bigram-trigram-tables - nlp-m2-05-laplace-smoothing-discount These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## M2 · Unigram estimation and complete worked examples Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/read/m2-count-to-probability--unigram-estimation/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/read/m2-count-to-probability/unigram-estimation Unit 3, pages 97–120 and appendix **Maximum likelihood estimation (MLE)** chooses probabilities equal to observed relative frequencies. In a unigram model, every word is predicted independently of its surrounding words. If a word occurs c times in N training tokens, its estimate is c/N. To score m specified words, multiply their m unigram probabilities. $$ P_{\mathrm{MLE}}(w)=\frac{C(w)}{N}\qquad P(w_1,\ldots,w_m)\approx\prod_{i=1}^{m}P(w_i) $$ Choose a tokenization convention before counting. Lowercasing, punctuation handling and boundary markers can change the vocabulary and counts. Never count words with one convention and form the denominator with another. A unigram model gives the same product to rearrangements of the same word multiset; that is one of its limitations. **Question. Train an unpadded, case-insensitive unigram model on “I like tea” and “I like coffee”. Find the probability of “I like tea”.** #### Given Training sentences: I like tea; I like coffee. Use lowercase words; exclude punctuation and sentence boundaries. #### To find The vocabulary, word probabilities and test-sentence probability. #### Formula and why it applies $$ P(w)=C(w)/N,\qquad P(I\ like\ tea)=P(I)P(like)P(tea) $$ The unigram approximation ignores order and uses total token count for every denominator. #### Solution ##### Step 1. Count every training token Each sentence contains three words, so N = 3+3 = 6. ##### Step 2. Count types and frequencies | Word | Count | MLE probability | | --- | --- | --- | | i | 2 | 2/6 = 1/3 | | like | 2 | 2/6 = 1/3 | | tea | 1 | 1/6 | | coffee | 1 | 1/6 | Vocabulary size V = 4. ##### Step 3. Write one factor per test word $$ P=\frac{2}{6}\times\frac{2}{6}\times\frac{1}{6} $$ ##### Step 4. Multiply the fractions $$ P=\frac{4}{216}=\frac{1}{54}\approx0.018519 $$ **Final answer.** The unigram sentence probability is approximately 0.018519. **Question. A corpus contains counts a: 4, b: 3 and c: 1. Find the unigram probability of “a b a”.** #### Given C(a)=4, C(b)=3, C(c)=1. No boundary markers. #### To find P(a b a). #### Formula and why it applies $$ P(a\,b\,a)=P(a)P(b)P(a) $$ A repeated test word contributes a probability factor at every occurrence; do not include it just once. #### Solution ##### Step 1. Find the total $$ N=4+3+1=8 $$ ##### Step 2. Estimate the required probabilities $$ P(a)=4/8=1/2,\qquad P(b)=3/8 $$ ##### Step 3. Use both occurrences of a $$ P=\frac12\times\frac38\times\frac12 $$ ##### Step 4. Simplify $$ P=\frac3{32}=0.09375 $$ **Final answer.** P(a b a) = 0.09375. **Yaad rakho:** Repeated word ka probability bhi repeat multiply hoga. **Question. An unsmoothed unigram model is trained on “red blue red”. Find the probability of “red green”.** #### Given N=3. Counts: red=2, blue=1, green=0. The test contains red and green. #### To find The unsmoothed test probability and the reason for its value. #### Formula and why it applies $$ P(red\ green)=P(red)P(green) $$ MLE assigns zero to an unseen outcome. Smoothing is needed if an unseen in-vocabulary item must receive probability. #### Solution ##### Step 1. Estimate red $$ P(red)=2/3 $$ ##### Step 2. Estimate the unseen word $$ P(green)=0/3=0 $$ ##### Step 3. Multiply $$ (2/3)\times0=0 $$ **Final answer.** Unsmoothed probability = 0. This reflects missing training evidence, not proof that the phrase is impossible. ### NLP M2 03: N-grams & Unigrams id : nlp-m2-03-ngrams-language-models title : NLP M2 03: N-grams & Unigrams title\_confirmed : true topic\_title : N-grams, chain rule and unigram models mode : m2 kind : focused status : available duration : 9:52 reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — N-grams and Laplace | Reviewed worked numericals id : overview-ngrams-laplace title : NLP M2 — N-grams and Laplace | Reviewed worked numericals title\_confirmed : true topic\_title : NLP M2: N-Grams & Laplace mode : m2 kind : supplemental status : available duration : 7:41 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Complete counts and four Laplace factors; corrected formulas, conditional-factor wording and rounding. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids : - nlp-m2-03-ngrams-language-models - nlp-m2-04-bigram-trigram-tables - nlp-m2-05-laplace-smoothing-discount These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## M2 · Bigram counts: build the table before multiplying Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/read/m2-count-to-probability--bigram-count-table/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/read/m2-count-to-probability/bigram-count-table Unit 3, pages 97–120 and appendix A **bigram probability table** uses the previous word as its row and the next word as its column. Divide each cell count by its row's total outgoing count. With one end marker on every sentence, each ordinary training word has a continuation, so its outgoing count equals its token frequency. Do not create a bigram that crosses from one sentence into the next. $$ P(w\mid h)=\frac{C(h,w)}{\sum_{v\in V}C(h,v)}=\frac{C(h,w)}{C(h)} $$ Add boundaries to each sentence separately → List adjacent pairs → Count repeated pairs → Divide each row by its total → Follow the test sentence through the table **Question. Train a bigram model on “I like tea”, “I like coffee” and “you like tea”. Use one start and one end marker. Find the probability of “I like tea”.** #### Given Corpus: I like tea ; I like coffee ; you like tea . Preserve these tokens. #### To find All nonzero bigram estimates and the test probability. #### Formula and why it applies $$ P(w\mid h)=C(h,w)/C(h) $$ Every sentence starts and ends independently. Four probability factors are required for the three test words plus end. #### Solution ##### Step 1. List the training pairs Sentence 1: (,I), (I,like), (like,tea), (tea,). Sentence 2: (,I), (I,like), (like,coffee), (coffee,). Sentence 3: (,you), (you,like), (like,tea), (tea,). ##### Step 2. Create the nonzero count table | History → next | Pair count | History count | Probability | | --- | --- | --- | --- | | → I | 2 | 3 | 2/3 | | → you | 1 | 3 | 1/3 | | I → like | 2 | 2 | 1 | | you → like | 1 | 1 | 1 | | like → tea | 2 | 3 | 2/3 | | like → coffee | 1 | 3 | 1/3 | | tea → | 2 | 2 | 1 | | coffee → | 1 | 1 | 1 | ##### Step 3. Write the test path $$ P=P(I\mid\langle s\rangle)P(like\mid I)P(tea\mid like)P(\langle/s\rangle\mid tea) $$ ##### Step 4. Substitute each matching row $$ P=\frac23\times1\times\frac23\times1 $$ ##### Step 5. Multiply $$ P=\frac49\approx0.444444 $$ **Final answer.** P(I like tea) = 4/9 ≈ 0.444444. **Question. Using the same three-sentence corpus given here—“I like tea”, “I like coffee”, “you like tea”—find the bigram probabilities of “you like coffee” and “tea like I”. Include boundaries.** #### Given Corpus with one start/end marker per sentence. Counts: start→you=1/3; you→like=1; like→coffee=1/3; coffee→end=1. No training sentence starts with tea. #### To find The two test probabilities. #### Formula and why it applies $$ P(\text{sentence})=\prod P(\text{next token}\mid\text{previous token}) $$ A model can assign a nonzero probability to a sentence it never saw if all of its local bigrams were seen. #### Solution ##### Step 1. Score you like coffee $$ P=\frac13\times1\times\frac13\times1=\frac19\approx0.111111 $$ ##### Step 2. Inspect tea like I $$ P(tea\mid\langle s\rangle)=0/3=0 $$ ##### Step 3. Apply the zero factor The full product is zero, regardless of the remaining factors. **Final answer.** P(you like coffee) = 1/9; P(tea like I) = 0. A new sentence is not necessarily a zero-probability sentence. **Question. Obtain the unsmoothed bigram probability of “students are from Vellore” from this corpus: “I am from Vellore”; “I am a teacher”; “students are good and are from various cities”; “students from Vellore do engineering”. Use sentence boundaries.** #### Given Four training sentences, each padded with one and . Test sequence: students are from Vellore . #### To find The five required conditional probabilities and their product. #### Formula and why it applies $$ P(w\mid h)=C(h,w)/C(h) $$ Count all occurrences in the entire corpus. In particular, are occurs twice inside the third sentence, and Vellore is followed by end once and do once. #### Solution ##### Step 1. Count the histories | History | Occurrences with continuation | | --- | --- | | | 4 | | students | 2 | | are | 2 | | from | 3 | | Vellore | 2 | ##### Step 2. Count the required pairs | Required pair | Count | Probability | | --- | --- | --- | | students | 2 | 2/4 | | students are | 1 | 1/2 | | are from | 1 | 1/2 | | from Vellore | 2 | 2/3 | | Vellore | 1 | 1/2 | ##### Step 3. Write the complete product $$ P=\frac24\times\frac12\times\frac12\times\frac23\times\frac12 $$ ##### Step 4. Multiply progressively $$ \frac24\times\frac12=\frac14,\quad \frac14\times\frac12=\frac18 $$ $$ \frac18\times\frac23=\frac1{12},\quad \frac1{12}\times\frac12=\frac1{24} $$ **Final answer.** P = 1/24 ≈ 0.0416667. **Yaad rakho:** Same sentence mein word do baar aaye toh count bhi do. ### Bigram & Trigram Probs id : nlp-m2-04-bigram-trigram-tables title : Bigram & Trigram Probs title\_confirmed : true topic\_title : Build bigram and trigram probabilities from a corpus mode : m2 kind : focused status : available duration : 10:05 reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — N-grams and Laplace | Reviewed worked numericals id : overview-ngrams-laplace title : NLP M2 — N-grams and Laplace | Reviewed worked numericals title\_confirmed : true topic\_title : NLP M2: N-Grams & Laplace mode : m2 kind : supplemental status : available duration : 7:41 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Complete counts and four Laplace factors; corrected formulas, conditional-factor wording and rounding. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids : - nlp-m2-03-ngrams-language-models - nlp-m2-04-bigram-trigram-tables - nlp-m2-05-laplace-smoothing-discount These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## M2 · Trigram estimates: two-word history, not two denominators Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/read/m2-count-to-probability--trigram-estimation/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/read/m2-count-to-probability/trigram-estimation Unit 3, pages 97–120 and appendix A trigram model predicts the next word from the two preceding words. The numerator is the count of a triple. The denominator is the count of its two-word history with an available next token. Do not multiply two independent bigram probabilities and call that a trigram probability. $$ P(w_i\mid w_{i-2},w_{i-1})=\frac{C(w_{i-2},w_{i-1},w_i)}{C(w_{i-2},w_{i-1})} $$ **Question. A corpus contains 12 occurrences of “prime minister”. Eight are followed by “of”, three by “said”, and one by “visited”. Estimate the three trigram probabilities.** #### Given History h = (prime, minister). Count of this history = 12. Triple counts for of, said, visited are 8, 3, 1. #### To find The conditional distribution of the next word. #### Formula and why it applies $$ P(w\mid prime,minister)=C(prime,minister,w)/12 $$ All three candidates share one two-word history, so they share denominator 12. #### Solution ##### Step 1. Calculate of $$ P(of\mid prime,minister)=8/12=2/3 $$ ##### Step 2. Calculate said $$ P(said\mid prime,minister)=3/12=1/4 $$ ##### Step 3. Calculate visited $$ P(visited\mid prime,minister)=1/12 $$ ##### Step 4. Confirm normalization $$ 8/12+3/12+1/12=12/12=1 $$ **Final answer.** The probabilities are 2/3, 1/4 and 1/12. **Question. Train a trigram model on “I like tea”, “I like coffee” and “you like tea”. Use two start markers and one end marker. Score “I like tea”.** #### Given Three sentences. Padded first sentence is I like tea ; pad the other two in the same way. #### To find Four trigram prediction probabilities and the sentence probability. #### Formula and why it applies $$ P=\prod_i P(w_i\mid w_{i-2},w_{i-1}) $$ Each prediction uses a two-token context; the two start markers supply the first history. #### Solution ##### Step 1. Count the first-word event The history (,) occurs three times, and two are followed by I. $$ P(I\mid\langle s\rangle,\langle s\rangle)=2/3 $$ ##### Step 2. Count the second-word event (,I) occurs twice; both are followed by like. $$ P(like\mid\langle s\rangle,I)=2/2=1 $$ ##### Step 3. Count the third-word event (I,like) occurs twice; one is followed by tea. $$ P(tea\mid I,like)=1/2 $$ ##### Step 4. Count the end event (like,tea) occurs twice; both are followed by end. $$ P(\langle/s\rangle\mid like,tea)=2/2=1 $$ ##### Step 5. Multiply all four $$ P=\frac23\times1\times\frac12\times1=\frac13 $$ **Final answer.** Trigram P(I like tea) = 1/3. It differs from the bigram estimate because the context and denominator changed. **Question. A trigram model supplies P(we|,)=0.4, P(study|,we)=0.5, P(NLP|we,study)=0.8 and P(|study,NLP)=0.25. Score “we study NLP”.** #### Given Four trigram conditional factors. The end marker is included. #### To find The complete sentence probability. #### Formula and why it applies $$ P=P(we\mid s,s)P(study\mid s,we)P(NLP\mid we,study)P(end\mid study,NLP) $$ Use each supplied two-word history exactly; do not swap its word order. #### Solution ##### Step 1. Insert the values $$ P=0.4\times0.5\times0.8\times0.25 $$ ##### Step 2. Multiply the first pair $$ 0.4\times0.5=0.2 $$ ##### Step 3. Include NLP $$ 0.2\times0.8=0.16 $$ ##### Step 4. Include the end $$ 0.16\times0.25=0.04 $$ **Final answer.** P = 0.04. **Yaad rakho:** Trigram: teen-word count upar, pehle do-word count neeche. **Why larger n is not automatically better.** More context can capture local order more precisely, but each longer history occurs less often. Parameter count, storage and sparsity grow. A model trained on Shakespeare may assign poor probabilities to financial news because training and test distributions differ. Keep a held-out test set; do not evaluate only on memorized training sentences. ### Bigram & Trigram Probs id : nlp-m2-04-bigram-trigram-tables title : Bigram & Trigram Probs title\_confirmed : true topic\_title : Build bigram and trigram probabilities from a corpus mode : m2 kind : focused status : available duration : 10:05 reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — N-grams and Laplace | Reviewed worked numericals id : overview-ngrams-laplace title : NLP M2 — N-grams and Laplace | Reviewed worked numericals title\_confirmed : true topic\_title : NLP M2: N-Grams & Laplace mode : m2 kind : supplemental status : available duration : 7:41 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Complete counts and four Laplace factors; corrected formulas, conditional-factor wording and rounding. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids : - nlp-m2-03-ngrams-language-models - nlp-m2-04-bigram-trigram-tables - nlp-m2-05-laplace-smoothing-discount These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## M2 · Class-style count-table questions and partial-table traps Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/read/m2-count-to-probability--class-count-table-questions/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/read/m2-count-to-probability/class-count-table-questions Unit 3, pages 97–120 and appendix **Question. Train a bigram model on “I am Sam”, “Sam I am”, and “I do not like green eggs and ham”. Include sentence boundaries. Find P(I|), P(Sam|), P(am|I), P(do|I), P(Sam|am) and P(|Sam).** #### Given Three sentences; one and per sentence; preserve capital I and Sam. #### To find Six bigram estimates. #### Formula and why it applies $$ P(w\mid h)=C(h,w)/C(h) $$ Count all matching histories, including repeated words in different sentence positions. #### Solution ##### Step 1. Count the histories START occurs 3 times; I occurs 3 times; am occurs 2 times; Sam occurs 2 times. ##### Step 2. Count the requested pairs | Pair | Pair count | History count | | --- | --- | --- | | START I | 2 | 3 | | START Sam | 1 | 3 | | I am | 2 | 3 | | I do | 1 | 3 | | am Sam | 1 | 2 | | Sam END | 1 | 2 | ##### Step 3. Divide pair by history | Requested probability | Result | | --- | --- | | P(I|START) | 2/3 | | P(Sam|START) | 1/3 | | P(am|I) | 2/3 | | P(do|I) | 1/3 | | P(Sam|am) | 1/2 | | P(END|Sam) | 1/2 | **Final answer.** The six estimates are 2/3,1/3,2/3,1/3,1/2,1/2 in the requested order. **Question. A bigram table gives P(i|)=0.25, P(want|i)=0.33, P(english|want)=0.0011, P(food|english)=0.5 and P(|food)=0.68. Score “i want english food” using these supplied rounded values.** #### Given All five transition probabilities are explicitly given. Include the end marker. #### To find The sentence probability from the supplied table. #### Formula and why it applies $$ P=P(i\mid START)P(want\mid i)P(english\mid want)P(food\mid english)P(END\mid food) $$ A table-based question uses the provided values; slight differences can arise if exact corpus fractions are used instead. #### Solution ##### Step 1. First two events $$ 0.25(0.33)=0.0825 $$ ##### Step 2. Add english $$ 0.0825(0.0011)=0.00009075 $$ ##### Step 3. Add food $$ 0.00009075(0.5)=0.000045375 $$ ##### Step 4. End the sentence $$ 0.000045375(0.68)=0.000030855 $$ **Final answer.** P≈0.000030855 using the supplied rounded probabilities. **Question. A corpus has vocabulary size 1446, C(i)=2533 and C(i,want)=827. Compute the unsmoothed and add-one-smoothed P(want|i). The displayed bigram table contains only eight vocabulary columns.** #### Given Full history count C(i)=2533; pair count 827; complete vocabulary size V=1446. The displayed table is only a subset. #### To find Both conditional probabilities. #### Formula and why it applies $$ P(want\mid i)=827/2533,\qquad P_L(want\mid i)=\frac{827+1}{2533+1446} $$ Use the supplied full history total. Summing only the eight displayed columns would wrongly ignore all other possible continuations. #### Solution ##### Step 1. Calculate the unsmoothed ratio $$ P=827/2533\approx0.326490 $$ ##### Step 2. Add one to the pair count $$ 827+1=828 $$ ##### Step 3. Add V to the history count $$ 2533+1446=3979 $$ ##### Step 4. Divide $$ P_L=828/3979\approx0.208092 $$ **Final answer.** Unsmoothed≈0.326490; smoothed≈0.208092. **Yaad rakho:** Partial table ka visible row-sum, full history total nahi hota. ### Bigram & Trigram Probs id : nlp-m2-04-bigram-trigram-tables title : Bigram & Trigram Probs title\_confirmed : true topic\_title : Build bigram and trigram probabilities from a corpus mode : m2 kind : focused status : available duration : 10:05 reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — N-grams and Laplace | Reviewed worked numericals id : overview-ngrams-laplace title : NLP M2 — N-grams and Laplace | Reviewed worked numericals title\_confirmed : true topic\_title : NLP M2: N-Grams & Laplace mode : m2 kind : supplemental status : available duration : 7:41 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Complete counts and four Laplace factors; corrected formulas, conditional-factor wording and rounding. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids : - nlp-m2-03-ngrams-language-models - nlp-m2-04-bigram-trigram-tables - nlp-m2-05-laplace-smoothing-discount These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## M2 · Perplexity: meaning, formula and four complete examples Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/read/m2-evaluation-smoothing--evaluation-perplexity/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/read/m2-evaluation-smoothing/evaluation-perplexity Unit 3, pages 121–156 **Intrinsic evaluation** evaluates a model directly on a held-out test corpus, for example using perplexity. **Extrinsic evaluation** places it inside a downstream application, such as a speech recognizer or translation system, and measures the application's performance. Extrinsic testing is closer to actual use but often slower and more expensive. A lower perplexity is not a guarantee of improvement on every downstream task. **Perplexity** is the inverse probability of a test sequence, normalized by the number of prediction events. Intuitively, it is the model's effective average number of equally plausible choices. A model that always gives the correct next word probability 1/4 has perplexity 4. Lower is better when the models are tested on the same data, vocabulary, tokenization and boundary convention. $$ \operatorname{PP}(W)=P(W)^{-1/N}=\sqrt[N]{\frac1{P(W)}} $$ | Symbol | Meaning | | --- | --- | | W | The complete test sequence. | | P(W) | Product of the probabilities assigned to the actual next tokens. | | N | Number of scored prediction events under the stated convention. | | Exponent −1/N | Take the reciprocal, then the Nth root. | For calculator entry, use parentheses: P raised to (−1 ÷ N), or (1 ÷ P) raised to (1 ÷ N). Retain several digits in intermediate products. The ordinary start marker is usually given context, not a predicted event; the end marker counts if its probability is included. **Question. A model assigns probability 0.0625 to a test sequence containing four scored tokens. Compute its perplexity.** #### Given P(W)=0.0625=1/16; N=4. #### To find PP(W). #### Formula and why it applies $$ \operatorname{PP}(W)=\left(\frac1{P(W)}\right)^{1/N} $$ The root removes the effect of sequence length; the reciprocal makes lower surprise correspond to lower perplexity. #### Solution ##### Step 1. Take the reciprocal $$ 1/P(W)=1/0.0625=16 $$ ##### Step 2. Use the fourth root $$ \operatorname{PP}=16^{1/4} $$ ##### Step 3. Evaluate $$ 2^4=16\quad\Rightarrow\quad\operatorname{PP}=2 $$ **Final answer.** Perplexity = 2. **Question. Two language models score the same three-token test sequence. Model A gives next-token probabilities 0.5, 0.5, 0.5. Model B gives 0.5, 0.25, 0.25. Compute their perplexities and select the better intrinsic result.** #### Given Same sequence and N=3 for both models. A factors: 0.5,0.5,0.5. B factors: 0.5,0.25,0.25. #### To find Both sequence probabilities and perplexities. #### Formula and why it applies $$ P(W)=\prod_i p_i,\qquad\operatorname{PP}=P(W)^{-1/3} $$ A fair comparison uses identical test tokens and scoring conventions. #### Solution ##### Step 1. Calculate A's probability $$ P_A=0.5\times0.5\times0.5=0.125=1/8 $$ ##### Step 2. Calculate A's perplexity $$ \operatorname{PP}_A=8^{1/3}=2 $$ ##### Step 3. Calculate B's probability $$ P_B=0.5\times0.25\times0.25=0.03125=1/32 $$ ##### Step 4. Calculate B's perplexity $$ \operatorname{PP}_B=32^{1/3}\approx3.174802 $$ ##### Step 5. Compare 2 is lower than 3.174802; A places more probability on the observed sequence. **Final answer.** Model A is better by perplexity on this test: 2 versus about 3.174802. **Question. The bigram training corpus is “This is a dog”, “This is a cat”, “I love my cat”, “This is my name”. Score “This is my cat”, including an end marker, and compute perplexity using five predicted events.** #### Given Each training sentence has one start and end marker. Test predictions: This, is, my, cat, . Thus N=5. #### To find Bigram sentence probability and perplexity. #### Formula and why it applies $$ P(w\mid h)=C(h,w)/C(h),\qquad\operatorname{PP}=P(W)^{-1/5} $$ Start is given context. Four words plus the end marker produce five probability factors. #### Solution ##### Step 1. Count the required transitions | Transition | Pair count / history count | | --- | --- | | →This | 3/4 | | This→is | 3/3 | | is→my | 1/3 | | my→cat | 1/2 | | cat→ | 2/2 | ##### Step 2. Form the product $$ P=\frac34\times1\times\frac13\times\frac12\times1=\frac18 $$ ##### Step 3. Take the reciprocal $$ 1/P=8 $$ ##### Step 4. Take the fifth root $$ \operatorname{PP}=8^{1/5}\approx1.515717 $$ **Final answer.** P(W)=1/8 and perplexity≈1.515717 using five predicted events. **If the question explicitly states N = 6:** use that supplied convention and report PP = 81/6 ≈ 1.414214. The class slide counts both boundary markers in N. Write the chosen N beside the formula; do not silently switch between 5 and 6. Also, This occurs three times and is follows it three times, so P(is|This)=3/3, not 3/4. **Question. A uniform model assigns probability 1/10 to each actual next word in a five-token test. Find its perplexity.** #### Given Five scored tokens, each with assigned probability 0.1. N=5. #### To find The sentence probability and perplexity. #### Formula and why it applies $$ P(W)=(1/10)^5,\qquad \operatorname{PP}=P(W)^{-1/5} $$ Uniform equal probabilities make the effective number of choices especially clear. #### Solution ##### Step 1. Multiply the five factors $$ P(W)=0.1^5=0.00001=10^{-5} $$ ##### Step 2. Invert $$ 1/P(W)=10^5 $$ ##### Step 3. Take the fifth root $$ \operatorname{PP}=(10^5)^{1/5}=10 $$ **Final answer.** Perplexity = 10. **Yaad rakho:** Probability high → surprise low → perplexity low. ### M2 06 — Perplexity, logarithms and calculator working id : nlp-m2-06-perplexity-logs-calculator title : M2 06 — Perplexity, logarithms and calculator working title\_confirmed : false topic\_title : Perplexity, logarithms and calculator working mode : m2 kind : focused status : queued duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — Perplexity | Reviewed formulas and solved numericals id : overview-perplexity-logs title : NLP M2 — Perplexity | Reviewed formulas and solved numericals title\_confirmed : true topic\_title : NLP M2 — Perplexity | Reviewed formulas and solved numericals mode : m2 kind : supplemental status : available duration : 9:35 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Readable formulas and all worked examples; corrected comparison conditions and context claims. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids : - nlp-m2-06-perplexity-logs-calculator These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## M2 · Log probabilities: replace long products with sums Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/read/m2-evaluation-smoothing--log-probabilities/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/read/m2-evaluation-smoothing/log-probabilities Unit 3, pages 121–156 Probabilities are at most 1, so multiplying many of them produces very small numbers. Computers can round extremely small positive products to zero: **numerical underflow**. Log probabilities convert multiplication into addition and are more stable for long sequences. Natural logs use base e; log₂ uses base 2. Choose one base consistently. $$ \ln P(W)=\sum_{i=1}^{N}\ln p_i $$ $$ P(W)=e^{\ln P(W)},\qquad \operatorname{PP}(W)=\exp\left(-\frac1N\sum_i\ln p_i\right) $$ The log of a probability below 1 is negative. A value closer to zero means a larger probability: ln(0.5) ≈ −0.693 is larger than ln(0.1) ≈ −2.303. A zero probability has log −∞; taking logs does not solve the unseen-event problem. Smoothing handles that separate problem. **Question. The next-word probabilities are 0.5, 0.25 and 0.5. Calculate the natural log probability, ordinary probability and perplexity.** #### Given p₁=0.5; p₂=0.25; p₃=0.5; N=3. #### To find ln P(W), P(W), and PP. #### Formula and why it applies $$ \ln P=\sum_i\ln p_i,\quad P=e^{\ln P},\quad\operatorname{PP}=e^{-\ln P/N} $$ Logarithms preserve products as sums and the exponent converts the result back. #### Solution ##### Step 1. Compute individual logs $$ \ln0.5\approx-0.693147,\quad\ln0.25\approx-1.386294 $$ ##### Step 2. Add all three terms $$ \ln P=-0.693147-1.386294-0.693147\approx-2.772589 $$ ##### Step 3. Recover the probability $$ P=e^{-2.772589}\approx0.0625 $$ ##### Step 4. Find average negative log probability $$ -\ln P/3\approx2.772589/3=0.924196 $$ ##### Step 5. Exponentiate $$ \operatorname{PP}=e^{0.924196}\approx2.519842 $$ **Final answer.** ln P≈−2.772589, P=0.0625, PP≈2.519842. **Question. A model has a total natural log probability −12 over six scored tokens. Calculate its perplexity.** #### Given ln P(W)=−12; N=6. #### To find PP without forming the tiny ordinary probability. #### Formula and why it applies $$ \operatorname{PP}=\exp(-\ln P(W)/N) $$ The logarithmic formula directly uses the supplied log score. #### Solution ##### Step 1. Negate the log score $$ -\ln P=12 $$ ##### Step 2. Divide by token count $$ 12/6=2 $$ ##### Step 3. Apply exponential $$ \operatorname{PP}=e^2\approx7.389056 $$ **Final answer.** Perplexity≈7.389056. **Question. A language model's average negative log₂ probability is 3 bits per token. Find its perplexity.** #### Given Cross-entropy H=3 bits/token. Log base is 2. #### To find PP. #### Formula and why it applies $$ \operatorname{PP}=2^H $$ The inverse operation of log₂ is raising 2 to that power; use e only for natural logs. #### Solution ##### Step 1. Identify the log base Bits mean base 2. ##### Step 2. Raise the base to H $$ \operatorname{PP}=2^3=8 $$ **Final answer.** Perplexity = 8. **Yaad rakho:** ln use kiya toh e se wapas; log₂ use kiya toh 2 se wapas. ### M2 06 — Perplexity, logarithms and calculator working id : nlp-m2-06-perplexity-logs-calculator title : M2 06 — Perplexity, logarithms and calculator working title\_confirmed : false topic\_title : Perplexity, logarithms and calculator working mode : m2 kind : focused status : queued duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — Perplexity | Reviewed formulas and solved numericals id : overview-perplexity-logs title : NLP M2 — Perplexity | Reviewed formulas and solved numericals title\_confirmed : true topic\_title : NLP M2 — Perplexity | Reviewed formulas and solved numericals mode : m2 kind : supplemental status : available duration : 9:35 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Readable formulas and all worked examples; corrected comparison conditions and context claims. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids : - nlp-m2-06-perplexity-logs-calculator These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## M2 · Laplace smoothing: why both numerator and denominator change Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/read/m2-evaluation-smoothing--laplace-basics/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/read/m2-evaluation-smoothing/laplace-basics Unit 3, pages 121–156 **Smoothing** reallocates some probability mass to events that received zero or unreliable estimates from finite training data. **Laplace, or add-one, smoothing** adds one pseudo-count to every possible next outcome. If there are V possible outcomes, the total added count is V, so adding 1 only to the numerator would produce probabilities that do not sum to 1. $$ P_{\mathrm{Laplace}}(w)=\frac{C(w)+1}{N+V} $$ $$ P_{\mathrm{Laplace}}(w\mid h)=\frac{C(h,w)+1}{C(h)+V} $$ For unigram smoothing, N is the total training token count. For bigram smoothing, C(h) is the count of the particular history. In both cases V is the number of outcomes being predicted, not the number of training sentences and not the number of distinct bigrams. When end is a possible prediction, include it in V; the start marker is usually context only. Observed counts: 3, 1, 0 → Add one to every outcome: 4, 2, 1 → New total: 7 → Smoothed probabilities: 4/7, 2/7, 1/7 **Question. A fixed unigram vocabulary is {a,b,c}. The training counts are 3,1,0. Compute unsmoothed and Laplace-smoothed probabilities.** #### Given C(a)=3, C(b)=1, C(c)=0; N=4; V=3. #### To find The complete distributions. #### Formula and why it applies $$ P(w)=C(w)/N,\qquad P_L(w)=(C(w)+1)/(N+V) $$ The zero-count c is still in the fixed vocabulary, so it must receive one pseudo-count and contribute to V. #### Solution ##### Step 1. Compute unsmoothed values | Word | Unsmoothed | | --- | --- | | a | 3/4 = 0.75 | | b | 1/4 = 0.25 | | c | 0/4 = 0 | ##### Step 2. Calculate the new denominator $$ N+V=4+3=7 $$ ##### Step 3. Add one to each numerator $$ C(a)+1=4,\quad C(b)+1=2,\quad C(c)+1=1 $$ ##### Step 4. Divide | Word | Smoothed | | --- | --- | | a | 4/7 ≈ 0.571429 | | b | 2/7 ≈ 0.285714 | | c | 1/7 ≈ 0.142857 | ##### Step 5. Confirm the total $$ 4/7+2/7+1/7=1 $$ **Final answer.** Smoothed probabilities: a=4/7, b=2/7, c=1/7. Common a loses mass; unseen c gains mass. **Question. After history I, a corpus records like three times, study once and sleep zero times. The allowed next vocabulary is {like,study,sleep}. Compute all Laplace-smoothed bigram probabilities.** #### Given C(I,like)=3; C(I,study)=1; C(I,sleep)=0. History count C(I)=4. V=3. #### To find P\_L(like|I), P\_L(study|I), P\_L(sleep|I). #### Formula and why it applies $$ P_L(w\mid I)=\frac{C(I,w)+1}{C(I)+V} $$ This is smoothing within the I row; use its total 4, not the global corpus token count. #### Solution ##### Step 1. Form the common denominator $$ C(I)+V=4+3=7 $$ ##### Step 2. Calculate like $$ P_L(like\mid I)=(3+1)/7=4/7 $$ ##### Step 3. Calculate study $$ P_L(study\mid I)=(1+1)/7=2/7 $$ ##### Step 4. Calculate sleep $$ P_L(sleep\mid I)=(0+1)/7=1/7 $$ **Final answer.** The row is (4/7, 2/7, 1/7). An unseen bigram is no longer forced to zero. **Question. A fixed vocabulary has five possible next tokens. A history h is unseen in training. What is its add-one probability for each next token?** #### Given C(h)=0; C(h,w)=0 for each outcome; V=5. #### To find P\_L(w|h) for each token. #### Formula and why it applies $$ P_L(w\mid h)=\frac{0+1}{0+V} $$ All outcomes receive equal pseudo-counts because the training data gives no preference for this history. #### Solution ##### Step 1. Add the pseudo-count $$ 0+1=1 $$ ##### Step 2. Form the denominator $$ 0+5=5 $$ ##### Step 3. Divide and sum $$ P_L(w\mid h)=1/5=0.2,\qquad5(0.2)=1 $$ **Final answer.** Every outcome gets 0.2 under this fixed-vocabulary model. **Vocabulary warning.** Add-one does not automatically define probabilities for every word in the world. Unknown words require a declared vocabulary/UNK convention. If a question supplies V, use that stated V rather than inventing additional tokens. **Yaad rakho:** Add-one: upar +1, neeche +V; bigram mein neeche history ka count. ### M2 05: Laplace Smoothing id : nlp-m2-05-laplace-smoothing-discount title : M2 05: Laplace Smoothing title\_confirmed : true topic\_title : Laplace smoothing, unseen sequences and adjusted counts mode : m2 kind : focused status : available duration : 11:11 reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — N-grams and Laplace | Reviewed worked numericals id : overview-ngrams-laplace title : NLP M2 — N-grams and Laplace | Reviewed worked numericals title\_confirmed : true topic\_title : NLP M2: N-Grams & Laplace mode : m2 kind : supplemental status : available duration : 7:41 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Complete counts and four Laplace factors; corrected formulas, conditional-factor wording and rounding. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids : - nlp-m2-03-ngrams-language-models - nlp-m2-04-bigram-trigram-tables - nlp-m2-05-laplace-smoothing-discount These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## M2 · A complete unseen-bigram question from corpus to final answer Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/read/m2-evaluation-smoothing--smoothed-sentence/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/read/m2-evaluation-smoothing/smoothed-sentence Unit 3, pages 121–156 **Question. Consider the corpus “I love to code in Python. I love to learn new things. I code every day.” Obtain the bigram probabilities needed for “I love Python”. Explain the zero-probability issue, then use Laplace smoothing to score it. Treat each sentence separately, preserve case, remove punctuation, and include an end token among predicted outcomes.** #### Given Three sentences padded with one start/end marker. Distinct words: I, love, to, code, in, Python, learn, new, things, every, day. With , V=12. Start is context only. #### To find Unsmoothed and smoothed P(I love Python | ). #### Formula and why it applies $$ P(w\mid h)=\frac{C(h,w)}{C(h)},\qquad P_L(w\mid h)=\frac{C(h,w)+1}{C(h)+12} $$ The test contains an unseen transition love→Python. Smoothing assigns probability to that transition and renormalizes every required history row. #### Solution ##### Step 1. Count the contexts | History | Count | | --- | --- | | | 3 | | I | 3 | | love | 2 | | Python | 1 | ##### Step 2. Count each required pair | Pair | Count | | --- | --- | | I | 3 | | I love | 2 | | love Python | 0 | | Python | 1 | ##### Step 3. Compute the unsmoothed product $$ P=\frac33\times\frac23\times\frac02\times\frac11=0 $$ Both words love and Python were seen, but their adjacent pair was not. This is an unseen bigram, not an unknown-word case. ##### Step 4. Smooth the first event $$ P_L(I\mid\langle s\rangle)=\frac{3+1}{3+12}=\frac4{15} $$ ##### Step 5. Smooth the second event $$ P_L(love\mid I)=\frac{2+1}{3+12}=\frac3{15} $$ ##### Step 6. Smooth the unseen event $$ P_L(Python\mid love)=\frac{0+1}{2+12}=\frac1{14} $$ ##### Step 7. Smooth the end event $$ P_L(\langle/s\rangle\mid Python)=\frac{1+1}{1+12}=\frac2{13} $$ ##### Step 8. Multiply the smoothed factors $$ P_L=\frac4{15}\times\frac3{15}\times\frac1{14}\times\frac2{13} $$ $$ P_L=\frac{24}{40950}=\frac4{6825}\approx0.000586081 $$ **Final answer.** Unsmoothed P=0. With the stated V=12 and boundaries, Laplace-smoothed P≈0.000586081. If an exam states a different convention—for example excluding boundary tokens from the entire calculation—recompute both V and the set of probability factors consistently. Do not combine a boundary-aware numerator with a boundary-free denominator. ### M2 05: Laplace Smoothing id : nlp-m2-05-laplace-smoothing-discount title : M2 05: Laplace Smoothing title\_confirmed : true topic\_title : Laplace smoothing, unseen sequences and adjusted counts mode : m2 kind : focused status : available duration : 11:11 reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — N-grams and Laplace | Reviewed worked numericals id : overview-ngrams-laplace title : NLP M2 — N-grams and Laplace | Reviewed worked numericals title\_confirmed : true topic\_title : NLP M2: N-Grams & Laplace mode : m2 kind : supplemental status : available duration : 7:41 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Complete counts and four Laplace factors; corrected formulas, conditional-factor wording and rounding. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids : - nlp-m2-03-ngrams-language-models - nlp-m2-04-bigram-trigram-tables - nlp-m2-05-laplace-smoothing-discount These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## M2 · Adjusted counts and relative discounting Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/read/m2-evaluation-smoothing--adjusted-count-discount/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/read/m2-evaluation-smoothing/adjusted-count-discount Unit 3, pages 121–156 The slides also express smoothing through an **adjusted count** c\*. Instead of writing the smoothed probability with denominator N+V, they rescale its numerator so the denominator can remain N. Algebraically these are equivalent: $$ c^*=(c+1)\frac{N}{N+V},\qquad P_L(w)=\frac{c^*}{N} $$ $$ d=\frac{c^*}{c}\quad(c>0) $$ The relative discount d compares adjusted and original counts. It is undefined when c=0 because division by zero is undefined. Add-one smoothing moves the distribution toward uniformity: frequent events are reduced, but some rare seen events may increase. It is not correct to claim that every nonzero count must decrease under Laplace smoothing. **Question. For N=20 and V=5, a word has original count c=10. Find its adjusted count, smoothed probability and relative discount.** #### Given N=20; V=5; c=10. #### To find c\*, P\_L and d. #### Formula and why it applies $$ c^*=(c+1)N/(N+V),\quad P_L=c^*/N,\quad d=c^*/c $$ These expressions rewrite add-one smoothing without changing its probability. #### Solution ##### Step 1. Form the new total $$ N+V=20+5=25 $$ ##### Step 2. Find the adjusted count $$ c^*=(10+1)\frac{20}{25}=11(0.8)=8.8 $$ ##### Step 3. Find the probability $$ P_L=8.8/20=0.44 $$ ##### Step 4. Find the relative discount $$ d=8.8/10=0.88 $$ **Final answer.** c\*=8.8, P\_L=0.44, d=0.88. The original probability 0.5 is reduced. **Question. Under N=20, V=5, a rare word occurs once. Compute its adjusted count and compare the original and smoothed probabilities.** #### Given N=20; V=5; c=1. #### To find c\*, original P and smoothed P\_L. #### Formula and why it applies $$ c^*=(c+1)N/(N+V) $$ The same smoothing rule may increase a rare event's probability. #### Solution ##### Step 1. Original probability $$ P=1/20=0.05 $$ ##### Step 2. Adjusted count $$ c^*=(1+1)(20/25)=1.6 $$ ##### Step 3. Smoothed probability $$ P_L=1.6/20=0.08 $$ **Final answer.** c\*=1.6 and P\_L=0.08, which exceeds the original 0.05. **Question. For N=20 and V=5, an in-vocabulary word has count zero. Find its adjusted count and smoothed probability. Can a relative discount be calculated?** #### Given c=0; N=20; V=5. #### To find c\*, P\_L and whether d exists. #### Formula and why it applies $$ c^*=(c+1)N/(N+V),\qquad P_L=c^*/N $$ The pseudo-count creates positive mass for the unseen vocabulary item. #### Solution ##### Step 1. Adjusted count $$ c^*=(0+1)(20/25)=0.8 $$ ##### Step 2. Smoothed probability $$ P_L=0.8/20=0.04 $$ ##### Step 3. Relative discount d would be 0.8/0. That is undefined, so do not report a finite discount. **Final answer.** c\*=0.8 and P\_L=0.04; relative discount is undefined for c=0. ### M2 05: Laplace Smoothing id : nlp-m2-05-laplace-smoothing-discount title : M2 05: Laplace Smoothing title\_confirmed : true topic\_title : Laplace smoothing, unseen sequences and adjusted counts mode : m2 kind : focused status : available duration : 11:11 reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — N-grams and Laplace | Reviewed worked numericals id : overview-ngrams-laplace title : NLP M2 — N-grams and Laplace | Reviewed worked numericals title\_confirmed : true topic\_title : NLP M2: N-Grams & Laplace mode : m2 kind : supplemental status : available duration : 7:41 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Complete counts and four Laplace factors; corrected formulas, conditional-factor wording and rounding. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids : - nlp-m2-03-ngrams-language-models - nlp-m2-04-bigram-trigram-tables - nlp-m2-05-laplace-smoothing-discount These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## M2 · Character-model perplexity: keep exact probabilities Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/read/m2-evaluation-smoothing--class-perplexity-characters/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/read/m2-evaluation-smoothing/class-perplexity-characters Unit 3, pages 121–156 **Question. A unigram character model gives P(A)=0.25, P(B)=0.50, P(C)=0.25. Find the perplexity of AAA and ABC. Then find both perplexities when the three characters are equally likely.** #### Given Each test string contains 3 scored characters, so N=3. Initial distribution 0.25,0.50,0.25; comparison distribution 1/3,1/3,1/3. #### To find Both sequence probabilities and perplexities under each distribution. #### Formula and why it applies $$ P(c_1c_2c_3)=P(c_1)P(c_2)P(c_3),\qquad PP=P(W)^{-1/3} $$ Characters, not words, are the model's prediction units in this question. Use exact 1/3 for uniform probabilities, rather than rounded 0.33. #### Solution ##### Step 1. Score AAA $$ P(AAA)=0.25^3=\frac1{64}=0.015625 $$ ##### Step 2. Find AAA perplexity $$ PP(AAA)=64^{1/3}=4 $$ ##### Step 3. Score ABC $$ P(ABC)=0.25(0.5)(0.25)=\frac1{32}=0.03125 $$ ##### Step 4. Find ABC perplexity $$ PP(ABC)=32^{1/3}\approx3.174802 $$ ##### Step 5. Change to an exactly uniform model $$ P(AAA)=P(ABC)=(1/3)^3=1/27 $$ ##### Step 6. Find both uniform-model perplexities $$ PP=27^{1/3}=3 $$ **Final answer.** Original model: AAA→4, ABC→3.174802. Uniform model: both→3. These values follow the stated formulas without intermediate rounding. ### M2 06 — Perplexity, logarithms and calculator working id : nlp-m2-06-perplexity-logs-calculator title : M2 06 — Perplexity, logarithms and calculator working title\_confirmed : false topic\_title : Perplexity, logarithms and calculator working mode : m2 kind : focused status : queued duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — Perplexity | Reviewed formulas and solved numericals id : overview-perplexity-logs title : NLP M2 — Perplexity | Reviewed formulas and solved numericals title\_confirmed : true topic\_title : NLP M2 — Perplexity | Reviewed formulas and solved numericals mode : m2 kind : supplemental status : available duration : 9:35 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Readable formulas and all worked examples; corrected comparison conditions and context claims. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids : - nlp-m2-06-perplexity-logs-calculator These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## M2 · Parts of speech and contextual ambiguity Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/read/m2-pos-tagging--word-classes-pos/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/read/m2-pos-tagging/word-classes-pos Unit 4, pages 3–46 **Part-of-speech (POS) tagging** assigns a grammatical category to every word token in context, such as noun, verb, adjective or determiner. It is a sequence-labeling task: the input is a sequence of words, and the output is a tag for each word. A tag describes the word's role in this sentence, not its meaning in every possible sentence. | Tag | Role | Example in context | | --- | --- | --- | | NN / NNS | Singular / plural common noun | a book / two books | | NNP | Proper noun | Mary, Mumbai | | VB / VBD / VBZ | Base / past / third-person singular verb | read / read / reads | | JJ | Adjective | the red book | | RB | Adverb | runs quickly | | DT | Determiner | the, a, this | | IN | Preposition or subordinating conjunction | on the table | | PRP | Personal pronoun | she, they | | MD | Modal auxiliary | can, will, should | **Open classes** readily accept new content words: nouns, verbs, adjectives and adverbs. Examples include chatbot and livestream. **Closed classes** contain comparatively stable grammatical/function words: determiners, pronouns, prepositions and conjunctions. These are tendencies, not a claim that language never changes. Input: Book that flight → Context: an imperative request → Output: Book/VB that/DT flight/NN Compare “hand me that **book**”: book is a noun here. Similarly, “I **can** swim” has can/MD, while “the **can** is empty” has can/NN. The spelling stays the same; neighboring words and sentence structure change the correct tag. This is **POS ambiguity**. | Sentence | Tagged interpretation | Reason | | --- | --- | --- | | They watch birds. | They/PRP watch/VBP birds/NNS | watch is the action. | | The watch stopped. | The/DT watch/NN stopped/VBD | watch is the object that stopped. | | She has a light bag. | She/PRP has/VBZ a/DT light/JJ bag/NN | light describes bag. | | Light the lamp. | Light/VB the/DT lamp/NN | Light is an imperative verb. | POS tags support parsing, named entity recognition, question answering, sentiment analysis and word-sense disambiguation. They do not by themselves identify every entity or fully resolve meaning. **Yaad rakho:** Word ka naam same ho sakta hai; sentence mein uska kaam tag decide karta hai. ### NLP M2 — POS tagging and Brill | Reviewed explanation and worked rule id : nlp-m2-07-pos-tags-tagger-methods title : NLP M2 — POS tagging and Brill | Reviewed explanation and worked rule title\_confirmed : true topic\_title : POS tags, ambiguity and tagging methods mode : m2 kind : focused status : available duration : 9:06 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Tagger methods, Brill net gain and the full conditional rule; clarified tagging and imperative explanations. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## M2 · Lexical, rule-based, stochastic, Brill and neural taggers Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/read/m2-pos-tagging--tagger-methods/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/read/m2-pos-tagging/tagger-methods Unit 4, pages 3–46 **A lexical baseline** gives each known word its most frequent training tag. It is simple and often useful, but ignores context. A **rule-based tagger** uses a lexicon of possible tags plus manually designed contextual or morphological rules to resolve ambiguity. For example, after a determiner, a noun reading of “can” may be preferred in “the can”. Rules are usually heuristics with exceptions. “A red car” shows that the word immediately after an article is not always a noun. The suffix -ed does not make “red” a past-tense verb. A robust rule system uses multiple linguistic constraints instead of treating such shortcuts as universal laws. | Method | How it chooses tags | Strength | Limitation | | --- | --- | --- | --- | | Most frequent / lexical | Most common training tag for the word | Simple baseline | Misses contextual changes and unknown words | | Rule-based | Lexicon plus handwritten rules | Readable, controllable linguistic decisions | Expert effort; rules may conflict or need updates | | Stochastic | Scores tag sequences using estimated probabilities | Uses corpus evidence and context | Sparse/unseen events; depends on training domain | | Transformation-based / Brill | Start with baseline, then apply learned correction rules | Readable learned corrections | Needs tagged training data and suitable rule templates | | Neural | Learns contextual representations and sequence predictions | Can exploit richer context | Needs training resources and data; less directly interpretable | **Transformation-based error-driven learning (Brill tagging)** combines a simple starting assignment with learned correction rules. First tag the training corpus with a baseline. Compare against the gold tags. Generate candidate correction rules from templates, such as “change MD to NN after DT”. Apply the rule that fixes the most errors net of newly introduced errors; repeat until further improvement is too small. At test time apply the learned ordered rule list. Baseline: The/DT can/MD was/VBD crushed/VBN → Context rule: change can from MD to NN after DT → Result: The/DT can/NN was/VBD crushed/VBN **Rule-based versus Brill:** a purely rule-based system depends on linguists specifying its rules; a Brill tagger learns which transformations improve a labeled corpus. A **stochastic tagger** such as an HMM finds a high-probability tag sequence using transition and emission probabilities. CRFs and neural sequence taggers are other learning approaches; they do not have to make the same HMM independence assumptions. **Exam-ready answer.** POS tagging assigns a grammatical tag to each word in context. A rule-based tagger uses lexicons and linguistic rules, whereas a stochastic tagger chooses tags through probabilities estimated from a corpus. For example, can is MD in “I can swim” but NN in “the can”; context resolves the ambiguity. ### NLP M2 — POS tagging and Brill | Reviewed explanation and worked rule id : nlp-m2-07-pos-tags-tagger-methods title : NLP M2 — POS tagging and Brill | Reviewed explanation and worked rule title\_confirmed : true topic\_title : POS tags, ambiguity and tagging methods mode : m2 kind : focused status : available duration : 9:06 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Tagger methods, Brill net gain and the full conditional rule; clarified tagging and imperative explanations. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## M2 · Generative versus discriminative models and Bayes' rule Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/read/m2-pos-tagging--generative-discriminative/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/read/m2-pos-tagging/generative-discriminative Unit 4, pages 3–46 Let x be the observed words and y be the desired tag sequence. A **generative model** describes their joint distribution P(x,y), often as P(y)P(x|y). A **discriminative model** learns P(y|x) directly or learns a decision boundary that separates possible labels. Both kinds can be trained with labeled data; “generative” does not mean “always unsupervised”. $$ P(y\mid x)=\frac{P(x\mid y)P(y)}{P(x)} $$ | Quantity | Meaning | POS interpretation | | --- | --- | --- | | P(y) | Prior | How plausible is this tag or tag sequence before seeing the words? | | P(x|y) | Likelihood | How plausible are these words under that tag sequence? | | P(x) | Evidence | Total probability of the observed words across possible labels. | | P(y|x) | Posterior | Probability of the tags after seeing the words. | | Aspect | Generative | Discriminative | | --- | --- | --- | | Models | Joint P(x,y), or prior × likelihood | Conditional P(y|x) or decision boundary | | Classification examples | Naive Bayes | Logistic regression, SVM | | Sequence-labeling examples | HMM | Linear-chain CRF | | Intuition | Model how a label could generate the observation | Model which label fits the observation | **Question. A token has two possible tags, noun N and verb V. P(N)=0.6, P(V)=0.4, P(book|N)=0.2 and P(book|V)=0.5. Calculate P(N|book) and P(V|book).** #### Given Two tags are exhaustive. Priors: 0.6,0.4. Word likelihoods: 0.2,0.5. #### To find Normalized posterior probabilities. #### Formula and why it applies $$ P(t\mid w)=\frac{P(w\mid t)P(t)}{\sum_{u}P(w\mid u)P(u)} $$ Bayes' rule combines prior and likelihood. Normalization is needed to convert scores into posterior probabilities. #### Solution ##### Step 1. Calculate the noun score $$ 0.2\times0.6=0.12 $$ ##### Step 2. Calculate the verb score $$ 0.5\times0.4=0.20 $$ ##### Step 3. Find the evidence $$ P(book)=0.12+0.20=0.32 $$ ##### Step 4. Normalize the noun score $$ P(N\mid book)=0.12/0.32=0.375 $$ ##### Step 5. Normalize the verb score $$ P(V\mid book)=0.20/0.32=0.625 $$ **Final answer.** Verb is more probable in this simplified model: 0.625 versus 0.375. **Question. Two candidate tags have prior-likelihood products 0.06 and 0.14. Find their normalized probabilities.** #### Given Unnormalized scores s₁=0.06 and s₂=0.14; these are the only candidates. #### To find The two posterior probabilities. #### Formula and why it applies $$ P_i=s_i/(s_1+s_2) $$ A common denominator converts the candidate scores to a distribution summing to one. #### Solution ##### Step 1. Add scores $$ 0.06+0.14=0.20 $$ ##### Step 2. Normalize first $$ 0.06/0.20=0.3 $$ ##### Step 3. Normalize second $$ 0.14/0.20=0.7 $$ **Final answer.** Posterior probabilities are 0.3 and 0.7. **Yaad rakho:** Generative: tag se word ka model. Discriminative: word se tag ka decision. ### Bayes Rule & Models id : nlp-m2-02-bayes-generative-discriminative title : Bayes Rule & Models title\_confirmed : true topic\_title : Bayes rule and generative versus discriminative models mode : m2 kind : focused status : available duration : 10:51 reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — Probability and Bayes | Reviewed step-by-step numericals id : overview-probability-bayes title : NLP M2 — Probability and Bayes | Reviewed step-by-step numericals title\_confirmed : true topic\_title : NLP M2 — Probability and Bayes | Reviewed step-by-step numericals mode : m2 kind : supplemental status : available duration : 8:53 reviewed : true notebook\_url : youtube\_url corrected\_visuals : true upload\_pending\_correction : true correction\_note : Complete HMM and Naive Bayes calculations; clarified joint scores versus model posteriors. Full transcript and detected slides checked; short voice corrections included. The corrected YouTube upload is still processing. related\_topic\_ids : - nlp-m2-01-foundations-counts-probability - nlp-m2-02-bayes-generative-discriminative These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## M2 · Markov chains: read a transition table and multiply a path Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/read/m2-hmm--markov-chain/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/read/m2-hmm/markov-chain Unit 4, pages 47–73 **A Markov chain** is a probabilistic model of a sequence of states. Under the first-order Markov assumption, the next state's probability depends on the current state rather than the complete previous history. The **initial probability** tells us how likely each first state is. A **transition probability** tells us how likely a move from one state to another is. $$ P(s_t\mid s_1,\ldots,s_{t-1})=P(s_t\mid s_{t-1}) $$ $$ P(s_1,\ldots,s_T)=P(s_1)\prod_{t=2}^{T}P(s_t\mid s_{t-1}) $$ For T states in a path, use one initial factor and T−1 transition factors. A row-oriented transition matrix must have each complete row sum to 1. Always read the row/column labels; some exam diagrams use a transposed arrangement. | Current → next | Rain | Dry | | --- | --- | --- | | Rain | 0.3 | 0.7 | | Dry | 0.2 | 0.8 | Initial probabilities: P(Rain)=0.4, P(Dry)=0.6. Rows describe the current state; columns describe the next state. **Question. For the Rain/Dry Markov chain with initial P(Rain)=0.4 and P(Dry)=0.6, and transitions Rain→Rain=0.3, Rain→Dry=0.7, Dry→Rain=0.2, Dry→Dry=0.8, find the probability of Dry, Dry, Rain, Rain.** #### Given | Current → next | Rain | Dry | | --- | --- | --- | | Rain | 0.3 | 0.7 | | Dry | 0.2 | 0.8 | Initial probabilities: P(Rain)=0.4, P(Dry)=0.6. Rows describe the current state; columns describe the next state. #### To find P(Dry,Dry,Rain,Rain). #### Formula and why it applies $$ P(D,D,R,R)=P(D)P(D\mid D)P(R\mid D)P(R\mid R) $$ Each step conditions on the immediately preceding state. The sequence contains four states, so it needs one initial plus three transition factors. #### Solution ##### Step 1. Start in Dry $$ P(D)=0.6 $$ ##### Step 2. Move Dry to Dry $$ 0.6\times0.8=0.48 $$ ##### Step 3. Move Dry to Rain $$ 0.48\times0.2=0.096 $$ ##### Step 4. Move Rain to Rain $$ 0.096\times0.3=0.0288 $$ **Final answer.** Path probability = 0.0288. **Question. For initial probabilities Rain=0.4, Dry=0.6 and transition rows Rain=(0.3,0.7), Dry=(0.2,0.8) in column order Rain,Dry, find the probability of Rain, Dry, Rain.** #### Given | Current → next | Rain | Dry | | --- | --- | --- | | Rain | 0.3 | 0.7 | | Dry | 0.2 | 0.8 | Initial probabilities: P(Rain)=0.4, P(Dry)=0.6. Rows describe the current state; columns describe the next state. #### To find P(Rain,Dry,Rain). #### Formula and why it applies $$ P(R,D,R)=P(R)P(D\mid R)P(R\mid D) $$ Do not replace Dry→Rain with Rain→Dry: the two directional probabilities differ. #### Solution ##### Step 1. Initial Rain $$ P(R)=0.4 $$ ##### Step 2. First transition $$ P(D\mid R)=0.7,\quad0.4(0.7)=0.28 $$ ##### Step 3. Second transition $$ P(R\mid D)=0.2,\quad0.28(0.2)=0.056 $$ **Final answer.** Path probability = 0.056. **Question. For the same fully specified chain—initial Rain=0.4, Dry=0.6; Rain→Rain=0.3, Rain→Dry=0.7, Dry→Rain=0.2, Dry→Dry=0.8—find P(Dry,Dry). Also find the conditional probability of next Dry given current Dry.** #### Given | Current → next | Rain | Dry | | --- | --- | --- | | Rain | 0.3 | 0.7 | | Dry | 0.2 | 0.8 | Initial probabilities: P(Rain)=0.4, P(Dry)=0.6. Rows describe the current state; columns describe the next state. #### To find The two-state joint probability and the one-step conditional probability. #### Formula and why it applies $$ P(D,D)=P(D)P(D\mid D) $$ The joint path includes an uncertain initial state. The conditional transition assumes the current state is already Dry. #### Solution ##### Step 1. Read the conditional transition $$ P(D\mid D)=0.8 $$ ##### Step 2. Include the initial state for the joint path $$ P(D,D)=0.6\times0.8=0.48 $$ **Final answer.** P(Dry,Dry)=0.48; P(next Dry|current Dry)=0.8. These answer different questions. **Yaad rakho:** Path: pehle start probability, phir har arrow ki probability. ### M2 08: Markov Paths & HMMs id : nlp-m2-08-markov-hmm-components title : M2 08: Markov Paths & HMMs title\_confirmed : true topic\_title : Markov paths, HMM components and training counts mode : m2 kind : focused status : available duration : 9:45 reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — HMM and Viterbi | Reviewed worked numericals id : overview-hmm-viterbi title : NLP M2 — HMM and Viterbi | Reviewed worked numericals title\_confirmed : true topic\_title : HMM Prob & Viterbi mode : m2 kind : supplemental status : available duration : 8:30 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Complete HMM factors, all Viterbi candidate scores and backpointers; clarified zero emissions. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids : - nlp-m2-08-markov-hmm-components - nlp-m2-09-hmm-joint-paths - nlp-m2-10-viterbi-decoding These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## M2 · Hidden states, observations, transitions and emissions Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/read/m2-hmm--hmm-components/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/read/m2-hmm/hmm-components Unit 4, pages 47–73 **A Hidden Markov Model (HMM)** is a probabilistic sequence model in which the underlying state sequence is unobserved, while each state emits an observed item. For POS tagging, words are observed and their grammatical tags are hidden. The task is to infer a suitable tag sequence from the words. Hidden: Noun → Hidden: Verb → Hidden: Noun Observed: birds → Observed: eat → Observed: seeds Horizontal state-to-state links are **transitions**. The relation from each hidden state to its observed word is an **emission**. A transition P(V|N) asks how often a verb tag follows a noun tag. An emission P(eat|V) asks how often the observed word eat appears among verb-tagged tokens. | Component | Notation | Meaning | | --- | --- | --- | | State set | S={N,V,…} | Possible hidden tags. | | Observation vocabulary | O={birds,eat,seeds,…} | Visible word tokens. | | Initial distribution | πⱼ=P(t₁=j) | Probability of starting in tag j. | | Transition matrix | aᵢⱼ=P(tₖ=j|tₖ₋₁=i) | Probability of next tag j after i. | | Emission matrix | bⱼ(w)=P(w|tag j) | Probability that state j emits word w. | Two basic assumptions are used: the current hidden state depends only on the previous hidden state, and the current observation depends only on the current hidden state. A standard first-order HMM therefore does not explicitly condition an emitted word on the whole sentence. $$ P(t_{1:T},w_{1:T})=\pi_{t_1}b_{t_1}(w_1)\prod_{k=2}^{T}a_{t_{k-1},t_k}b_{t_k}(w_k) $$ Read the formula as **start × first emission × transition × emission × transition × emission…**. There are T emission factors and T−1 transitions, plus one initial factor. If the question supplies an explicit end-state probability, multiply that too. Do not invent an end factor when the model has none. **Three different questions.** (1) Scoring a specified tag path uses the joint product above. (2) Finding the probability of the observed words sums scores over all possible tag paths. (3) Finding the best tag path maximizes the path score, typically using Viterbi. A joint path score is not a normalized posterior P(tags|words). **Yaad rakho:** Transition = tag se agla tag. Emission = tag se visible word. ### M2 08: Markov Paths & HMMs id : nlp-m2-08-markov-hmm-components title : M2 08: Markov Paths & HMMs title\_confirmed : true topic\_title : Markov paths, HMM components and training counts mode : m2 kind : focused status : available duration : 9:45 reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — HMM and Viterbi | Reviewed worked numericals id : overview-hmm-viterbi title : NLP M2 — HMM and Viterbi | Reviewed worked numericals title\_confirmed : true topic\_title : HMM Prob & Viterbi mode : m2 kind : supplemental status : available duration : 8:30 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Complete HMM factors, all Viterbi candidate scores and backpointers; clarified zero emissions. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids : - nlp-m2-08-markov-hmm-components - nlp-m2-09-hmm-joint-paths - nlp-m2-10-viterbi-decoding These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## M2 · Estimate transition and emission probabilities from tagged data Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/read/m2-hmm--transition-emission-estimation/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/read/m2-hmm/transition-emission-estimation Unit 4, pages 47–73 $$ P(t'\mid t)=\frac{C(t,t')}{C(t)\text{ with a following tag/end event}} $$ $$ P(w\mid t)=\frac{C(w,t)}{C(t)} $$ In a transition denominator, count the previous tag's outgoing events. In an emission denominator, count all tokens bearing that tag. With explicit end transitions the totals align conveniently. Keep the two numerator types separate: tag→tag counts are not word→tag counts. **Question. A tagged corpus contains 116454 determiner tags DT, each counted with an outgoing event. DT is followed by NN 56509 times. Estimate P(NN|DT).** #### Given C(DT)=116454; C(DT,NN)=56509. #### To find The DT→NN transition probability. #### Formula and why it applies $$ P(NN\mid DT)=C(DT,NN)/C(DT) $$ The denominator is the number of previous DT contexts because DT is the condition. #### Solution ##### Step 1. Identify the transition count DT followed by NN occurs 56509 times. ##### Step 2. Identify the conditional total There are 116454 DT contexts. ##### Step 3. Divide $$ P(NN\mid DT)=\frac{56509}{116454}\approx0.485247 $$ **Final answer.** P(NN|DT)≈0.485247. **Question. The tag VBZ occurs 21627 times, and 10073 of those tokens are the word “is”. Estimate P(is|VBZ).** #### Given C(VBZ)=21627; C(is,VBZ)=10073. #### To find The emission probability of is from VBZ. #### Formula and why it applies $$ P(is\mid VBZ)=C(is,VBZ)/C(VBZ) $$ The condition is the tag VBZ; we ask which word it emits. #### Solution ##### Step 1. Choose the word-tag count The required word is is with tag VBZ: count 10073. ##### Step 2. Choose the tag total All VBZ-tagged word tokens: count 21627. ##### Step 3. Divide $$ P(is\mid VBZ)=\frac{10073}{21627}\approx0.465760 $$ **Final answer.** P(is|VBZ)≈0.465760. This is different from P(VBZ|is). **Question. Use the tagged sentences “dogs/N chase/V cats/N” and “cats/N sleep/V”. Estimate the initial tag probabilities, all nonzero transitions including END, and emissions.** #### Given Corpus: START dogs/N chase/V cats/N END; START cats/N sleep/V END. #### To find Initial distribution, transition rows and emission rows. #### Formula and why it applies $$ \pi_t=C(START,t)/2,\quad a_{tu}=C(t,u)/C(t),\quad b_t(w)=C(w,t)/C(t) $$ The corpus explicitly tells us the tags, so we can count training frequencies directly. #### Solution ##### Step 1. Count tag tokens N occurs 3 times: dogs, cats, cats. V occurs 2 times: chase, sleep. ##### Step 2. Count sentence starts Both sentences start with N. $$ \pi_N=2/2=1,\quad \pi_V=0/2=0 $$ ##### Step 3. Count transitions | Pair | Count | | --- | --- | | N→V | 2 | | N→END | 1 | | V→N | 1 | | V→END | 1 | ##### Step 4. Normalize transition rows | Current | To N | To V | To END | | --- | --- | --- | --- | | N | 0 | 2/3 | 1/3 | | V | 1/2 | 0 | 1/2 | ##### Step 5. Count and normalize emissions | Tag | dogs | cats | chase | sleep | | --- | --- | --- | --- | --- | | N | 1/3 | 2/3 | 0 | 0 | | V | 0 | 0 | 1/2 | 1/2 | **Final answer.** π=(1,0); transitions and emissions are given in the completed tables. Each complete probability row sums to 1. **Yaad rakho:** Tag→tag table aur tag→word table ko mix mat karna. ### M2 08: Markov Paths & HMMs id : nlp-m2-08-markov-hmm-components title : M2 08: Markov Paths & HMMs title\_confirmed : true topic\_title : Markov paths, HMM components and training counts mode : m2 kind : focused status : available duration : 9:45 reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — HMM and Viterbi | Reviewed worked numericals id : overview-hmm-viterbi title : NLP M2 — HMM and Viterbi | Reviewed worked numericals title\_confirmed : true topic\_title : HMM Prob & Viterbi mode : m2 kind : supplemental status : available duration : 8:30 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Complete HMM factors, all Viterbi candidate scores and backpointers; clarified zero emissions. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids : - nlp-m2-08-markov-hmm-components - nlp-m2-09-hmm-joint-paths - nlp-m2-10-viterbi-decoding These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## M2 · HMM path probabilities: three complete numerical examples Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/read/m2-hmm--hot-cold-paths/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/read/m2-hmm/hot-cold-paths Unit 4, pages 47–73 The following model uses hidden weather states Hot (H) and Cold (C). Observations are numbers 1, 2 and 3. These numbers are observation labels, not quantities to multiply directly. Their emission probabilities come from the table. | Initial state | Probability | | --- | --- | | H | 0.8 | | C | 0.2 | | From → to | H | C | | --- | --- | --- | | H | 0.7 | 0.3 | | C | 0.4 | 0.6 | | Emission state | Observation 1 | Observation 2 | Observation 3 | | --- | --- | --- | --- | | H | 0.2 | 0.4 | 0.4 | | C | 0.5 | 0.4 | 0.1 | **Question. An HMM has π(H)=0.8, π(C)=0.2; H→H=0.7, H→C=0.3, C→H=0.4, C→C=0.6. Emissions for observations (1,2,3) are H:(0.2,0.4,0.4) and C:(0.5,0.4,0.1). Find the joint probability of hidden path H,C,H and observations 1,3,1.** #### Given Hidden path H→C→H. Observations 1→3→1. Model probabilities are specified in the question. There is no END factor. #### To find P(H,C,H; 1,3,1). #### Formula and why it applies $$ P=\pi_H b_H(1)a_{HC}b_C(3)a_{CH}b_H(1) $$ There are three hidden states and observations: three emissions, two transitions, and one initial probability. #### Solution ##### Step 1. Start Hot and emit 1 $$ 0.8\times0.2=0.16 $$ ##### Step 2. Transition Hot to Cold $$ 0.16\times0.3=0.048 $$ ##### Step 3. Cold emits 3 $$ 0.048\times0.1=0.0048 $$ ##### Step 4. Transition Cold to Hot $$ 0.0048\times0.4=0.00192 $$ ##### Step 5. Hot emits the last 1 $$ 0.00192\times0.2=0.000384 $$ **Final answer.** Joint probability = 0.000384 = 3.84×10⁻⁴. **Question. For π(H)=0.8, π(C)=0.2; H→H=0.7, H→C=0.3, C→H=0.4, C→C=0.6; emissions H:(0.2,0.4,0.4), C:(0.5,0.4,0.1) for labels (1,2,3), find the joint probability of path H,H,H and observations 1,3,1.** #### Given State path H→H→H, observations 1→3→1; no END factor. #### To find P(H,H,H;1,3,1). #### Formula and why it applies $$ P=\pi_H b_H(1)a_{HH}b_H(3)a_{HH}b_H(1) $$ The state remains Hot; each transition still contributes its probability 0.7. #### Solution ##### Step 1. Initial state and first emission $$ 0.8(0.2)=0.16 $$ ##### Step 2. First self-transition $$ 0.16(0.7)=0.112 $$ ##### Step 3. Emit observation 3 $$ 0.112(0.4)=0.0448 $$ ##### Step 4. Second self-transition $$ 0.0448(0.7)=0.03136 $$ ##### Step 5. Emit final observation 1 $$ 0.03136(0.2)=0.006272 $$ **Final answer.** Joint probability = 0.006272. A self-transition is not automatically probability 1. **Question. For π(H)=0.8, π(C)=0.2; transitions H:(0.7,0.3), C:(0.4,0.6) in next-state order H,C; emissions H:(0.2,0.4,0.4), C:(0.5,0.4,0.1) for observations 1,2,3, find the score of path C,C,H with observations 1,3,1.** #### Given Hidden path C→C→H. Observation sequence 1→3→1. No end-state factor. #### To find The joint path-and-observation probability. #### Formula and why it applies $$ P=\pi_C b_C(1)a_{CC}b_C(3)a_{CH}b_H(1) $$ The initial state is Cold, so use π(C), not π(H). Each emission is selected from the state at the same position. #### Solution ##### Step 1. Start Cold and emit 1 $$ 0.2(0.5)=0.1 $$ ##### Step 2. Stay Cold $$ 0.1(0.6)=0.06 $$ ##### Step 3. Emit 3 from Cold $$ 0.06(0.1)=0.006 $$ ##### Step 4. Move Cold to Hot $$ 0.006(0.4)=0.0024 $$ ##### Step 5. Emit 1 from Hot $$ 0.0024(0.2)=0.00048 $$ **Final answer.** Joint probability = 0.00048. **Yaad rakho:** Har column par: state ka emission. Columns ke beech: transition. ### HMM Path Probabilities id : nlp-m2-09-hmm-joint-paths title : HMM Path Probabilities title\_confirmed : true topic\_title : Complete HMM path scores and observation probabilities mode : m2 kind : focused status : available duration : 8:34 reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — HMM and Viterbi | Reviewed worked numericals id : overview-hmm-viterbi title : NLP M2 — HMM and Viterbi | Reviewed worked numericals title\_confirmed : true topic\_title : HMM Prob & Viterbi mode : m2 kind : supplemental status : available duration : 8:30 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Complete HMM factors, all Viterbi candidate scores and backpointers; clarified zero emissions. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids : - nlp-m2-08-markov-hmm-components - nlp-m2-09-hmm-joint-paths - nlp-m2-10-viterbi-decoding These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## M2 · POS HMM worked corpus: Mary, Jane, can, will and spot Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/read/m2-hmm--tagging-corpus-case/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/read/m2-hmm/tagging-corpus-case Unit 4, pages 47–73 The class example contains words that can be either names or ordinary verbs/modal auxiliaries. The term is **modal** (M), not “model”. To avoid guessing tags from spelling, the training corpus below supplies them explicitly and lowercases words for the emission vocabulary. | Training sentence | Gold tags | | --- | --- | | mary jane can see will | N N M V N | | spot will see mary | N M V N | | will jane spot mary | M N V N | | mary will pat spot | N M V N | There are 9 noun tokens, 4 modal tokens and 4 verb tokens. Each sentence includes a start and end transition. For nouns, the next event can be another noun, modal, verb or END. | Current → next | N | M | V | END | | --- | --- | --- | --- | --- | | START | 3/4 | 1/4 | 0 | 0 | | N | 1/9 | 3/9 | 1/9 | 4/9 | | M | 1/4 | 0 | 3/4 | 0 | | V | 4/4 | 0 | 0 | 0 | | Tag → word | mary | jane | can | see | will | spot | pat | | --- | --- | --- | --- | --- | --- | --- | --- | | N | 4/9 | 2/9 | 0 | 0 | 1/9 | 2/9 | 0 | | M | 0 | 0 | 1/4 | 0 | 3/4 | 0 | 0 | | V | 0 | 0 | 0 | 2/4 | 0 | 1/4 | 1/4 | **Question. Using the fully tagged four-sentence corpus above, calculate the joint score of “will jane spot mary” assigned tags M,N,V,N, including END.** #### Given Required factors from the tables: START→M=1/4; emit will|M=3/4; M→N=1/4; jane|N=2/9; N→V=1/9; spot|V=1/4; V→N=1; mary|N=4/9; N→END=4/9. #### To find The specified tag-path score, including words and sentence end. #### Formula and why it applies $$ P=\pi_M b_M(will)a_{MN}b_N(jane)a_{NV}b_V(spot)a_{VN}b_N(mary)a_{N,END} $$ Both transitions and emissions are needed. A tag transition alone cannot tell us whether a particular word is plausible. #### Solution ##### Step 1. Start and emit will $$ \frac14\times\frac34=\frac3{16} $$ ##### Step 2. Move to N and emit jane $$ \frac3{16}\times\frac14\times\frac29=\frac1{96} $$ ##### Step 3. Move to V and emit spot $$ \frac1{96}\times\frac19\times\frac14=\frac1{3456} $$ ##### Step 4. Move to N and emit mary $$ \frac1{3456}\times1\times\frac49=\frac1{7776} $$ ##### Step 5. End the sentence $$ \frac1{7776}\times\frac49=\frac1{17496}\approx0.0000571559 $$ **Final answer.** The joint score is 1/17496 ≈ 0.0000571559. **Question. For the same tagged corpus and tables, explain the unsmoothed score of “will can spot mary” assigned tags M,V,N,N.** #### Given The proposed second word is can with hidden tag V. In the training corpus, can occurs only with M, so b\_V(can)=0. #### To find Whether the complete path score is zero and what this means. #### Formula and why it applies $$ P(\text{path,words})=(\text{other required factors})\times b_V(can) $$ A product containing one zero factor is zero even if every other factor is positive. #### Solution ##### Step 1. Find the critical emission $$ b_V(can)=C(can,V)/C(V)=0/4=0 $$ ##### Step 2. Use the full-product consequence $$ P(\text{path,words})=0 $$ ##### Step 3. Interpret correctly This unsmoothed corpus model has no evidence for can as a verb. It does not prove that a verb usage of can is impossible in English. **Final answer.** The unsmoothed path score is zero. ### HMM Path Probabilities id : nlp-m2-09-hmm-joint-paths title : HMM Path Probabilities title\_confirmed : true topic\_title : Complete HMM path scores and observation probabilities mode : m2 kind : focused status : available duration : 8:34 reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — HMM and Viterbi | Reviewed worked numericals id : overview-hmm-viterbi title : NLP M2 — HMM and Viterbi | Reviewed worked numericals title\_confirmed : true topic\_title : HMM Prob & Viterbi mode : m2 kind : supplemental status : available duration : 8:30 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Complete HMM factors, all Viterbi candidate scores and backpointers; clarified zero emissions. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids : - nlp-m2-08-markov-hmm-components - nlp-m2-09-hmm-joint-paths - nlp-m2-10-viterbi-decoding These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## M2 · Why exhaustive search grows exponentially Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/read/m2-hmm-decoding--sequence-count/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/read/m2-hmm-decoding/sequence-count Unit 4 page 73: Viterbi motivation, explained with small worked extensions If each of T observed words can receive any of N tags, there are N choices at each position. Thus the total number of complete tag sequences is N multiplied by itself T times. This counts possible assignments before grammar rules or zero probabilities eliminate any. $$ \text{Number of tag sequences}=N^T $$ **Question. A four-word sentence is tagged using three possible POS tags per word. How many complete tag sequences are possible?** #### Given N=3 possible tags per position; T=4 word positions. #### To find The number of tag sequences. #### Formula and why it applies $$ M=N^T $$ Every independent position contributes a factor of three choices. #### Solution ##### Step 1. Write one factor per position $$ M=3\times3\times3\times3 $$ ##### Step 2. Multiply $$ 3\times3=9,\quad9\times3=27,\quad27\times3=81 $$ **Final answer.** 81 possible tag sequences. **Question. A five-word sentence has two candidate tags per word. Count its tag sequences.** #### Given N=2; T=5. #### To find M. #### Formula and why it applies $$ M=N^T $$ The length is the exponent, not a multiplier. #### Solution ##### Step 1. Substitute $$ M=2^5 $$ ##### Step 2. Evaluate $$ 2\times2\times2\times2\times2=32 $$ **Final answer.** 32 possible sequences. **Question. Using 45 possible tags, compare the number of tag sequences for sentences of length 2 and 3.** #### Given N=45. First T=2, then T=3. #### To find Both counts and the growth factor. #### Formula and why it applies $$ M_2=45^2,\quad M_3=45^3 $$ Adding one position multiplies the possibilities by the number of tag choices. #### Solution ##### Step 1. Two words $$ 45^2=2025 $$ ##### Step 2. Three words $$ 45^3=2025\times45=91125 $$ ##### Step 3. Compare $$ 91125/2025=45 $$ **Final answer.** 2 words: 2025 paths. 3 words: 91125 paths. One additional word multiplies the count by 45. The slides introduce Viterbi to avoid enumerating every path. It reuses the best prefix score for each ending state. For a dense first-order HMM, it needs O(TN²) recurrence work instead of enumerating NT complete paths. **Yaad rakho:** Choices per word ki power = sentence length. ### M2 10 Viterbi Decoding id : nlp-m2-10-viterbi-decoding title : M2 10 Viterbi Decoding title\_confirmed : true topic\_title : Viterbi decoding with backpointers mode : m2 kind : focused status : available duration : 9:34 reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — HMM and Viterbi | Reviewed worked numericals id : overview-hmm-viterbi title : NLP M2 — HMM and Viterbi | Reviewed worked numericals title\_confirmed : true topic\_title : HMM Prob & Viterbi mode : m2 kind : supplemental status : available duration : 8:30 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Complete HMM factors, all Viterbi candidate scores and backpointers; clarified zero emissions. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids : - nlp-m2-08-markov-hmm-components - nlp-m2-09-hmm-joint-paths - nlp-m2-10-viterbi-decoding These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## M2 · Viterbi decoding: compare incoming paths, keep the best, trace back Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/read/m2-hmm-decoding--viterbi-explained/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/read/m2-hmm-decoding/viterbi-explained Unit 4 page 73: Viterbi motivation, explained with small worked extensions **The Viterbi algorithm** finds the most likely hidden-state sequence for a fixed observation sequence. At each position, it keeps the best path ending in each state. This is safe because, under the HMM assumptions, future extensions depend on that ending state, not the discarded details of the prefix. $$ \delta_1(j)=\pi_jb_j(o_1) $$ $$ \delta_t(j)=b_j(o_t)\max_i[\delta_{t-1}(i)a_{ij}] $$ $$ \psi_t(j)=\arg\max_i[\delta_{t-1}(i)a_{ij}] $$ | Symbol | Meaning in ordinary words | | --- | --- | | δₜ(j) | Score of the best prefix that ends in state j at position t. | | ψₜ(j) | Previous state that supplied that best prefix; keep it as a backpointer. | | max | Choose the larger candidate score; do not add. | | argmax | Remember which previous state won, rather than only its value. | Initialize: start × first emission → For each next state: compare incoming scores → Multiply winner by this state's emission → Keep score + winning predecessor → Choose best final state and trace arrows backward Do not choose the best state independently at each word. The best last state can come from a different best predecessor than a local guess. Backpointers recover one consistent complete path. If scores tie, either tied path is optimal under the model; state the tie. **Question. Use an HMM with π(H)=0.8, π(C)=0.2, transitions H:(0.7,0.3), C:(0.4,0.6) in H,C order, and emissions H:(0.2,0.4,0.4), C:(0.5,0.4,0.1) for observations 1,2,3. Find the most likely hidden path for observations 1,3,1 using Viterbi.** #### Given Initial probabilities: π(H)=0.8, π(C)=0.2. Transition matrix: H→H=0.7, H→C=0.3, C→H=0.4, C→C=0.6. Emissions for labels (1,2,3): H=(0.2,0.4,0.4), C=(0.5,0.4,0.1). There is no END-state factor. #### To find The best hidden-state sequence and its joint score. #### Formula and why it applies $$ \delta_1(j)=\pi_jb_j(o_1),\quad\delta_t(j)=b_j(o_t)\max_i[\delta_{t-1}(i)a_{ij}] $$ Viterbi keeps two best prefix scores per column, one ending H and one ending C. #### Solution ##### Step 1. Initialize at observation 1 $$ \delta_1(H)=0.8(0.2)=0.16 $$ $$ \delta_1(C)=0.2(0.5)=0.10 $$ ##### Step 2. At observation 3, calculate the H cell $$ H\to H:\ 0.16(0.7)=0.112 $$ $$ C\to H:\ 0.10(0.4)=0.040 $$ $$ \delta_2(H)=\max(0.112,0.040)(0.4)=0.0448 $$ Backpointer for this H cell: H. ##### Step 3. At observation 3, calculate the C cell $$ H\to C:\ 0.16(0.3)=0.048 $$ $$ C\to C:\ 0.10(0.6)=0.060 $$ $$ \delta_2(C)=\max(0.048,0.060)(0.1)=0.006 $$ Backpointer for this C cell: C. ##### Step 4. At final observation 1, calculate H $$ H\to H:\ 0.0448(0.7)=0.03136 $$ $$ C\to H:\ 0.006(0.4)=0.0024 $$ $$ \delta_3(H)=0.03136(0.2)=0.006272 $$ Backpointer: H. ##### Step 5. At final observation 1, calculate C $$ H\to C:\ 0.0448(0.3)=0.01344 $$ $$ C\to C:\ 0.006(0.6)=0.0036 $$ $$ \delta_3(C)=0.01344(0.5)=0.00672 $$ Backpointer: H. ##### Step 6. Select the final state 0.00672 > 0.006272, so the best final state is C. ##### Step 7. Trace back Final C came from H at position2. That H came from H at position1. Reading forward gives H,H,C. ##### Step 8. Verify the recovered complete path $$ 0.8(0.2)(0.7)(0.4)(0.3)(0.5)=0.00672 $$ **Final answer.** Best path H→H→C with joint score 0.00672. **Question. For π(H)=0.8, π(C)=0.2; transitions H:(0.7,0.3), C:(0.4,0.6); emissions H:(0.2,0.4,0.4), C:(0.5,0.4,0.1) for labels1,2,3, decode observations 3,1.** #### Given Initial probabilities: π(H)=0.8, π(C)=0.2. Transition matrix: H→H=0.7, H→C=0.3, C→H=0.4, C→C=0.6. Emissions for labels (1,2,3): H=(0.2,0.4,0.4), C=(0.5,0.4,0.1). There is no END-state factor. #### To find Best two-state hidden path and score. #### Formula and why it applies $$ \delta_1(j)=\pi_jb_j(3),\quad\delta_2(j)=b_j(1)\max_i[\delta_1(i)a_{ij}] $$ The same recurrence applies even though the observation order changed. #### Solution ##### Step 1. Initialize observation 3 $$ \delta_1(H)=0.8(0.4)=0.32,\quad\delta_1(C)=0.2(0.1)=0.02 $$ ##### Step 2. Calculate the final H $$ \max(0.32(0.7),0.02(0.4))(0.2) $$ $$ =\max(0.224,0.008)(0.2)=0.0448 $$ Winning predecessor H. ##### Step 3. Calculate the final C $$ \max(0.32(0.3),0.02(0.6))(0.5) $$ $$ =\max(0.096,0.012)(0.5)=0.048 $$ Winning predecessor H. ##### Step 4. Select and trace Final C wins with 0.048 and came from H. The path is H,C. **Final answer.** Best path H→C, score 0.048. **Question. For π(H)=0.8, π(C)=0.2; transitions H:(0.7,0.3), C:(0.4,0.6); emissions H:(0.2,0.4,0.4), C:(0.5,0.4,0.1) for labels1,2,3, decode observations 2,2.** #### Given Initial probabilities: π(H)=0.8, π(C)=0.2. Transition matrix: H→H=0.7, H→C=0.3, C→H=0.4, C→C=0.6. Emissions for labels (1,2,3): H=(0.2,0.4,0.4), C=(0.5,0.4,0.1). There is no END-state factor. #### To find Best path and score. #### Formula and why it applies $$ \delta_1(j)=\pi_jb_j(2),\quad\delta_2(j)=b_j(2)\max_i[\delta_1(i)a_{ij}] $$ Both states emit2 with probability 0.4, but their initial and transition probabilities still differ. #### Solution ##### Step 1. Initialize $$ \delta_1(H)=0.8(0.4)=0.32,\quad\delta_1(C)=0.2(0.4)=0.08 $$ ##### Step 2. Final H $$ \max(0.32(0.7),0.08(0.4))(0.4) $$ $$ =\max(0.224,0.032)(0.4)=0.0896 $$ Predecessor H. ##### Step 3. Final C $$ \max(0.32(0.3),0.08(0.6))(0.4) $$ $$ =\max(0.096,0.048)(0.4)=0.0384 $$ Predecessor H. ##### Step 4. Select and trace The final H score 0.0896 is larger. Its predecessor is H. **Final answer.** Best path H→H with score 0.0896. **Yaad rakho:** Har destination ke liye incoming paths compare; winner ka arrow yaad. ### M2 10 Viterbi Decoding id : nlp-m2-10-viterbi-decoding title : M2 10 Viterbi Decoding title\_confirmed : true topic\_title : Viterbi decoding with backpointers mode : m2 kind : focused status : available duration : 9:34 reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — HMM and Viterbi | Reviewed worked numericals id : overview-hmm-viterbi title : NLP M2 — HMM and Viterbi | Reviewed worked numericals title\_confirmed : true topic\_title : HMM Prob & Viterbi mode : m2 kind : supplemental status : available duration : 8:30 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Complete HMM factors, all Viterbi candidate scores and backpointers; clarified zero emissions. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids : - nlp-m2-08-markov-hmm-components - nlp-m2-09-hmm-joint-paths - nlp-m2-10-viterbi-decoding These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## M2 · A path score is not the probability of the observed sentence Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/read/m2-hmm-decoding--joint-vs-posterior/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/read/m2-hmm-decoding/joint-vs-posterior Unit 4 page 73: Viterbi motivation, explained with small worked extensions **Question. An HMM has π(H)=0.8, π(C)=0.2; transitions H:(0.7,0.3), C:(0.4,0.6); emissions H:(0.2,0.4,0.4), C:(0.5,0.4,0.1) for labels1,2,3. For observations 1,3, calculate the probability of the observations by summing all four two-state paths.** #### Given Initial probabilities: π(H)=0.8, π(C)=0.2. Transition matrix: H→H=0.7, H→C=0.3, C→H=0.4, C→C=0.6. Emissions for labels (1,2,3): H=(0.2,0.4,0.4), C=(0.5,0.4,0.1). There is no END-state factor. #### To find P(observations 1,3), not just the best-path score. #### Formula and why it applies $$ P(o_1,o_2)=\sum_{s_1,s_2}\pi_{s_1}b_{s_1}(o_1)a_{s_1s_2}b_{s_2}(o_2) $$ The hidden path is unspecified, so its mutually exclusive alternatives must be summed. #### Solution ##### Step 1. Score H,H $$ 0.8(0.2)(0.7)(0.4)=0.0448 $$ ##### Step 2. Score H,C $$ 0.8(0.2)(0.3)(0.1)=0.0048 $$ ##### Step 3. Score C,H $$ 0.2(0.5)(0.4)(0.4)=0.016 $$ ##### Step 4. Score C,C $$ 0.2(0.5)(0.6)(0.1)=0.006 $$ ##### Step 5. Add all paths $$ P(1,3)=0.0448+0.0048+0.016+0.006=0.0716 $$ **Final answer.** Observation probability =0.0716. The best single-path score 0.0448 is a different quantity. If asked for the posterior of path H,H given observations 1,3, normalize: 0.0448/0.0716 ≈0.625698. Viterbi chooses the best path using joint scores because the observation denominator is common to every candidate. It does not compute that denominator by taking a maximum. ### HMM Path Probabilities id : nlp-m2-09-hmm-joint-paths title : HMM Path Probabilities title\_confirmed : true topic\_title : Complete HMM path scores and observation probabilities mode : m2 kind : focused status : available duration : 8:34 reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — HMM and Viterbi | Reviewed worked numericals id : overview-hmm-viterbi title : NLP M2 — HMM and Viterbi | Reviewed worked numericals title\_confirmed : true topic\_title : HMM Prob & Viterbi mode : m2 kind : supplemental status : available duration : 8:30 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Complete HMM factors, all Viterbi candidate scores and backpointers; clarified zero emissions. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids : - nlp-m2-08-markov-hmm-components - nlp-m2-09-hmm-joint-paths - nlp-m2-10-viterbi-decoding These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## M2 · Chunking groups words; chinking removes words from a candidate chunk Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/read/m2-chunks-grammar--chunking-chinking/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/read/m2-chunks-grammar/chunking-chinking Unit 4, pages 14–19, 74–97 **Chunking**, also called shallow parsing, groups adjacent tagged words into meaningful non-overlapping units, commonly noun phrases. It usually identifies local phrase boundaries without producing the complete hierarchical structure of a sentence. **Chinking** removes specified tokens from an existing candidate chunk, splitting or trimming it. The/DT small/JJ cat/NN sat/VBD on/IN the/DT mat/NN → NP: [The small cat] → Outside: sat on → NP: [the mat] A POS tagger labels individual words. A chunker groups words into phrases. A full parser shows how phrases relate recursively to form the entire sentence. These are related tasks with different outputs. | Operation | Input | Output | | --- | --- | --- | | POS tagging | The small cat | The/DT small/JJ cat/NN | | NP chunking | Tagged tokens | [NP The small cat] | | Chinking | A broad candidate chunk | Remove verbs/prepositions, keep noun-phrase segments | | Full parsing | Sentence + grammar | A nested tree with sentence, noun phrase and verb phrase nodes | **Rule-based chunking** uses tag patterns. Statistical or learned chunkers infer boundaries from labeled examples. A useful NP rule is an optional determiner, followed by zero or more adjectives, followed by a noun. In NLTK-style notation: `NP: {
?*}`. Here ? means zero or one occurrence; \* means zero or more; + means one or more when used. The braces mean “form a chunk around the matched sequence”. **Question. Apply the NP rule optional DT, zero or more JJ, then one NN to “The/DT quick/JJ brown/JJ fox/NN jumps/VBZ over/IN the/DT lazy/JJ dog/NN”.** #### Given Tagged input is supplied. Rule: DT? JJ\* NN. #### To find All noun-phrase chunks. #### Formula and why it applies `NP → DT? JJ* NN` The rule groups a noun with its preceding determiner and adjectives; a verb or preposition does not match this NP pattern. #### Solution ##### Step 1. Start at The The matches optional DT; quick and brown both match JJ\*; fox matches NN. ##### Step 2. Form the first NP **[NP The quick brown fox]** ##### Step 3. Skip nonmatching tokens jumps/VBZ and over/IN do not satisfy the NP pattern. ##### Step 4. Match the second NP the is DT, lazy is JJ, dog is NN: **[NP the lazy dog]**. **Final answer.** [NP The quick brown fox] jumps over [NP the lazy dog]. **Question. Using NP → DT? JJ\* NN, identify the chunks in “bright/JJ stars/NN shine/VBP and/CC a/DT moon/NN glows/VBZ”. Use the supplied tags exactly.** #### Given Input tags label stars as NN for this exercise. Rule requires optional DT, zero or more JJ, then NN. #### To find The matched NP spans. #### Formula and why it applies `NP → DT? JJ* NN` The optional determiner may be absent and the adjective repetition may be empty. #### Solution ##### Step 1. Match bright stars No determiner is present, which DT? permits. bright matches JJ; stars matches NN. ##### Step 2. Inspect shine and and VBP and CC fall outside the NP rule. ##### Step 3. Match a moon a matches DT; no adjective is present; moon matches NN. **Final answer.** Chunks are [NP bright stars] and [NP a moon]. With ordinary Penn tagging, stars is usually NNS; broaden the noun pattern to include NNS when that tag is used. **Question. Begin with one candidate chunk containing “The/DT cat/NN sat/VBD on/IN the/DT mat/NN”. Chink out verbs and prepositions. Show the remaining chunks.** #### Given Initial candidate: [The cat sat on the mat]. Remove tags VBD and IN. #### To find The chunk segments after exclusion. #### Formula and why it applies Chinking removes matched tokens from the existing chunk. Removing the middle verb/preposition region divides the original candidate into two remaining contiguous spans. #### Solution ##### Step 1. Locate the tokens to remove sat is VBD and on is IN. ##### Step 2. Remove those tokens from chunk membership The remaining token groups are “The cat” and “the mat”. sat and on still exist in the sentence; they are simply outside these chunks. ##### Step 3. Write the final chunked sentence **[The cat]** sat on **[the mat]**. **Final answer.** Two chunks remain: The cat; the mat. **Yaad rakho:** Chunking = andar group karo; chinking = unwanted tokens ko group se bahar karo. ### NLP M2 — Chunking and Chinking | Reviewed rules and worked examples id : nlp-m2-11-chunking-chinking title : NLP M2 — Chunking and Chinking | Reviewed rules and worked examples title\_confirmed : true topic\_title : Chunking and chinking from POS-tagged words mode : m2 kind : focused status : available duration : 8:22 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Correct POS labels including JJ = adjective, full NP rule application and chinking boundaries. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## M2 · CFG: terminals, nonterminals, production rules and parse trees Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/read/m2-chunks-grammar--cfg-foundation/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/read/m2-chunks-grammar/cfg-foundation Unit 4, pages 14–19, 74–97 **A context-free grammar (CFG)** specifies a language using production rules whose left side is a single nonterminal. Formally, G=(N,Σ,R,S): N is the set of nonterminals, Σ the terminal vocabulary, R the production rules, and S the start symbol. A rule can expand its nonterminal regardless of the neighboring symbols; that is the meaning of “context-free”. | Concept | Plain explanation | Example | | --- | --- | --- | | Terminal | A word at a leaf; it is not expanded further | the, girl, read | | Nonterminal | A grammatical category that can be expanded | S, NP, VP, Det, N, V | | Production | A permitted replacement | S → NP VP | | Start symbol | Category for a complete sentence | S | | Lexicon | Word-to-category rules | N → girl | paper | | Derivation | Sequence of rule applications | S ⇒ NP VP ⇒ Det N VP ⇒ … | | Parse tree | Hierarchical picture of those expansions | Root S, internal phrases, word leaves | $$ S\to NP\ VP,\quad NP\to Det\ N,\quad VP\to V\ NP $$ Read S→NP VP as “a sentence can contain a noun phrase followed by a verb phrase”. It does not mean subtraction or multiplication. A vertical bar in a grammar rule means alternatives: Det→the | a allows either the or a. This is different from the probability bar meaning “given”. S: the whole sentence → NP: the girl + VP: read the paper → NP→Det N; VP→V NP → Leaves in order: the / girl / read / the / paper **A constituent** is a group of words that acts as a grammatical unit, such as “the girl”. **Syntactic parsing** assigns a grammatical structure to an input sentence using a grammar. It can support grammar checking, information extraction, translation and later semantic analysis. Failure under one restricted grammar means that grammar could not derive the sentence; it does not necessarily prove the sentence is invalid English. **A treebank** is a corpus annotated with parse trees. It supplies training/evaluation structures for parsers and lets linguists study recurring syntactic patterns. A tagged corpus gives word tags; a treebank additionally records relationships among phrases. | Representation | Example information retained | | --- | --- | | Bag of words | Words occur, but word order/grammar is lost | | POS tags | Each word's local grammatical category | | Chunks | Short phrase boundaries | | Parse tree | Nested constituent structure of the whole sentence | **Yaad rakho:** Grammar rule ek expansion instruction hai: S ko NP aur VP mein kholo. ### NLP M2 — CFG and top-down parsing | Reviewed derivations and trees id : nlp-m2-12-cfg-top-down-parsing title : NLP M2 — CFG and top-down parsing | Reviewed derivations and trees title\_confirmed : true topic\_title : CFG, ambiguity and top-down parsing mode : m2 kind : focused status : available duration : 8:54 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Complete grammars, derivations and valid parse trees; corrected the acceptance condition. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## M2 · Ambiguity and grammar coverage Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/read/m2-chunks-grammar--structural-ambiguity/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/read/m2-chunks-grammar/structural-ambiguity Unit 4, pages 14–19, 74–97 **Structural ambiguity** occurs when a sentence has more than one valid parse tree. In “I saw the man with a telescope”, the prepositional phrase “with a telescope” can attach to saw (the instrument I used) or to the man (the man possessing the telescope). The words and their order stay the same, but the grouping changes. | Reading | Attachment | Meaning | | --- | --- | --- | | I [saw the man] [with a telescope] | PP modifies VP | I used a telescope to see the man. | | I saw [the man with a telescope] | PP modifies NP | The man had a telescope. | A parser may need context, probabilities or semantic constraints to choose between trees. A plain CFG describes permitted structures but does not itself assign a probability to each tree. Do not invent rule probabilities for an ordinary CFG question. Lexical ambiguity also matters: book can be a noun or verb. In “Book the flight”, the imperative analysis begins S→VP, followed by VP→V NP. A parser that initially chooses book as a noun may reach a dead end and must try another permitted analysis. **Yaad rakho:** Same words, alag grouping → alag meaning. ### NLP M2 — CFG and top-down parsing | Reviewed derivations and trees id : nlp-m2-12-cfg-top-down-parsing title : NLP M2 — CFG and top-down parsing | Reviewed derivations and trees title\_confirmed : true topic\_title : CFG, ambiguity and top-down parsing mode : m2 kind : focused status : available duration : 8:54 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Complete grammars, derivations and valid parse trees; corrected the acceptance condition. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## M2 · Recursive-descent top-down parsing Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/read/m2-parsing--top-down-theory/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/read/m2-parsing/top-down-theory Unit 4, pages 98–116 **Top-down parsing** starts at the grammar's start symbol S and expands nonterminals until the leaves match the input words in order. **Recursive descent** commonly implements each nonterminal as a procedure that tries its productions. It builds from the root toward the leaves. Start with S → Choose an allowed S production → Expand its next nonterminal → Match the resulting terminal to the input → Continue, or backtrack if the choice fails **Algorithm.** Start with S as the current goal. Choose a production for the leftmost unexpanded nonterminal. Replace that nonterminal by the production's right side. If a terminal appears, compare it with the next input token. If it matches, consume the token; if it does not, reject that branch and try another production. Accept only when the complete input is consumed and all grammar goals are satisfied. A **leftmost derivation** always expands the leftmost remaining nonterminal first. Top-down parsing can make guesses that the actual words cannot support, so it may need backtracking. A naive recursive-descent parser can loop on left-recursive rules such as NP→NP PP unless the grammar or algorithm is changed. Predictive LL(1) parsing chooses productions using one lookahead token when the grammar permits; not every English CFG is LL(1). | Approach | Begins with | Direction | Common difficulty | | --- | --- | --- | --- | | Top-down | Start symbol S | Root to word leaves | Predictions that do not match input; left recursion | | Bottom-up | Actual input tokens | Leaves to root S | Reductions that cannot contribute to a complete sentence | **Yaad rakho:** Top-down: sentence ka structure pehle, words tak neeche aao. ### NLP M2 — CFG and top-down parsing | Reviewed derivations and trees id : nlp-m2-12-cfg-top-down-parsing title : NLP M2 — CFG and top-down parsing | Reviewed derivations and trees title\_confirmed : true topic\_title : CFG, ambiguity and top-down parsing mode : m2 kind : focused status : available duration : 8:54 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Complete grammars, derivations and valid parse trees; corrected the acceptance condition. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## M2 · Three full top-down derivations with parse trees Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/read/m2-parsing--top-down-examples/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/read/m2-parsing/top-down-examples Unit 4, pages 98–116 **Question. Using S→NP VP, NP→Det N, VP→V NP, Det→the, N→girl|paper and V→read, give a leftmost top-down derivation and parse tree for “the girl read the paper”.** #### Given | Rule type | Rules | | --- | --- | | Structure | S → NP VP; NP → Det N; VP → V NP | | Lexicon | Det → the; N → girl | paper; V → read | #### To find A successful derivation and tree whose leaves equal the input. #### Formula and why it applies At every stage expand the leftmost nonterminal using a supplied production. A top-down derivation begins at S; the input determines which terminal alternatives can succeed. #### Solution ##### Step 1. Begin at the sentence symbol S ##### Step 2. Expand the sentence S ⇒ NP VP ##### Step 3. Expand its first noun phrase ⇒ Det N VP ##### Step 4. Match the first determiner ⇒ the N VP ##### Step 5. Match the subject noun ⇒ the girl VP ##### Step 6. Expand the verb phrase ⇒ the girl V NP ##### Step 7. Match the verb ⇒ the girl read NP ##### Step 8. Expand the object noun phrase ⇒ the girl read Det N ##### Step 9. Match its determiner ⇒ the girl read the N ##### Step 10. Match the last noun ⇒ the girl read the paper ##### Step 11. Draw the tree Parse tree: the girl read the paperSNPDettheNgirlVPVreadNPDettheNpaper Parse tree: the girl read the paper **Final answer.** The grammar derives the complete sentence. The subject NP is “the girl”; the object NP is “the paper”. **Question. Use S→NP VP, NP→Det N|N, VP→V NP, Det→A, N→restaurant|Dosas and V→serves to parse “A restaurant serves Dosas” by a leftmost top-down derivation.** #### Given | Rule type | Rules | | --- | --- | | Structure | S → NP VP; NP → Det N | N; VP → V NP | | Lexicon | Det → A; N → restaurant | Dosas; V → serves | #### To find A derivation and parse tree. #### Formula and why it applies Expand the leftmost nonterminal; choose NP→Det N for the subject and NP→N for the object. The object has no determiner, so forcing NP→Det N there would fail. #### Solution ##### Step 1. Start and expand S S ⇒ NP VP ##### Step 2. Choose the subject NP ⇒ Det N VP ##### Step 3. Match A ⇒ A N VP ##### Step 4. Match restaurant ⇒ A restaurant VP ##### Step 5. Expand the VP ⇒ A restaurant V NP ##### Step 6. Match serves ⇒ A restaurant serves NP ##### Step 7. Choose the object NP rule ⇒ A restaurant serves N ##### Step 8. Match Dosas ⇒ A restaurant serves Dosas ##### Step 9. Draw the successful tree Parse tree: A restaurant serves DosasSNPDetANrestaurantVPVservesNPNDosas Parse tree: A restaurant serves Dosas **Final answer.** The parse succeeds with subject NP→Det N and object NP→N. **Question. Using S→NP VP|VP, NP→Det N, VP→V NP, Det→that, N→book|flight and V→book, derive the imperative “book that flight” and explain the role of book.** #### Given | Rule type | Rules | | --- | --- | | Structure | S → NP VP | VP; NP → Det N; VP → V NP | | Lexicon | Det → that; N → book | flight; V → book | #### To find The successful derivation, tree and POS of book. #### Formula and why it applies Choose a production for S that can derive the actual input; an imperative may consist of VP. The grammar allows book as both noun and verb, but this imperative needs the verb reading. #### Solution ##### Step 1. Choose the imperative sentence rule S ⇒ VP ##### Step 2. Expand the verb phrase ⇒ V NP ##### Step 3. Choose the verb book ⇒ book NP ##### Step 4. Expand the object ⇒ book Det N ##### Step 5. Match that ⇒ book that N ##### Step 6. Match flight ⇒ book that flight ##### Step 7. Draw the tree Imperative parse: book that flightSVPVbookNPDetthatNflight Imperative parse: book that flight **Final answer.** book is V; that flight is NP; S→VP derives the imperative. ### NLP M2 — CFG and top-down parsing | Reviewed derivations and trees id : nlp-m2-12-cfg-top-down-parsing title : NLP M2 — CFG and top-down parsing | Reviewed derivations and trees title\_confirmed : true topic\_title : CFG, ambiguity and top-down parsing mode : m2 kind : focused status : available duration : 8:54 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Complete grammars, derivations and valid parse trees; corrected the acceptance condition. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## M2 · Shift-reduce bottom-up parsing: stack, input and actions Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/read/m2-parsing--shift-reduce-theory/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/read/m2-parsing/shift-reduce-theory Unit 4, pages 98–116 **Bottom-up parsing** starts from the input tokens and combines them into larger constituents until the start symbol is reached. A **shift-reduce parser** maintains a stack of recognized material and a list of remaining input tokens. | Action | Exact operation | Example | | --- | --- | --- | | Shift | Move the next input token onto the stack | Stack: the; remaining input: dog runs | | Reduce | Replace a suffix of the stack matching a rule's RHS with its LHS | Det N becomes NP using NP→Det N | | Accept | Stack is S and input is exhausted | Stack: S; input: empty | | Fail/backtrack | Chosen reductions cannot complete a parse | A noun reading of book blocks an imperative parse | Reduction uses a production in reverse: the grammar NP→Det N expands an NP top-down, but reduces Det N into NP bottom-up. The stack's **rightmost item is its top** in the traces below. Each row shows the state **after** the listed action, avoiding an off-by-one mismatch between actions and stack contents. A lexical reduction such as N→dog still counts as a reduction. Do not skip the sequence from word to category to phrase when the question asks for a complete trace. Backtracking means returning to an earlier choice and trying another applicable rule, not changing the original input. **Yaad rakho:** Shift = input se stack. Reduce = chhote pieces se bada grammatical unit. ### M2 13 — Shift-reduce parsing and backtracking id : nlp-m2-13-bottom-up-shift-reduce title : M2 13 — Shift-reduce parsing and backtracking title\_confirmed : false topic\_title : Shift-reduce parsing and backtracking mode : m2 kind : focused status : queued duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## M2 · Three full shift-reduce traces Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/read/m2-parsing--shift-reduce-examples/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/read/m2-parsing/shift-reduce-examples Unit 4, pages 98–116 **Question. Show a complete shift-reduce parse of “the big dog sees the cat” using S→NP VP, NP→Det N|Det Adj N, VP→V NP, Det→the, Adj→big, N→dog|cat and V→sees.** #### Given Grammar is as specified. Stack starts empty; input is the big dog sees the cat. Top of stack is on the right. #### To find The stack, remaining input and action at every step. #### Formula and why it applies Shift tokens; reduce matched right-hand sides to their left-hand nonterminals. The first noun phrase includes an adjective and must use NP→Det Adj N. #### Solution ##### Step 1. Trace each action | Step / action | Stack after action | Remaining input | | --- | --- | --- | | 0. Start | ∅ | the big dog sees the cat | | 1. Shift the | the | big dog sees the cat | | 2. Reduce Det→the | Det | big dog sees the cat | | 3. Shift big | Det big | dog sees the cat | | 4. Reduce Adj→big | Det Adj | dog sees the cat | | 5. Shift dog | Det Adj dog | sees the cat | | 6. Reduce N→dog | Det Adj N | sees the cat | | 7. Reduce NP→Det Adj N | NP | sees the cat | | 8. Shift sees | NP sees | the cat | | 9. Reduce V→sees | NP V | the cat | | 10. Shift the | NP V the | cat | | 11. Reduce Det→the | NP V Det | cat | | 12. Shift cat | NP V Det cat | ∅ | | 13. Reduce N→cat | NP V Det N | ∅ | | 14. Reduce NP→Det N | NP V NP | ∅ | | 15. Reduce VP→V NP | NP VP | ∅ | | 16. Reduce S→NP VP | S | ∅ | ##### Step 2. Check acceptance Input is empty and stack is exactly S; therefore accept. ##### Step 3. Draw the resulting structure Parse tree: the big dog sees the catSNPDettheAdjbigNdogVPVseesNPDettheNcat Parse tree: the big dog sees the cat **Final answer.** Accepted. The parser builds both noun phrases, then VP, then S. **Question. Use S→NP VP, NP→Det N, VP→V NP, Det→the|this, N→man|book and V→read to give a shift-reduce trace for “the man read this book”.** #### Given The input has five tokens. Stack is empty initially. The lexical verb read is supplied by V→read. #### To find A complete bottom-up trace. #### Formula and why it applies Reduce only when the top stack sequence matches a supplied production's right side. Both noun phrases follow Det N, and the verb phrase follows V NP. #### Solution ##### Step 1. Trace all operations | Step / action | Stack after action | Remaining input | | --- | --- | --- | | 0. Start | ∅ | the man read this book | | 1. Shift the | the | man read this book | | 2. Reduce Det→the | Det | man read this book | | 3. Shift man | Det man | read this book | | 4. Reduce N→man | Det N | read this book | | 5. Reduce NP→Det N | NP | read this book | | 6. Shift read | NP read | this book | | 7. Reduce V→read | NP V | this book | | 8. Shift this | NP V this | book | | 9. Reduce Det→this | NP V Det | book | | 10. Shift book | NP V Det book | ∅ | | 11. Reduce N→book | NP V Det N | ∅ | | 12. Reduce NP→Det N | NP V NP | ∅ | | 13. Reduce VP→V NP | NP VP | ∅ | | 14. Reduce S→NP VP | S | ∅ | ##### Step 2. Verify the final state The complete sentence has reduced to S with no input left. **Final answer.** Accepted. Subject: the man. Predicate: read this book. Object: this book. **Question. Parse “book the flight” bottom-up using S→NP VP|VP, NP→Det N, VP→V NP, Det→the, N→book|flight, V→book. Explain why choosing the noun reading first can fail and show the successful alternative.** #### Given book has two lexical categories N and V. The input is an imperative. Stack starts empty. #### To find The failed choice and a complete successful trace with backtracking. #### Formula and why it applies Try lexical reductions allowed by the grammar; on a dead end, return to the ambiguous choice and try another. Grammar ambiguity requires a search choice. The noun reading cannot supply the required imperative verb. #### Solution ##### Step 1. Show the noun-first dead end Shift book and reduce it to N. Continue shifting/reducing the flight to Det N. Reducing Det N→NP leaves stack N NP and empty input. No supplied rule reduces N NP into S, so this branch fails. ##### Step 2. Return to the lexical choice Restore the state just after shifting book, then choose V→book instead of N→book. ##### Step 3. Trace the successful branch | Step / action | Stack after action | Remaining input | | --- | --- | --- | | 0. Restored after shifting book | book | the flight | | 1. Reduce V→book | V | the flight | | 2. Shift the | V the | flight | | 3. Reduce Det→the | V Det | flight | | 4. Shift flight | V Det flight | ∅ | | 5. Reduce N→flight | V Det N | ∅ | | 6. Reduce NP→Det N | V NP | ∅ | | 7. Reduce VP→V NP | VP | ∅ | | 8. Reduce S→VP | S | ∅ | ##### Step 4. Accept the alternative The verb reading produces S and consumes all tokens. **Final answer.** The noun-first branch fails; backtracking to book/V yields the successful imperative parse. **Yaad rakho:** Wrong branch fail hua toh sentence wrong nahi; next legal rule try karo. ### M2 13 — Shift-reduce parsing and backtracking id : nlp-m2-13-bottom-up-shift-reduce title : M2 13 — Shift-reduce parsing and backtracking title\_confirmed : false topic\_title : Shift-reduce parsing and backtracking mode : m2 kind : focused status : queued duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## M2 · Entity spans, categories and context Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/read/m2-ner--ner-definition/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/read/m2-ner/ner-definition Unit 4, pages 117–128 **Named Entity Recognition (NER)** identifies text spans referring to entities and assigns their types, such as PERSON, ORGANIZATION and LOCATION. Many practical schemas also include DATE, TIME, MONEY or domain-specific types. The exact allowed labels depend on the application's annotation scheme. NER has two parts: **boundary detection** finds where the entity starts and ends; **classification** assigns the appropriate type. A multiword name should normally be one entity span: “New York” is a location span, not two unrelated location decisions. Text: Mary joined Google in New York on Monday. → Mary → PERSON → Google → ORGANIZATION → New York → LOCATION → Monday → DATE Not every noun is a named entity. In “a developer works at Google”, developer is an ordinary job description while Google is a named organization. A specialized HR system may define a JOB\_TITLE category, but it must state that schema explicitly. NER identifies mentions; entity linking is the separate task of connecting a mention to a unique database identity. | Task | Question it answers | Example | | --- | --- | --- | | POS | What grammatical role does this token have? | Google/NNP | | Chunking | Which adjacent words form a phrase? | [NP a skilled developer] | | NER | Which span names an entity, and of what type? | [Google] ORGANIZATION | | Entity linking | Which real entity does this mention denote? | Connect Paris to the intended city or person record | **Applications.** NER structures search results and documents, extracts organizations/people from news, routes support tickets by product or organization, supports recommendation, and extracts specialized entities from healthcare or legal text. Context is essential: “Apple released a laptop” names an organization, while “she ate an apple” describes fruit. **Yaad rakho:** NER mein do kaam: naam kahan se kahan tak, aur kis category ka. ### M2 14 — Named entities, boundaries and recognition approaches id : nlp-m2-14-named-entity-recognition title : M2 14 — Named entities, boundaries and recognition approaches title\_confirmed : false topic\_title : Named entities, boundaries and recognition approaches mode : m2 kind : focused status : queued duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## M2 · Dictionary, rules, machine learning and deep learning Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/read/m2-ner--ner-approaches/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/read/m2-ner/ner-approaches Unit 4, pages 117–128 | Approach | How it works | Example | Main limitation | | --- | --- | --- | --- | | Dictionary / gazetteer | Match text against lists of known names | A city list includes Mumbai and Pune | New names, spelling variants and ambiguous names are missed | | Rule-based | Use word shapes, regular expressions and context | A title such as Dr. followed by a name-like span suggests PERSON | Manual maintenance; exceptions and domain/language dependence | | Machine learning | Learn a labeling rule from annotated examples using features | Capitalization, suffixes, POS and surrounding words help a classifier/CRF | Requires labeled data and useful features | | Deep learning | Learn contextual word/character representations and predict labels | A contextual sequence encoder labels multiword entity spans | Training/domain dependence and resource demands | A **pattern-based rule** examines the form of text, such as an email pattern or a date format. A **context-based rule** examines neighboring words: “Dr.” can support a following PERSON span, while “headquartered in” can support a following LOCATION span. Capitalization alone is insufficient because sentence-initial common nouns are also capitalized. A dictionary may find “May” as a person's name, but “in May” may refer to a month. A learned model uses surrounding evidence to disambiguate. Hybrid systems can combine dictionaries, rules and learned models; these categories are practical tools rather than mutually exclusive restrictions. **Preprocessing for NER.** Tokenize while preserving token offsets and useful capitalization. Blindly removing capital letters or stopwords before recognition can damage names such as “Bank of India”. NER output should preserve the original span so the reader can locate it in the input. **Exam-ready 3-mark answer.** NER locates named-entity spans and classifies them into categories such as person, organization and location. A dictionary-based approach matches known name lists; a rule-based approach uses patterns or context; machine-learning and deep-learning approaches learn entity decisions from labeled data and representations. For example, in “Mary works at Google in Mumbai”, Mary is PERSON, Google is ORGANIZATION and Mumbai is LOCATION. ### M2 14 — Named entities, boundaries and recognition approaches id : nlp-m2-14-named-entity-recognition title : M2 14 — Named entities, boundaries and recognition approaches title\_confirmed : false topic\_title : Named entities, boundaries and recognition approaches mode : m2 kind : focused status : queued duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## M2 · Three complete entity-recognition examples Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/read/m2-ner--ner-worked-examples/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/read/m2-ner/ner-worked-examples Unit 4, pages 117–128 **Question. Identify PERSON, ORGANIZATION and LOCATION entities in “Asha Rao joined Infosys in Bengaluru.”** #### Given Sentence: Asha Rao joined Infosys in Bengaluru. Allowed entity types: PERSON, ORGANIZATION, LOCATION. #### To find Complete entity spans and labels. #### Formula and why it applies Find the span first, then assign its type from context. A multiword personal name is a single entity; the verb and preposition remain outside entities. #### Solution ##### Step 1. Identify the person span Asha Rao names the person who joined, so take both words together. ##### Step 2. Identify the organization Infosys is the organization joined. ##### Step 3. Identify the location Bengaluru names the city. ##### Step 4. Write the output | Span | Label | | --- | --- | | Asha Rao | PERSON | | Infosys | ORGANIZATION | | Bengaluru | LOCATION | **Final answer.** [Asha Rao]PERSON joined [Infosys]ORGANIZATION in [Bengaluru]LOCATION. **Question. Using PERSON, ORGANIZATION, LOCATION and DATE, identify entities in “Dr. Meera Shah visited New Delhi on 5 October 2026.” Treat professional titles as outside the PERSON span for this exercise.** #### Given Allowed types and title-boundary convention are specified. Tokens include a multiword name, city and date. #### To find Entity boundaries and types. #### Formula and why it applies Boundary rules come from the annotation schema; use the stated title convention. Dr. supplies context for a person but is excluded from that span here. #### Solution ##### Step 1. Locate the person Meera Shah is the PERSON span; Dr. is outside by instruction. ##### Step 2. Locate the location New Delhi is one LOCATION span. ##### Step 3. Locate the date 5 October 2026 is one DATE span. ##### Step 4. Check organization mentions No organization is named in this sentence. **Final answer.** Meera Shah→PERSON; New Delhi→LOCATION; 5 October 2026→DATE. **Question. Explain the entity label of Apple in “Apple opened an office in Mumbai” and apple in “Ravi ate an apple”. Use PERSON, ORGANIZATION and LOCATION only.** #### Given Two contexts contain the same surface word apart from capitalization. The schema contains no FOOD category. #### To find Contextual labels and the ordinary noun case. #### Formula and why it applies NER labels a mention in context, not every occurrence of a dictionary string identically. A company can open an office; the object eaten in the second sentence is a fruit. #### Solution ##### Step 1. Analyze the first sentence Apple refers to the company, so label it ORGANIZATION. Mumbai is LOCATION. ##### Step 2. Analyze the second sentence Ravi is PERSON. apple is an ordinary fruit noun and receives no entity label under this schema. ##### Step 3. State the dictionary limitation Blindly labeling every apple as an organization would produce a false positive. **Final answer.** First: Apple→ORGANIZATION, Mumbai→LOCATION. Second: Ravi→PERSON; apple is outside the entity types. ### M2 14 — Named entities, boundaries and recognition approaches id : nlp-m2-14-named-entity-recognition title : M2 14 — Named entities, boundaries and recognition approaches title\_confirmed : false topic\_title : Named entities, boundaries and recognition approaches mode : m2 kind : focused status : queued duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## M2 · Lexeme, lemma, sense and context Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/read/u5-meaning--u5-words/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/read/u5-meaning/u5-words Unit 5 · PDF pages 1–22 **Semantics is the study of meaning in language.** Lexical semantics studies meanings of words and relationships between their meanings. Sentence semantics studies how word meanings and sentence structure combine. A **word form** is what appears in the sentence. A **lexeme** is the underlying vocabulary item whose grammatical forms belong together. A **lemma** is the dictionary form used to name it. A **sense** is one particular meaning. | Term | Example | What to understand | | --- | --- | --- | | Word forms | run, runs, ran, running | Different grammatical forms | | Lexeme | RUN | The shared underlying vocabulary item | | Lemma | run | Dictionary heading | | Sense | run a race / run a business | Different meanings need context | | Lexicon | A language's vocabulary resource | Stores lexical items and information about them | The four forms *run, ran, runs, running* belong to the same lexeme. They are not four unrelated lexemes. “I deposited cash in the bank” and “We sat by the bank of the river” contain the same written form but different senses. word: bank → context: cash, deposit → sense: financial institution **Syntax** asks how words are arranged. **Semantics** asks what the expression means. **Pragmatics** uses situation and speaker intention: “Can you open the window?” is grammatically a question about ability but is normally a request. Form = jo likha; lemma = dictionary naam; sense = iss sentence mein matlab. ### M2 15 — Lexical meaning, semantic relations and WordNet id : nlp-m2-15-lexical-meaning-wordnet title : M2 15 — Lexical meaning, semantic relations and WordNet title\_confirmed : false topic\_title : Lexical meaning, semantic relations and WordNet mode : m2 kind : focused status : queued duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## M2 · Every lexical relation with examples Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/read/u5-meaning--u5-relations/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/read/u5-meaning/u5-relations Unit 5 · PDF pages 1–22 **A lexical relation connects words or senses by a meaning relationship.** State the relation and its direction before giving the example. | Relation | Exam definition | Examples and direction | | --- | --- | --- | | Homonymy | The same form has unrelated meanings. | bank: financial institution / river edge; bat: animal / sports implement | | Polysemy | One lexical item has multiple related senses. | head: body part / head of a department; paper: material / article | | Synonymy | Different words have identical or closely similar meanings in a specified sense. | sofa / couch; automobile / car | | Antonymy | Words have opposite meanings. | hot / cold; alive / dead; buy / sell | | Hyponymy | A more specific concept is a type of a more general one. | rose is a flower; car is a vehicle | | Hypernymy | A broader concept includes a more specific one. | flower is a hypernym of rose | | Meronymy | A concept is a part of another concept. | wheel is part of car; finger is part of hand | | Holonymy | A concept is the whole containing a part. | car is a holonym of wheel | ### How to distinguish the commonly confused pairs For homonymy versus polysemy, ask whether the senses share an identifiable meaning. A river bank and a financial bank are unrelated. A financial bank and a blood bank share a repository/storage idea, illustrating polysemy in the supplied notes. For hyponymy versus meronymy, use the sentence test: **“X is a kind of Y”** indicates hyponymy; **“X is part of Y”** indicates meronymy. A wheel is not a type of car. | Expression | Correct relation | Reason | | --- | --- | --- | | sparrow → bird | Hyponymy | A sparrow is a kind of bird | | bird → sparrow | Hypernymy | Bird is the broader category | | keyboard → computer | Meronymy | Keyboard is a component | | computer → keyboard | Holonymy | Computer is the containing whole | | mouth of a person / mouth of a river | Polysemy | Both senses share the idea of an opening; the river use is a related extension | Synonyms are not always interchangeable in every context or register. “Big mistake” and “large mistake” can sound different even though big and large overlap in meaning. Antonyms can be gradable (hot/cold), complementary (alive/dead) or relational converses (buy/sell). HYPO = specific type; HYPER = bigger category. MERO = part; HOLO = whole. ### M2 15 — Lexical meaning, semantic relations and WordNet id : nlp-m2-15-lexical-meaning-wordnet title : M2 15 — Lexical meaning, semantic relations and WordNet title\_confirmed : false topic\_title : Lexical meaning, semantic relations and WordNet mode : m2 kind : focused status : queued duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## M2 · WordNet: synsets, glosses and hierarchy Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/read/u5-meaning--u5-wordnet/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/read/u5-meaning/u5-wordnet Unit 5 · PDF pages 1–22 **WordNet is a lexical database that groups word senses into synonym sets called synsets and links them through semantic and lexical relations.** It is a network of meanings, not merely a list of similar spellings. A synset represents one concept. Its **lemmas** are the words that can express that concept. A **gloss** is a short definition, often accompanied by an example. An ambiguous word can occur in several synsets. vehicle (general) → car / automobile (same concept) → sports car (specific) This is a simplified teaching hierarchy, not a claim about every intermediate WordNet node. Going upward reaches hypernyms; going downward reaches hyponyms. Multiple parents can exist, so the structure need not be a single simple tree. | Operation | Meaning | Illustration | | --- | --- | --- | | Get senses of a word | Find candidate synsets | bank → financial sense, river-edge sense, … | | Get lemmas | Find synonyms within one synset | car, automobile, motorcar | | Get definition/examples | Read the gloss and uses | Choose the intended concept | | Get hypernyms | Move to broader categories | car → vehicle | | Get hyponyms | Move to narrower categories | vehicle → particular vehicle categories | | Get meronyms/holonyms | Find part/whole links | car ↔ wheel | ### WordNet versus a thesaurus A thesaurus mainly groups related expressions for vocabulary lookup. WordNet explicitly distinguishes senses and labels relations such as hypernymy, hyponymy and meronymy. This supports WSD, query expansion, lexical similarity and semantic search. ### Three uses explained 1. **Search expansion:** a query for “automobile” can retrieve relevant documents using “car”. Expansion should use the right sense. 2. **WSD:** definitions of the senses of “bank” provide words that can be compared with the sentence context. 3. **Similarity:** concepts near one another in an “is-a” hierarchy tend to be semantically related. Path-based measures depend on which senses were selected. WordNet mein spelling ka group nahi—meaning ka synset, aur meanings ke beech labelled links. ### M2 15 — Lexical meaning, semantic relations and WordNet id : nlp-m2-15-lexical-meaning-wordnet title : M2 15 — Lexical meaning, semantic relations and WordNet title\_confirmed : false topic\_title : Lexical meaning, semantic relations and WordNet mode : m2 kind : focused status : queued duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## M2 · What WSD solves and how approaches differ Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/read/u5-wsd--u5-wsd-basics/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/read/u5-wsd/u5-wsd-basics Unit 5 · PDF pages 23–48 **Word sense disambiguation (WSD) is the task of selecting the intended sense of an ambiguous word in its context from a set of possible senses.** The output is a sense label, not a different replacement word. Input: “He grilled the bass.” → Candidate senses: fish / low sound → Context: grilled → Output: fish sense The **target word** is the ambiguous word. The **context** is the surrounding evidence. The **sense inventory** lists candidate meanings, often using WordNet. A **feature** is a usable clue: nearby word, position, part-of-speech tag or a phrase. | Approach | What it needs | How it decides | Limitation | | --- | --- | --- | --- | | Knowledge based | Dictionary, thesaurus or semantic graph | Definition overlap, category evidence or graph scores | Short definitions and missing knowledge | | Supervised | Examples already labelled with senses | Learns context → sense mapping | Labelling cost, limited rare-sense examples | | Semi-supervised | Small labelled set plus unlabelled data | Bootstraps from reliable seed examples | Early errors can spread | | Unsupervised sense induction | Unlabelled contextual occurrences | Clusters similar contexts | Clusters are not automatically named dictionary senses | | Neural/contextual | Learned word/context representations | Predicts or compares a context representation | Needs suitable data/model and evaluation | **Bag-of-words features** record nearby words while ignoring exact positions. **Collocational features** preserve a positional phrase such as “river bank”, “bank account”, “chocolate bar” or “bar exam”. ### Why WSD matters Machine translation needs the correct target-language meaning; information retrieval should distinguish river-bank travel from banking services; text-to-speech may need different pronunciations for “bass”; information extraction should avoid connecting entities through the wrong relation. WSD: same spelling ke options mein context dekh ke sahi meaning select. ### M2 16 WSD and Lesk Scoring id : nlp-m2-16-lesk-wsd title : M2 16 WSD and Lesk Scoring title\_confirmed : true topic\_title : Word-sense disambiguation and Lesk overlap scoring mode : m2 kind : focused status : available duration : 10:17 reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — Word-sense disambiguation | Reviewed Lesk and Bayes numericals id : overview-wsd title : NLP M2 — Word-sense disambiguation | Reviewed Lesk and Bayes numericals title\_confirmed : true topic\_title : WSD: Detective Math Toolkit mode : m2 kind : supplemental status : available duration : 6:11 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Lesk variants, conditional Bayes definitions, smoothing and both worked posteriors; corrected spoken claims. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids : - nlp-m2-16-lesk-wsd - nlp-m2-17-walker-graph-wsd - nlp-m2-18-naive-bayes-wsd These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## M2 · Lesk algorithm, from words to overlap scores Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/read/u5-wsd--u5-lesk/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/read/u5-wsd/u5-lesk Unit 5 · PDF pages 23–48 **Lesk is a knowledge-based WSD method that chooses senses whose dictionary descriptions have the greatest overlap with contextual evidence.** Original Lesk compares definitions of neighbouring words' senses. A common simplified version compares each target-sense gloss directly with the sentence context. State which version you use. ### Simplified Lesk procedure 1. Remove the target word from the context; tokenize and normalize the remaining content words. 2. Obtain the gloss for each candidate sense and apply the same normalization. 3. Count shared words between the context and each gloss. 4. Select the sense with maximum overlap. If scores tie, report the tie or use a specified tie-break rule. $$ \operatorname{score}(s)=|C\cap G_s|,\qquad s^*=\arg\max_s\operatorname{score}(s) $$ $$ C $$ is the set of context words, $$ G_s $$ the set of words in sense $$ s $$ 's gloss, $$ \cap $$ means common elements, $$ |\cdot| $$ means count, and $$ \arg\max $$ means “the sense giving the largest score”. This version counts unique matching words once. **Question. Use simplified Lesk to disambiguate “bank” in “The river water rose near the bank.” Use C = {river, water, rose, near}; financial gloss = {institution, money, deposit}; river gloss = {land, river, water, edge}.** #### Given The normalized context and two gloss sets are given. Count exact shared words. #### To find The score of each sense and the selected meaning. #### Formula and why it applies Use $$ \operatorname{score}(s)=|C\cap G_s| $$ because the task specifies context–gloss overlap. #### Step 1. Compare the financial sense No word occurs in both {river, water, rose, near} and {institution, money, deposit}. Overlap = {}. #### Step 2. Count the financial score $$ \operatorname{score}(F)=0 $$ . #### Step 3. Compare the river sense Shared words are {river, water}. “Rose” and “near” are not in the gloss. #### Step 4. Count and select $$ \operatorname{score}(R)=2 $$ . Since $$ 2>0 $$ , choose the river-edge sense. **Answer.** Bank means land beside a river. **Question. Use simplified Lesk for “The dog barked loudly.” Treat barked as bark. Context C = {dog, loudly}; sound gloss = {sharp, cry, dog}; tree gloss = {outer, covering, tree}; ship gloss = {sailing, ship}.** #### Given Three glosses and normalized context {dog, loudly}. #### To find Select the sense of bark. #### Formula and why it applies $$ \operatorname{score}(s)=|C\cap G_s| $$ . Lemmatization lets barked match the target lemma bark; it does not create additional overlap words. #### Step 1. Sound sense Intersection = {dog}; score = 1. #### Step 2. Tree sense Intersection = {}; score = 0. #### Step 3. Ship sense Intersection = {}; score = 0. #### Step 4. Compare The sound sense has the unique maximum: 1 > 0. **Answer.** Bark is the dog's cry. **Question. Apply definition–definition Lesk to “pine cone”. Pine senses: P1 = {evergreen, tree, needle, leaf}; P2 = {waste, sorrow, illness}. Cone senses: C1 = {solid, shape, circular}; C2 = {fruit, evergreen, tree}.** #### Given Two candidate senses for each word. Tokens are already normalized; tree and trees would be reduced to the same form if necessary. #### To find The most compatible pair of senses. #### Formula and why it applies Use $$ |G_{P_i}\cap G_{C_j}| $$ for every pair because this is original-style gloss–gloss comparison. #### Step 1. P1 with C1 No shared word → 0. #### Step 2. P1 with C2 Shared {evergreen, tree} → 2. #### Step 3. P2 with each cone sense P2–C1 has no overlap → 0; P2–C2 has no overlap → 0. #### Step 4. Choose the best pair The four scores are 0, 2, 0, 0. Maximum 2 belongs to P1–C2. **Answer.** Pine is the evergreen tree; cone is its fruit-like reproductive structure. **Limitations:** exact overlap misses synonyms, short glosses may produce zeros, and different preprocessing choices change the score. Extended Lesk may include related synset definitions or longer phrase overlaps; do not silently substitute it for a simple question. Lesk: context aur dictionary mein common words gino; sabse bada overlap jeeta. ### M2 16 WSD and Lesk Scoring id : nlp-m2-16-lesk-wsd title : M2 16 WSD and Lesk Scoring title\_confirmed : true topic\_title : Word-sense disambiguation and Lesk overlap scoring mode : m2 kind : focused status : available duration : 10:17 reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — Word-sense disambiguation | Reviewed Lesk and Bayes numericals id : overview-wsd title : NLP M2 — Word-sense disambiguation | Reviewed Lesk and Bayes numericals title\_confirmed : true topic\_title : WSD: Detective Math Toolkit mode : m2 kind : supplemental status : available duration : 6:11 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Lesk variants, conditional Bayes definitions, smoothing and both worked posteriors; corrected spoken claims. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids : - nlp-m2-16-lesk-wsd - nlp-m2-17-walker-graph-wsd - nlp-m2-18-naive-bayes-wsd These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## M2 · Thesaurus / Walker approach, with three complete examples Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/read/u5-wsd--u5-walker/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/read/u5-wsd/u5-walker Unit 5 · PDF pages 23–48 **The Walker approach uses thesaurus categories as semantic evidence.** Associate each candidate sense with a category. Count or weight context words that support that category. Choose the largest total. $$ \operatorname{score}(s)=\sum_{w\in C} I(w,\operatorname{category}(s)) $$ $$ I=1 $$ if the supplied category mapping says the context word supports the sense, otherwise $$ I=0 $$ . The summation symbol means add all those contributions. Weighted questions may replace 1 with a supplied weight. Category membership must be supplied or justified; it is not guessed from word frequency. **Question. Disambiguate bank using Walker scoring. Context words are money, interest, annum, fetch. The finance category supports money, interest and annum; the location category supports none of these words.** #### Given Two categories: Finance and Location. Each supported word contributes 1. #### To find The two totals and selected bank sense. #### Formula and why it applies Sum the category indicators for each sense. #### Step 1. Make a contribution table | Word | Finance | Location | | --- | --- | --- | | money | 1 | 0 | | interest | 1 | 0 | | annum | 1 | 0 | | fetch | 0 | 0 | #### Step 2. Add Finance $$ 1+1+1+0=3 $$ . #### Step 3. Add Location $$ 0+0+0+0=0 $$ . #### Step 4. Select 3 > 0, so select the financial institution. **Answer.** Finance, with score 3 versus 0. **Question. Use Walker scoring for “The fisherman caught bass beside the lake.” Context {fisherman,caught,lake}. The Fish category contains all three clues; the Music category contains none.** #### Given Fish and Music senses, binary category membership given. #### To find The correct bass sense. #### Formula and why it applies Score = sum of context-category matches. #### Step 1. Record Fish matches fisherman → 1; caught → 1; lake → 1. #### Step 2. Add Fish score $$ 1+1+1=3 $$ . #### Step 3. Add Music score $$ 0+0+0=0 $$ . #### Step 4. Choose Fish has the larger score. **Answer.** Bass refers to a fish. **Question. Score “The player used a bat in the match.” Context {player,match}. Sport supports both words with weights 2 and 1; Animal supports neither. Use the supplied weighted Walker scheme.** #### Given Sport weights {player:2, match:1}; Animal weights {player:0, match:0}. #### To find Weighted sense scores. #### Formula and why it applies Use the sum of supplied weights, not an unweighted count, because the question explicitly gives weights. #### Step 1. Sport contribution of player Add 2. #### Step 2. Sport contribution of match Add 1. Sport total = 2 + 1 = 3. #### Step 3. Animal total 0 + 0 = 0. #### Step 4. Compare 3 > 0 selects the sporting implement. **Answer.** Bat is the sports implement. Walker: dictionary ke exact words nahi, context kis topic/category ko support karta hai woh count. ### M2 17 — Walker category scoring and graph-based WSD id : nlp-m2-17-walker-graph-wsd title : M2 17 — Walker category scoring and graph-based WSD title\_confirmed : false topic\_title : Walker category scoring and graph-based WSD mode : m2 kind : focused status : queued duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — Word-sense disambiguation | Reviewed Lesk and Bayes numericals id : overview-wsd title : NLP M2 — Word-sense disambiguation | Reviewed Lesk and Bayes numericals title\_confirmed : true topic\_title : WSD: Detective Math Toolkit mode : m2 kind : supplemental status : available duration : 6:11 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Lesk variants, conditional Bayes definitions, smoothing and both worked posteriors; corrected spoken claims. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids : - nlp-m2-16-lesk-wsd - nlp-m2-17-walker-graph-wsd - nlp-m2-18-naive-bayes-wsd These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## M2 · Random-walk WSD and supervised alternatives Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/read/u5-wsd--u5-graph/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/read/u5-wsd/u5-graph Unit 5 · PDF pages 23–48 **Graph-based WSD represents candidate senses as vertices and semantic relationships as weighted edges.** A random walk repeatedly transfers score along these edges. Senses supported by other strongly connected context senses tend to obtain higher ranks. 1. Create a vertex for every candidate sense of every relevant word. 2. Connect semantically related senses; similarity or gloss overlap can set edge weights. 3. Normalize outgoing edge weights into transition probabilities. 4. Run a graph-ranking method such as PageRank. 5. For each target word, choose its highest-ranked candidate sense. church context → bell: instrument sense ↔ ring: sound sense → mutually supported senses receive rank One common rank update is $$ r_{t+1}(j)=\frac{1-d}{N}+d\sum_i r_t(i)P(i\rightarrow j). $$ $$ r_t(i) $$ is current score of vertex $$ i $$ , $$ P(i\to j) $$ is its outgoing probability to $$ j $$ , $$ N $$ is the number of vertices, and $$ d $$ is the probability of following a graph edge. The remaining probability $$ 1-d $$ gives a uniform restart here. A personalized variant uses a nonuniform restart. The course requires the graph method concept; numerical details depend on supplied ranking rules. **Question. A two-vertex toy sense graph has transition matrix rows A:[0.2,0.8], B:[0.4,0.6]. Starting scores are A=0.5, B=0.5. Find one random-walk update with no restart.** #### Given Rows denote source vertices, columns destination vertices. d = 1. #### To find New scores at A and B. #### Formula and why it applies Use $$ r_{t+1}(j)=\sum_i r_t(i)P(i\to j) $$ : incoming contributions are added. #### Step 1. Contribution to A from A $$ 0.5\times0.2=0.1 $$ . #### Step 2. Contribution to A from B $$ 0.5\times0.4=0.2 $$ , hence $$ r_1(A)=0.1+0.2=0.3 $$ . #### Step 3. Contribution to B from A and B $$ 0.5\times0.8=0.4 $$ ; $$ 0.5\times0.6=0.3 $$ . #### Step 4. Add B and verify total $$ r_1(B)=0.4+0.3=0.7 $$ ; $$ 0.3+0.7=1 $$ . **Answer.** A=0.3, B=0.7 after one update. This is one iteration, not a converged PageRank result. ### Other approaches explained without assuming ML knowledge **A supervised classifier** learns from examples such as “cash … bank” labelled FINANCE and “river … bank” labelled RIVER. A decision list applies ranked context rules; a decision tree branches on clues; an SVM separates feature representations; a neural model learns contextual patterns. Naive Bayes, developed next, multiplies class-specific context probabilities. **Semi-supervised bootstrapping** begins with a few labelled examples or high-confidence rules, labels suitable unlabelled cases, and retrains. **Unsupervised clustering** groups usages by context similarity without gold sense labels; an additional mapping is needed to name the discovered groups. Graph WSD: meanings ko jodo; jo meaning context ke strong meanings se supported hai uska rank badhega. ### M2 17 — Walker category scoring and graph-based WSD id : nlp-m2-17-walker-graph-wsd title : M2 17 — Walker category scoring and graph-based WSD title\_confirmed : false topic\_title : Walker category scoring and graph-based WSD mode : m2 kind : focused status : queued duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — Word-sense disambiguation | Reviewed Lesk and Bayes numericals id : overview-wsd title : NLP M2 — Word-sense disambiguation | Reviewed Lesk and Bayes numericals title\_confirmed : true topic\_title : WSD: Detective Math Toolkit mode : m2 kind : supplemental status : available duration : 6:11 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Lesk variants, conditional Bayes definitions, smoothing and both worked posteriors; corrected spoken claims. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids : - nlp-m2-16-lesk-wsd - nlp-m2-17-walker-graph-wsd - nlp-m2-18-naive-bayes-wsd These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## M2 · Prior, likelihood and posterior—before the formula Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/read/u5-naive-bayes--u5-bayes-meaning/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/read/u5-naive-bayes/u5-bayes-meaning Unit 5 · PDF pages 49–57 **A probability is a number from 0 to 1 measuring how likely an event is under a stated model.** A class probability of $$ 3/6=0.5 $$ means 3 of 6 training examples belong to that class. **Conditional probability** means probability inside a restricted group. $$ P(\text{loan}\mid F) $$ reads “probability of the word loan given the finance sense”. The vertical bar means “given”; it is not division. | Symbol | Name | Plain meaning | | --- | --- | --- | | $$ P(F) $$ | Prior | How common finance is before reading the new context | | $$ P(C\mid F) $$ | Likelihood | How compatible the observed context is with finance | | $$ P(F\mid C) $$ | Posterior | How likely finance is after seeing the context | | $$ P(C) $$ | Evidence | Total probability of this context under all senses | $$ P(F\mid C)=\frac{P(C\mid F)P(F)}{P(C)}. $$ Bayes' theorem reverses a conditional direction using the prior. Do not swap $$ P(F\mid C) $$ and $$ P(C\mid F) $$ : one asks about the sense, the other about the evidence. The **naive conditional independence assumption** says that, once the sense is fixed, the context-word features can be multiplied as if they were independent. It is an approximation: natural-language words are not actually independent in general. Training examples → count classes → Prior per sense → Count words within each sense → Likelihood of query clues → Multiply → compare → normalize if asked Prior = pehle kitna common; likelihood = clue kitna fit; posterior = clue dekhne ke baad meaning ki chance. ### Naive Bayes WSD Numerical id : nlp-m2-18-naive-bayes-wsd title : Naive Bayes WSD Numerical title\_confirmed : true topic\_title : Naive Bayes WSD: a complete corpus numerical mode : m2 kind : focused status : available duration : 11:35 reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — Word-sense disambiguation | Reviewed Lesk and Bayes numericals id : overview-wsd title : NLP M2 — Word-sense disambiguation | Reviewed Lesk and Bayes numericals title\_confirmed : true topic\_title : WSD: Detective Math Toolkit mode : m2 kind : supplemental status : available duration : 6:11 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Lesk variants, conditional Bayes definitions, smoothing and both worked posteriors; corrected spoken claims. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids : - nlp-m2-16-lesk-wsd - nlp-m2-17-walker-graph-wsd - nlp-m2-18-naive-bayes-wsd These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## M2 · Multinomial Naive Bayes: the exact solving method Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/read/u5-naive-bayes--u5-bayes-steps/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/read/u5-naive-bayes/u5-bayes-steps Unit 5 · PDF pages 49–57 Naive Bayes selects the sense with the largest prior × likelihood product. The context evidence denominator is identical across candidate senses, so it can be omitted for comparison. $$ \operatorname{score}(s)=P(s)\prod_j P(w_j\mid s). $$ For add-one multinomial smoothing: $$ P(w\mid s)=\frac{C(w,s)+1}{N_s+V}. $$ $$ C(w,s) $$ counts occurrences of word $$ w $$ in training contexts of sense $$ s $$ ; $$ N_s $$ is the total number of word occurrences in those contexts; $$ V $$ is the shared vocabulary size. It is not the number of training sentences. Repeated query words contribute repeated factors. ### Always keep these counts separate Class priors use **number of examples**. Word likelihoods in this multinomial model use **number of word occurrences**. The vocabulary counts **unique words across all classes**. 1. List the class examples and calculate priors. 2. Write one shared vocabulary and count its size. 3. Count total word occurrences in each class. 4. Count each query word in each class. 5. Apply the given smoothing rule. 6. Multiply each prior by all its query likelihoods. 7. Compare scores; divide each score by their sum only if a normalized posterior is requested. $$ P(s\mid C)=\frac{\operatorname{score}(s)}{\sum_{s'}\operatorname{score}(s')}. $$ **Why smoothing is needed:** an unseen clue gives $$ 0/N_s=0 $$ ; one zero makes the whole product zero. Add-one smoothing redistributes probability across every vocabulary word and prevents these zero factors. Use it only when requested, taught for the task, or state the assumption. **Different model warning:** a Bernoulli Naive Bayes question counts presence/absence per document and may include absence factors. Do not use its denominator in a multinomial word-count question. Pehle sentences se prior; phir words se likelihood; dono counts mix mat kar. ### Naive Bayes WSD Numerical id : nlp-m2-18-naive-bayes-wsd title : Naive Bayes WSD Numerical title\_confirmed : true topic\_title : Naive Bayes WSD: a complete corpus numerical mode : m2 kind : focused status : available duration : 11:35 reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — Word-sense disambiguation | Reviewed Lesk and Bayes numericals id : overview-wsd title : NLP M2 — Word-sense disambiguation | Reviewed Lesk and Bayes numericals title\_confirmed : true topic\_title : WSD: Detective Math Toolkit mode : m2 kind : supplemental status : available duration : 6:11 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Lesk variants, conditional Bayes definitions, smoothing and both worked posteriors; corrected spoken claims. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids : - nlp-m2-16-lesk-wsd - nlp-m2-17-walker-graph-wsd - nlp-m2-18-naive-bayes-wsd These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## M2 · Mam's bank numerical—every count and every multiplication Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/read/u5-naive-bayes--u5-bayes-mam/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/read/u5-naive-bayes/u5-bayes-mam Unit 5 · PDF pages 49–57 **Question. Classify the sense of “bank” using the training table below. The sentence is: “As the stream flowed gently past the shore, we set up camp by the bank of the river, discussing the possibility of borrowing a loan to buy a boat for future trips.” Use only the highlighted feature words stream, shore and loan, and add-one multinomial smoothing.** #### Given | Training row | Context words | Sense | | --- | --- | --- | | 1 | money, deposit, loan | Finance | | 2 | river, water, shore | Riverbank | | 3 | account, savings, investment | Finance | | 4 | boat, river, stream | Riverbank | | 5 | branch, deposit, check | Finance | | 6 | fishing, shore, river | Riverbank | Query feature set: {stream, shore, loan}. Other sentence words are not included as features in this supplied example. #### To find The most probable bank sense and normalized posteriors. #### Formula and why it applies Naive Bayes selects the sense with the largest prior × likelihood product. The context evidence denominator is identical across candidate senses, so it can be omitted for comparison. $$ \operatorname{score}(s)=P(s)\prod_j P(w_j\mid s). $$ For add-one multinomial smoothing: $$ P(w\mid s)=\frac{C(w,s)+1}{N_s+V}. $$ $$ C(w,s) $$ counts occurrences of word $$ w $$ in training contexts of sense $$ s $$ ; $$ N_s $$ is the total number of word occurrences in those contexts; $$ V $$ is the shared vocabulary size. It is not the number of training sentences. Repeated query words contribute repeated factors. #### Step 1. Calculate priors Finance appears in 3 of the 6 rows; Riverbank appears in 3 of 6. $$ P(F)=\frac36=\frac12,\qquad P(R)=\frac36=\frac12. $$ #### Step 2. Construct the vocabulary {money, deposit, loan, river, water, shore, account, savings, investment, boat, stream, fishing, branch, check}. Thus V=14. Repeated deposit or river is counted once in vocabulary. #### Step 3. Count word occurrences per class There are 3 finance rows × 3 words = 9 word occurrences. There are 3 river rows × 3 words = 9. Thus $$ N_F=N_R=9 $$ , and each smoothed denominator is $$ 9+14=23 $$ . #### Step 4. Count the three query words | Word | Finance count | Riverbank count | | --- | --- | --- | | stream | 0 | 1 | | shore | 0 | 2 | | loan | 1 | 0 | #### Step 5. Calculate finance likelihoods $$ P(\text{stream}\mid F)=\frac{0+1}{23}=\frac1{23} $$ $$ P(\text{shore}\mid F)=\frac{0+1}{23}=\frac1{23} $$ $$ P(\text{loan}\mid F)=\frac{1+1}{23}=\frac2{23} $$ #### Step 6. Multiply the finance score $$ \operatorname{score}(F)=\frac12\times\frac1{23}\times\frac1{23}\times\frac2{23}=\frac1{23^3}=\frac1{12167}\approx0.00008219. $$ #### Step 7. Calculate river likelihoods $$ P(\text{stream}\mid R)=\frac{1+1}{23}=\frac2{23} $$ $$ P(\text{shore}\mid R)=\frac{2+1}{23}=\frac3{23} $$ $$ P(\text{loan}\mid R)=\frac{0+1}{23}=\frac1{23} $$ #### Step 8. Multiply the river score $$ \operatorname{score}(R)=\frac12\times\frac2{23}\times\frac3{23}\times\frac1{23}=\frac3{12167}\approx0.00024657. $$ #### Step 9. Compare scores 3/12167 is larger than 1/12167. Riverbank wins even though “loan” is financial evidence: the combined clues matter. #### Step 10. Normalize if a posterior is requested $$ P(F\mid C)=\frac{1/12167}{4/12167}=\frac14=0.25 $$ $$ P(R\mid C)=\frac{3/12167}{4/12167}=\frac34=0.75. $$ **Answer.** Bank is the riverbank sense. Scores are 1/12167 and 3/12167; normalized model posteriors are 0.25 and 0.75. **Without smoothing:** stream and shore are absent in Finance; loan is absent in Riverbank. Both unsmoothed scores become zero, so there is no valid winning class from those products. Also, unnormalized scores should not be labelled as final posterior probabilities. Zero word? Add-one: count+1 upar, total words+vocabulary neeche. Last mein score bada wala sense. ### Naive Bayes WSD Numerical id : nlp-m2-18-naive-bayes-wsd title : Naive Bayes WSD Numerical title\_confirmed : true topic\_title : Naive Bayes WSD: a complete corpus numerical mode : m2 kind : focused status : available duration : 11:35 reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — Word-sense disambiguation | Reviewed Lesk and Bayes numericals id : overview-wsd title : NLP M2 — Word-sense disambiguation | Reviewed Lesk and Bayes numericals title\_confirmed : true topic\_title : WSD: Detective Math Toolkit mode : m2 kind : supplemental status : available duration : 6:11 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Lesk variants, conditional Bayes definitions, smoothing and both worked posteriors; corrected spoken claims. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids : - nlp-m2-16-lesk-wsd - nlp-m2-17-walker-graph-wsd - nlp-m2-18-naive-bayes-wsd These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## M2 · More WSD numericals: given likelihoods, unequal priors and repeated clues Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/read/u5-naive-bayes--u5-bayes-practice/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/read/u5-naive-bayes/u5-bayes-practice Unit 5 · PDF pages 49–57 **Question. For bass, let P(Fish)=0.6 and P(Music)=0.4. Given Fish likelihoods P(caught|Fish)=0.5, P(lake|Fish)=0.4 and Music likelihoods P(caught|Music)=0.05, P(lake|Music)=0.1, classify context {caught,lake} and normalize.** #### Given Priors and likelihoods are provided directly; no smoothing or training-count estimation is requested. #### To find The winning sense and its posterior. #### Formula and why it applies $$ \operatorname{score}(s)=P(s)P(\text{caught}\mid s)P(\text{lake}\mid s) $$ , because the two features are treated as conditionally independent. #### Step 1. Fish multiplication $$ 0.6\times0.5=0.3 $$ , then $$ 0.3\times0.4=0.12 $$ . #### Step 2. Music multiplication $$ 0.4\times0.05=0.02 $$ , then $$ 0.02\times0.1=0.002 $$ . #### Step 3. Total evidence $$ 0.12+0.002=0.122 $$ . #### Step 4. Normalize $$ P(Fish\mid C)=0.12/0.122\approx0.983607 $$ $$ P(Music\mid C)=0.002/0.122\approx0.016393. $$ **Answer.** Fish, with model posterior approximately 0.983607. **Question. Disambiguate bank from the repeated context tokens {loan, loan, river}. Priors are Finance=2/3 and River=1/3. Vocabulary V=4. Finance has N=8 total tokens, loan count 3 and river count 0; River has N=4, loan count 0 and river count 2. Apply add-one smoothing.** #### Given Multinomial counts are supplied. The two appearances of loan must both contribute. #### To find Scores and normalized Finance posterior. #### Formula and why it applies Naive Bayes selects the sense with the largest prior × likelihood product. The context evidence denominator is identical across candidate senses, so it can be omitted for comparison. $$ \operatorname{score}(s)=P(s)\prod_j P(w_j\mid s). $$ For add-one multinomial smoothing: $$ P(w\mid s)=\frac{C(w,s)+1}{N_s+V}. $$ $$ C(w,s) $$ counts occurrences of word $$ w $$ in training contexts of sense $$ s $$ ; $$ N_s $$ is the total number of word occurrences in those contexts; $$ V $$ is the shared vocabulary size. It is not the number of training sentences. Repeated query words contribute repeated factors. #### Step 1. Finance denominator $$ 8+4=12 $$ . $$ P(loan\mid F)=(3+1)/12=1/3 $$ ; $$ P(river\mid F)=(0+1)/12=1/12 $$ . #### Step 2. River denominator $$ 4+4=8 $$ . $$ P(loan\mid R)=(0+1)/8=1/8 $$ ; $$ P(river\mid R)=(2+1)/8=3/8 $$ . #### Step 3. Finance product $$ \operatorname{score}(F)=\frac23\times\frac13\times\frac13\times\frac1{12}=\frac1{162}\approx0.00617284. $$ #### Step 4. River product $$ \operatorname{score}(R)=\frac13\times\frac18\times\frac18\times\frac38=\frac1{512}\approx0.001953125. $$ #### Step 5. Compare and normalize $$ P(F\mid C)=\frac{1/162}{1/162+1/512}=\frac{512}{674}=\frac{256}{337}\approx0.759644. $$ **Answer.** Finance wins. Repeated loan contributes a squared likelihood; do not reduce the query to a unique-word set. ### Answer-writing order in an exam Write the class priors, vocabulary size, class token totals, a likelihood table, both score products and the final class. Keep fractions until the comparison where possible. If the question only asks classification, normalization is optional; if it asks posterior probabilities, normalization is required. Repeated word aaye toh factor bhi repeat. Sirf word-set bana diya toh multinomial answer badal jayega. ### Naive Bayes WSD Numerical id : nlp-m2-18-naive-bayes-wsd title : Naive Bayes WSD Numerical title\_confirmed : true topic\_title : Naive Bayes WSD: a complete corpus numerical mode : m2 kind : focused status : available duration : 11:35 reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — Word-sense disambiguation | Reviewed Lesk and Bayes numericals id : overview-wsd title : NLP M2 — Word-sense disambiguation | Reviewed Lesk and Bayes numericals title\_confirmed : true topic\_title : WSD: Detective Math Toolkit mode : m2 kind : supplemental status : available duration : 6:11 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Lesk variants, conditional Bayes definitions, smoothing and both worked posteriors; corrected spoken claims. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids : - nlp-m2-16-lesk-wsd - nlp-m2-17-walker-graph-wsd - nlp-m2-18-naive-bayes-wsd These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## M2 · The M2 formula sheet Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/formulas/formula-index--all/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/formulas/formula-index/all M2 · formulas with meaning · full examples follow Read each formula with its meaning. The remaining lessons in this tab contain the complete worked examples for these methods. Use the same vocabulary, boundary, case and token-count conventions throughout one solution. ### N-gram count $$ \max(T-n+1,0) $$ Unpadded sequence of T tokens. Count only valid contiguous windows. ### Unigram MLE $$ P(w)=C(w)/N $$ N is the training-token total under the stated boundary convention. ### Bigram MLE $$ P(w_i\mid w_{i-1})=C(w_{i-1},w_i)/C(w_{i-1}) $$ Denominator = history count. ### Trigram MLE $$ P(w_i\mid w_{i-2},w_{i-1})=C(w_{i-2},w_{i-1},w_i)/C(w_{i-2},w_{i-1}) $$ Denominator = two-word history count. ### Add-one / Laplace $$ P(w\mid h)=\frac{C(hw)+1}{C(h)+|V|} $$ Add one to every possible vocabulary continuation. ### Sentence probability $$ P(W)\approx\prod_iP(w_i\mid w_{i-1}) $$ Multiply in order; include an end factor if specified. ### Perplexity $$ PP=P(W)^{-1/N} $$ N is the number of predicted tokens/events used by the question. ### Emission $$ P(w\mid t)=C(w,t)/C(t) $$ Word given tag; count tokens with that tag. ### Transition $$ P(t_j\mid t_i)=C(t_i\to t_j)/\sum_k C(t_i\to t_k) $$ Next tag given previous tag; count outgoing transitions. ### HMM joint path $$ P(W,T)=\pi_{t_1}b_{t_1}(w_1)\prod_{i=2}^{n}a_{t_{i-1},t_i}b_{t_i}(w_i) $$ Multiply initial, emission and transition factors; add end probability if used. ### Tag sequences $$ K^T $$ K tags, T word positions; count before constraints. ### Viterbi $$ \delta_i(t)=b_t(w_i)\max_s[\delta_{i-1}(s)a_{s,t}] $$ Choose the largest predecessor score; store a backpointer. ### Lesk $$ score(s)=|context\cap gloss(s)| $$ Use the specified normalisation and overlap convention. ### Naive Bayes class prior $$ P(c)=D_c/D $$ D counts labelled examples; this is not the word-token total. ### Smoothed NB likelihood $$ P(w\mid c)=\frac{C(w,c)+1}{N_c+|V|} $$ N\_c counts training tokens in class c. ### NB decision $$ \hat c=\arg\max_c P(c)\prod_iP(w_i\mid c) $$ Select label with largest score; normalise only for posterior values. **Yaad rakho:** Condition jis par hai, denominator usi ka; sequence mein multiply; options mein maximum. --- ## M2 · Unigrams, bigrams and trigrams: construct them correctly Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/formulas/f-counts--n-grams-construction/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/formulas/f-counts/n-grams-construction Formulas explained through full worked examples **Definition.** An n-gram is a contiguous sequence of n items. For word n-grams, the items are consecutive word tokens. A unigram contains one word; a bigram contains two; a trigram contains three. Character n-grams use characters instead, so always identify the unit being counted. Tokens: we | study | natural | language → Move a window one token at a time → Bigrams: we study / study natural / natural language For m tokens without padding, the number of n-gram *positions* is max(0, m−n+1). Repeated positions may contain the same n-gram type. A bag-of-n-grams representation counts each n-gram type and loses the overall sentence order beyond the local order inside each n-gram. $$ \text{Number of unpadded n-gram positions}=\max(0,m-n+1) $$ **Question. List all word unigrams, bigrams and trigrams for “the prime minister of our country”. Do not add boundary tokens.** #### Given Six tokens: the, prime, minister, of, our, country. n = 1, 2 and 3. #### To find All three lists and their counts. #### Formula and why it applies $$ \#\text{positions}=m-n+1 $$ Each window must contain consecutive words and move one position. No wraparound or invented final word is allowed. #### Solution ##### Step 1. Take one word at a time Unigrams: (the), (prime), (minister), (of), (our), (country). Count = 6−1+1 = 6. ##### Step 2. Take two consecutive words Bigrams: (the, prime), (prime, minister), (minister, of), (of, our), (our, country). Count = 6−2+1 = 5. ##### Step 3. Take three consecutive words Trigrams: (the, prime, minister), (prime, minister, of), (minister, of, our), (of, our, country). Count = 6−3+1 = 4. **Final answer.** 6 unigrams, 5 bigrams and 4 trigrams. **Question. Construct bigrams for “I like tea” using one start and one end marker.** #### Given Original tokens: I, like, tea. Add and once each. #### To find The boundary-aware bigram sequence. #### Formula and why it applies $$ \#\text{bigrams}=(m+2)-1=m+1 $$ The start context predicts the first word, and the last word predicts that the sentence ends. #### Solution ##### Step 1. Write the padded sequence I like tea ##### Step 2. Move a two-token window (, I), (I, like), (like, tea), (tea, ). ##### Step 3. Count the windows $$ 3+1=4 $$ **Final answer.** There are four boundary-aware bigrams. **Question. Construct trigrams for “I like tea” using two start markers and one end marker.** #### Given Original sentence has three words. For a trigram model each predicted token needs two prior context tokens. #### To find All trigram prediction events. #### Formula and why it applies $$ P(w_i\mid w_{i-2},w_{i-1}) $$ Two start markers provide a complete context even for the first word. #### Solution ##### Step 1. Pad with two starts I like tea ##### Step 2. Slide a three-token window (, , I), (, I, like), (I, like, tea), (like, tea, ). ##### Step 3. Count predicted events The model predicts I, like, tea and the end marker: four predictions. **Final answer.** Four trigrams. Start markers are context; they are not ordinary predicted words. **Question. Give the bag-of-bigrams counts for “a b a b” without padding.** #### Given Four tokens in their given order. #### To find Bigram positions, distinct bigram types and their counts. #### Formula and why it applies $$ \#\text{positions}=4-2+1=3 $$ A bag stores frequencies; repeated bigrams must not be discarded. #### Solution ##### Step 1. Enumerate all positions (a,b), (b,a), (a,b). ##### Step 2. Group identical pairs | Bigram type | Count | | --- | --- | | a b | 2 | | b a | 1 | **Final answer.** Three bigram positions but two distinct bigram types. **Yaad rakho:** n-gram mein n words; context mein n−1 words. --- ## M2 · Unigram estimation and complete worked examples Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/formulas/f-counts--unigram-estimation/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/formulas/f-counts/unigram-estimation Formulas explained through full worked examples **Maximum likelihood estimation (MLE)** chooses probabilities equal to observed relative frequencies. In a unigram model, every word is predicted independently of its surrounding words. If a word occurs c times in N training tokens, its estimate is c/N. To score m specified words, multiply their m unigram probabilities. $$ P_{\mathrm{MLE}}(w)=\frac{C(w)}{N}\qquad P(w_1,\ldots,w_m)\approx\prod_{i=1}^{m}P(w_i) $$ Choose a tokenization convention before counting. Lowercasing, punctuation handling and boundary markers can change the vocabulary and counts. Never count words with one convention and form the denominator with another. A unigram model gives the same product to rearrangements of the same word multiset; that is one of its limitations. **Question. Train an unpadded, case-insensitive unigram model on “I like tea” and “I like coffee”. Find the probability of “I like tea”.** #### Given Training sentences: I like tea; I like coffee. Use lowercase words; exclude punctuation and sentence boundaries. #### To find The vocabulary, word probabilities and test-sentence probability. #### Formula and why it applies $$ P(w)=C(w)/N,\qquad P(I\ like\ tea)=P(I)P(like)P(tea) $$ The unigram approximation ignores order and uses total token count for every denominator. #### Solution ##### Step 1. Count every training token Each sentence contains three words, so N = 3+3 = 6. ##### Step 2. Count types and frequencies | Word | Count | MLE probability | | --- | --- | --- | | i | 2 | 2/6 = 1/3 | | like | 2 | 2/6 = 1/3 | | tea | 1 | 1/6 | | coffee | 1 | 1/6 | Vocabulary size V = 4. ##### Step 3. Write one factor per test word $$ P=\frac{2}{6}\times\frac{2}{6}\times\frac{1}{6} $$ ##### Step 4. Multiply the fractions $$ P=\frac{4}{216}=\frac{1}{54}\approx0.018519 $$ **Final answer.** The unigram sentence probability is approximately 0.018519. **Question. A corpus contains counts a: 4, b: 3 and c: 1. Find the unigram probability of “a b a”.** #### Given C(a)=4, C(b)=3, C(c)=1. No boundary markers. #### To find P(a b a). #### Formula and why it applies $$ P(a\,b\,a)=P(a)P(b)P(a) $$ A repeated test word contributes a probability factor at every occurrence; do not include it just once. #### Solution ##### Step 1. Find the total $$ N=4+3+1=8 $$ ##### Step 2. Estimate the required probabilities $$ P(a)=4/8=1/2,\qquad P(b)=3/8 $$ ##### Step 3. Use both occurrences of a $$ P=\frac12\times\frac38\times\frac12 $$ ##### Step 4. Simplify $$ P=\frac3{32}=0.09375 $$ **Final answer.** P(a b a) = 0.09375. **Yaad rakho:** Repeated word ka probability bhi repeat multiply hoga. **Question. An unsmoothed unigram model is trained on “red blue red”. Find the probability of “red green”.** #### Given N=3. Counts: red=2, blue=1, green=0. The test contains red and green. #### To find The unsmoothed test probability and the reason for its value. #### Formula and why it applies $$ P(red\ green)=P(red)P(green) $$ MLE assigns zero to an unseen outcome. Smoothing is needed if an unseen in-vocabulary item must receive probability. #### Solution ##### Step 1. Estimate red $$ P(red)=2/3 $$ ##### Step 2. Estimate the unseen word $$ P(green)=0/3=0 $$ ##### Step 3. Multiply $$ (2/3)\times0=0 $$ **Final answer.** Unsmoothed probability = 0. This reflects missing training evidence, not proof that the phrase is impossible. --- ## M2 · Bigram counts: build the table before multiplying Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/formulas/f-counts--bigram-count-table/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/formulas/f-counts/bigram-count-table Formulas explained through full worked examples A **bigram probability table** uses the previous word as its row and the next word as its column. Divide each cell count by its row's total outgoing count. With one end marker on every sentence, each ordinary training word has a continuation, so its outgoing count equals its token frequency. Do not create a bigram that crosses from one sentence into the next. $$ P(w\mid h)=\frac{C(h,w)}{\sum_{v\in V}C(h,v)}=\frac{C(h,w)}{C(h)} $$ Add boundaries to each sentence separately → List adjacent pairs → Count repeated pairs → Divide each row by its total → Follow the test sentence through the table **Question. Train a bigram model on “I like tea”, “I like coffee” and “you like tea”. Use one start and one end marker. Find the probability of “I like tea”.** #### Given Corpus: I like tea ; I like coffee ; you like tea . Preserve these tokens. #### To find All nonzero bigram estimates and the test probability. #### Formula and why it applies $$ P(w\mid h)=C(h,w)/C(h) $$ Every sentence starts and ends independently. Four probability factors are required for the three test words plus end. #### Solution ##### Step 1. List the training pairs Sentence 1: (,I), (I,like), (like,tea), (tea,). Sentence 2: (,I), (I,like), (like,coffee), (coffee,). Sentence 3: (,you), (you,like), (like,tea), (tea,). ##### Step 2. Create the nonzero count table | History → next | Pair count | History count | Probability | | --- | --- | --- | --- | | → I | 2 | 3 | 2/3 | | → you | 1 | 3 | 1/3 | | I → like | 2 | 2 | 1 | | you → like | 1 | 1 | 1 | | like → tea | 2 | 3 | 2/3 | | like → coffee | 1 | 3 | 1/3 | | tea → | 2 | 2 | 1 | | coffee → | 1 | 1 | 1 | ##### Step 3. Write the test path $$ P=P(I\mid\langle s\rangle)P(like\mid I)P(tea\mid like)P(\langle/s\rangle\mid tea) $$ ##### Step 4. Substitute each matching row $$ P=\frac23\times1\times\frac23\times1 $$ ##### Step 5. Multiply $$ P=\frac49\approx0.444444 $$ **Final answer.** P(I like tea) = 4/9 ≈ 0.444444. **Question. Using the same three-sentence corpus given here—“I like tea”, “I like coffee”, “you like tea”—find the bigram probabilities of “you like coffee” and “tea like I”. Include boundaries.** #### Given Corpus with one start/end marker per sentence. Counts: start→you=1/3; you→like=1; like→coffee=1/3; coffee→end=1. No training sentence starts with tea. #### To find The two test probabilities. #### Formula and why it applies $$ P(\text{sentence})=\prod P(\text{next token}\mid\text{previous token}) $$ A model can assign a nonzero probability to a sentence it never saw if all of its local bigrams were seen. #### Solution ##### Step 1. Score you like coffee $$ P=\frac13\times1\times\frac13\times1=\frac19\approx0.111111 $$ ##### Step 2. Inspect tea like I $$ P(tea\mid\langle s\rangle)=0/3=0 $$ ##### Step 3. Apply the zero factor The full product is zero, regardless of the remaining factors. **Final answer.** P(you like coffee) = 1/9; P(tea like I) = 0. A new sentence is not necessarily a zero-probability sentence. **Question. Obtain the unsmoothed bigram probability of “students are from Vellore” from this corpus: “I am from Vellore”; “I am a teacher”; “students are good and are from various cities”; “students from Vellore do engineering”. Use sentence boundaries.** #### Given Four training sentences, each padded with one and . Test sequence: students are from Vellore . #### To find The five required conditional probabilities and their product. #### Formula and why it applies $$ P(w\mid h)=C(h,w)/C(h) $$ Count all occurrences in the entire corpus. In particular, are occurs twice inside the third sentence, and Vellore is followed by end once and do once. #### Solution ##### Step 1. Count the histories | History | Occurrences with continuation | | --- | --- | | | 4 | | students | 2 | | are | 2 | | from | 3 | | Vellore | 2 | ##### Step 2. Count the required pairs | Required pair | Count | Probability | | --- | --- | --- | | students | 2 | 2/4 | | students are | 1 | 1/2 | | are from | 1 | 1/2 | | from Vellore | 2 | 2/3 | | Vellore | 1 | 1/2 | ##### Step 3. Write the complete product $$ P=\frac24\times\frac12\times\frac12\times\frac23\times\frac12 $$ ##### Step 4. Multiply progressively $$ \frac24\times\frac12=\frac14,\quad \frac14\times\frac12=\frac18 $$ $$ \frac18\times\frac23=\frac1{12},\quad \frac1{12}\times\frac12=\frac1{24} $$ **Final answer.** P = 1/24 ≈ 0.0416667. **Yaad rakho:** Same sentence mein word do baar aaye toh count bhi do. --- ## M2 · Trigram estimates: two-word history, not two denominators Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/formulas/f-counts--trigram-estimation/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/formulas/f-counts/trigram-estimation Formulas explained through full worked examples A trigram model predicts the next word from the two preceding words. The numerator is the count of a triple. The denominator is the count of its two-word history with an available next token. Do not multiply two independent bigram probabilities and call that a trigram probability. $$ P(w_i\mid w_{i-2},w_{i-1})=\frac{C(w_{i-2},w_{i-1},w_i)}{C(w_{i-2},w_{i-1})} $$ **Question. A corpus contains 12 occurrences of “prime minister”. Eight are followed by “of”, three by “said”, and one by “visited”. Estimate the three trigram probabilities.** #### Given History h = (prime, minister). Count of this history = 12. Triple counts for of, said, visited are 8, 3, 1. #### To find The conditional distribution of the next word. #### Formula and why it applies $$ P(w\mid prime,minister)=C(prime,minister,w)/12 $$ All three candidates share one two-word history, so they share denominator 12. #### Solution ##### Step 1. Calculate of $$ P(of\mid prime,minister)=8/12=2/3 $$ ##### Step 2. Calculate said $$ P(said\mid prime,minister)=3/12=1/4 $$ ##### Step 3. Calculate visited $$ P(visited\mid prime,minister)=1/12 $$ ##### Step 4. Confirm normalization $$ 8/12+3/12+1/12=12/12=1 $$ **Final answer.** The probabilities are 2/3, 1/4 and 1/12. **Question. Train a trigram model on “I like tea”, “I like coffee” and “you like tea”. Use two start markers and one end marker. Score “I like tea”.** #### Given Three sentences. Padded first sentence is I like tea ; pad the other two in the same way. #### To find Four trigram prediction probabilities and the sentence probability. #### Formula and why it applies $$ P=\prod_i P(w_i\mid w_{i-2},w_{i-1}) $$ Each prediction uses a two-token context; the two start markers supply the first history. #### Solution ##### Step 1. Count the first-word event The history (,) occurs three times, and two are followed by I. $$ P(I\mid\langle s\rangle,\langle s\rangle)=2/3 $$ ##### Step 2. Count the second-word event (,I) occurs twice; both are followed by like. $$ P(like\mid\langle s\rangle,I)=2/2=1 $$ ##### Step 3. Count the third-word event (I,like) occurs twice; one is followed by tea. $$ P(tea\mid I,like)=1/2 $$ ##### Step 4. Count the end event (like,tea) occurs twice; both are followed by end. $$ P(\langle/s\rangle\mid like,tea)=2/2=1 $$ ##### Step 5. Multiply all four $$ P=\frac23\times1\times\frac12\times1=\frac13 $$ **Final answer.** Trigram P(I like tea) = 1/3. It differs from the bigram estimate because the context and denominator changed. **Question. A trigram model supplies P(we|,)=0.4, P(study|,we)=0.5, P(NLP|we,study)=0.8 and P(|study,NLP)=0.25. Score “we study NLP”.** #### Given Four trigram conditional factors. The end marker is included. #### To find The complete sentence probability. #### Formula and why it applies $$ P=P(we\mid s,s)P(study\mid s,we)P(NLP\mid we,study)P(end\mid study,NLP) $$ Use each supplied two-word history exactly; do not swap its word order. #### Solution ##### Step 1. Insert the values $$ P=0.4\times0.5\times0.8\times0.25 $$ ##### Step 2. Multiply the first pair $$ 0.4\times0.5=0.2 $$ ##### Step 3. Include NLP $$ 0.2\times0.8=0.16 $$ ##### Step 4. Include the end $$ 0.16\times0.25=0.04 $$ **Final answer.** P = 0.04. **Yaad rakho:** Trigram: teen-word count upar, pehle do-word count neeche. **Why larger n is not automatically better.** More context can capture local order more precisely, but each longer history occurs less often. Parameter count, storage and sparsity grow. A model trained on Shakespeare may assign poor probabilities to financial news because training and test distributions differ. Keep a held-out test set; do not evaluate only on memorized training sentences. --- ## M2 · Laplace smoothing: why both numerator and denominator change Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/formulas/f-smoothing--laplace-basics/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/formulas/f-smoothing/laplace-basics Formulas explained through full worked examples **Smoothing** reallocates some probability mass to events that received zero or unreliable estimates from finite training data. **Laplace, or add-one, smoothing** adds one pseudo-count to every possible next outcome. If there are V possible outcomes, the total added count is V, so adding 1 only to the numerator would produce probabilities that do not sum to 1. $$ P_{\mathrm{Laplace}}(w)=\frac{C(w)+1}{N+V} $$ $$ P_{\mathrm{Laplace}}(w\mid h)=\frac{C(h,w)+1}{C(h)+V} $$ For unigram smoothing, N is the total training token count. For bigram smoothing, C(h) is the count of the particular history. In both cases V is the number of outcomes being predicted, not the number of training sentences and not the number of distinct bigrams. When end is a possible prediction, include it in V; the start marker is usually context only. Observed counts: 3, 1, 0 → Add one to every outcome: 4, 2, 1 → New total: 7 → Smoothed probabilities: 4/7, 2/7, 1/7 **Question. A fixed unigram vocabulary is {a,b,c}. The training counts are 3,1,0. Compute unsmoothed and Laplace-smoothed probabilities.** #### Given C(a)=3, C(b)=1, C(c)=0; N=4; V=3. #### To find The complete distributions. #### Formula and why it applies $$ P(w)=C(w)/N,\qquad P_L(w)=(C(w)+1)/(N+V) $$ The zero-count c is still in the fixed vocabulary, so it must receive one pseudo-count and contribute to V. #### Solution ##### Step 1. Compute unsmoothed values | Word | Unsmoothed | | --- | --- | | a | 3/4 = 0.75 | | b | 1/4 = 0.25 | | c | 0/4 = 0 | ##### Step 2. Calculate the new denominator $$ N+V=4+3=7 $$ ##### Step 3. Add one to each numerator $$ C(a)+1=4,\quad C(b)+1=2,\quad C(c)+1=1 $$ ##### Step 4. Divide | Word | Smoothed | | --- | --- | | a | 4/7 ≈ 0.571429 | | b | 2/7 ≈ 0.285714 | | c | 1/7 ≈ 0.142857 | ##### Step 5. Confirm the total $$ 4/7+2/7+1/7=1 $$ **Final answer.** Smoothed probabilities: a=4/7, b=2/7, c=1/7. Common a loses mass; unseen c gains mass. **Question. After history I, a corpus records like three times, study once and sleep zero times. The allowed next vocabulary is {like,study,sleep}. Compute all Laplace-smoothed bigram probabilities.** #### Given C(I,like)=3; C(I,study)=1; C(I,sleep)=0. History count C(I)=4. V=3. #### To find P\_L(like|I), P\_L(study|I), P\_L(sleep|I). #### Formula and why it applies $$ P_L(w\mid I)=\frac{C(I,w)+1}{C(I)+V} $$ This is smoothing within the I row; use its total 4, not the global corpus token count. #### Solution ##### Step 1. Form the common denominator $$ C(I)+V=4+3=7 $$ ##### Step 2. Calculate like $$ P_L(like\mid I)=(3+1)/7=4/7 $$ ##### Step 3. Calculate study $$ P_L(study\mid I)=(1+1)/7=2/7 $$ ##### Step 4. Calculate sleep $$ P_L(sleep\mid I)=(0+1)/7=1/7 $$ **Final answer.** The row is (4/7, 2/7, 1/7). An unseen bigram is no longer forced to zero. **Question. A fixed vocabulary has five possible next tokens. A history h is unseen in training. What is its add-one probability for each next token?** #### Given C(h)=0; C(h,w)=0 for each outcome; V=5. #### To find P\_L(w|h) for each token. #### Formula and why it applies $$ P_L(w\mid h)=\frac{0+1}{0+V} $$ All outcomes receive equal pseudo-counts because the training data gives no preference for this history. #### Solution ##### Step 1. Add the pseudo-count $$ 0+1=1 $$ ##### Step 2. Form the denominator $$ 0+5=5 $$ ##### Step 3. Divide and sum $$ P_L(w\mid h)=1/5=0.2,\qquad5(0.2)=1 $$ **Final answer.** Every outcome gets 0.2 under this fixed-vocabulary model. **Vocabulary warning.** Add-one does not automatically define probabilities for every word in the world. Unknown words require a declared vocabulary/UNK convention. If a question supplies V, use that stated V rather than inventing additional tokens. **Yaad rakho:** Add-one: upar +1, neeche +V; bigram mein neeche history ka count. --- ## M2 · A complete unseen-bigram question from corpus to final answer Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/formulas/f-smoothing--smoothed-sentence/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/formulas/f-smoothing/smoothed-sentence Formulas explained through full worked examples **Question. Consider the corpus “I love to code in Python. I love to learn new things. I code every day.” Obtain the bigram probabilities needed for “I love Python”. Explain the zero-probability issue, then use Laplace smoothing to score it. Treat each sentence separately, preserve case, remove punctuation, and include an end token among predicted outcomes.** #### Given Three sentences padded with one start/end marker. Distinct words: I, love, to, code, in, Python, learn, new, things, every, day. With , V=12. Start is context only. #### To find Unsmoothed and smoothed P(I love Python | ). #### Formula and why it applies $$ P(w\mid h)=\frac{C(h,w)}{C(h)},\qquad P_L(w\mid h)=\frac{C(h,w)+1}{C(h)+12} $$ The test contains an unseen transition love→Python. Smoothing assigns probability to that transition and renormalizes every required history row. #### Solution ##### Step 1. Count the contexts | History | Count | | --- | --- | | | 3 | | I | 3 | | love | 2 | | Python | 1 | ##### Step 2. Count each required pair | Pair | Count | | --- | --- | | I | 3 | | I love | 2 | | love Python | 0 | | Python | 1 | ##### Step 3. Compute the unsmoothed product $$ P=\frac33\times\frac23\times\frac02\times\frac11=0 $$ Both words love and Python were seen, but their adjacent pair was not. This is an unseen bigram, not an unknown-word case. ##### Step 4. Smooth the first event $$ P_L(I\mid\langle s\rangle)=\frac{3+1}{3+12}=\frac4{15} $$ ##### Step 5. Smooth the second event $$ P_L(love\mid I)=\frac{2+1}{3+12}=\frac3{15} $$ ##### Step 6. Smooth the unseen event $$ P_L(Python\mid love)=\frac{0+1}{2+12}=\frac1{14} $$ ##### Step 7. Smooth the end event $$ P_L(\langle/s\rangle\mid Python)=\frac{1+1}{1+12}=\frac2{13} $$ ##### Step 8. Multiply the smoothed factors $$ P_L=\frac4{15}\times\frac3{15}\times\frac1{14}\times\frac2{13} $$ $$ P_L=\frac{24}{40950}=\frac4{6825}\approx0.000586081 $$ **Final answer.** Unsmoothed P=0. With the stated V=12 and boundaries, Laplace-smoothed P≈0.000586081. If an exam states a different convention—for example excluding boundary tokens from the entire calculation—recompute both V and the set of probability factors consistently. Do not combine a boundary-aware numerator with a boundary-free denominator. --- ## M2 · Adjusted counts and relative discounting Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/formulas/f-smoothing--adjusted-count-discount/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/formulas/f-smoothing/adjusted-count-discount Formulas explained through full worked examples The slides also express smoothing through an **adjusted count** c\*. Instead of writing the smoothed probability with denominator N+V, they rescale its numerator so the denominator can remain N. Algebraically these are equivalent: $$ c^*=(c+1)\frac{N}{N+V},\qquad P_L(w)=\frac{c^*}{N} $$ $$ d=\frac{c^*}{c}\quad(c>0) $$ The relative discount d compares adjusted and original counts. It is undefined when c=0 because division by zero is undefined. Add-one smoothing moves the distribution toward uniformity: frequent events are reduced, but some rare seen events may increase. It is not correct to claim that every nonzero count must decrease under Laplace smoothing. **Question. For N=20 and V=5, a word has original count c=10. Find its adjusted count, smoothed probability and relative discount.** #### Given N=20; V=5; c=10. #### To find c\*, P\_L and d. #### Formula and why it applies $$ c^*=(c+1)N/(N+V),\quad P_L=c^*/N,\quad d=c^*/c $$ These expressions rewrite add-one smoothing without changing its probability. #### Solution ##### Step 1. Form the new total $$ N+V=20+5=25 $$ ##### Step 2. Find the adjusted count $$ c^*=(10+1)\frac{20}{25}=11(0.8)=8.8 $$ ##### Step 3. Find the probability $$ P_L=8.8/20=0.44 $$ ##### Step 4. Find the relative discount $$ d=8.8/10=0.88 $$ **Final answer.** c\*=8.8, P\_L=0.44, d=0.88. The original probability 0.5 is reduced. **Question. Under N=20, V=5, a rare word occurs once. Compute its adjusted count and compare the original and smoothed probabilities.** #### Given N=20; V=5; c=1. #### To find c\*, original P and smoothed P\_L. #### Formula and why it applies $$ c^*=(c+1)N/(N+V) $$ The same smoothing rule may increase a rare event's probability. #### Solution ##### Step 1. Original probability $$ P=1/20=0.05 $$ ##### Step 2. Adjusted count $$ c^*=(1+1)(20/25)=1.6 $$ ##### Step 3. Smoothed probability $$ P_L=1.6/20=0.08 $$ **Final answer.** c\*=1.6 and P\_L=0.08, which exceeds the original 0.05. **Question. For N=20 and V=5, an in-vocabulary word has count zero. Find its adjusted count and smoothed probability. Can a relative discount be calculated?** #### Given c=0; N=20; V=5. #### To find c\*, P\_L and whether d exists. #### Formula and why it applies $$ c^*=(c+1)N/(N+V),\qquad P_L=c^*/N $$ The pseudo-count creates positive mass for the unseen vocabulary item. #### Solution ##### Step 1. Adjusted count $$ c^*=(0+1)(20/25)=0.8 $$ ##### Step 2. Smoothed probability $$ P_L=0.8/20=0.04 $$ ##### Step 3. Relative discount d would be 0.8/0. That is undefined, so do not report a finite discount. **Final answer.** c\*=0.8 and P\_L=0.04; relative discount is undefined for c=0. --- ## M2 · Perplexity: meaning, formula and four complete examples Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/formulas/f-smoothing--evaluation-perplexity/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/formulas/f-smoothing/evaluation-perplexity Formulas explained through full worked examples **Intrinsic evaluation** evaluates a model directly on a held-out test corpus, for example using perplexity. **Extrinsic evaluation** places it inside a downstream application, such as a speech recognizer or translation system, and measures the application's performance. Extrinsic testing is closer to actual use but often slower and more expensive. A lower perplexity is not a guarantee of improvement on every downstream task. **Perplexity** is the inverse probability of a test sequence, normalized by the number of prediction events. Intuitively, it is the model's effective average number of equally plausible choices. A model that always gives the correct next word probability 1/4 has perplexity 4. Lower is better when the models are tested on the same data, vocabulary, tokenization and boundary convention. $$ \operatorname{PP}(W)=P(W)^{-1/N}=\sqrt[N]{\frac1{P(W)}} $$ | Symbol | Meaning | | --- | --- | | W | The complete test sequence. | | P(W) | Product of the probabilities assigned to the actual next tokens. | | N | Number of scored prediction events under the stated convention. | | Exponent −1/N | Take the reciprocal, then the Nth root. | For calculator entry, use parentheses: P raised to (−1 ÷ N), or (1 ÷ P) raised to (1 ÷ N). Retain several digits in intermediate products. The ordinary start marker is usually given context, not a predicted event; the end marker counts if its probability is included. **Question. A model assigns probability 0.0625 to a test sequence containing four scored tokens. Compute its perplexity.** #### Given P(W)=0.0625=1/16; N=4. #### To find PP(W). #### Formula and why it applies $$ \operatorname{PP}(W)=\left(\frac1{P(W)}\right)^{1/N} $$ The root removes the effect of sequence length; the reciprocal makes lower surprise correspond to lower perplexity. #### Solution ##### Step 1. Take the reciprocal $$ 1/P(W)=1/0.0625=16 $$ ##### Step 2. Use the fourth root $$ \operatorname{PP}=16^{1/4} $$ ##### Step 3. Evaluate $$ 2^4=16\quad\Rightarrow\quad\operatorname{PP}=2 $$ **Final answer.** Perplexity = 2. **Question. Two language models score the same three-token test sequence. Model A gives next-token probabilities 0.5, 0.5, 0.5. Model B gives 0.5, 0.25, 0.25. Compute their perplexities and select the better intrinsic result.** #### Given Same sequence and N=3 for both models. A factors: 0.5,0.5,0.5. B factors: 0.5,0.25,0.25. #### To find Both sequence probabilities and perplexities. #### Formula and why it applies $$ P(W)=\prod_i p_i,\qquad\operatorname{PP}=P(W)^{-1/3} $$ A fair comparison uses identical test tokens and scoring conventions. #### Solution ##### Step 1. Calculate A's probability $$ P_A=0.5\times0.5\times0.5=0.125=1/8 $$ ##### Step 2. Calculate A's perplexity $$ \operatorname{PP}_A=8^{1/3}=2 $$ ##### Step 3. Calculate B's probability $$ P_B=0.5\times0.25\times0.25=0.03125=1/32 $$ ##### Step 4. Calculate B's perplexity $$ \operatorname{PP}_B=32^{1/3}\approx3.174802 $$ ##### Step 5. Compare 2 is lower than 3.174802; A places more probability on the observed sequence. **Final answer.** Model A is better by perplexity on this test: 2 versus about 3.174802. **Question. The bigram training corpus is “This is a dog”, “This is a cat”, “I love my cat”, “This is my name”. Score “This is my cat”, including an end marker, and compute perplexity using five predicted events.** #### Given Each training sentence has one start and end marker. Test predictions: This, is, my, cat, . Thus N=5. #### To find Bigram sentence probability and perplexity. #### Formula and why it applies $$ P(w\mid h)=C(h,w)/C(h),\qquad\operatorname{PP}=P(W)^{-1/5} $$ Start is given context. Four words plus the end marker produce five probability factors. #### Solution ##### Step 1. Count the required transitions | Transition | Pair count / history count | | --- | --- | | →This | 3/4 | | This→is | 3/3 | | is→my | 1/3 | | my→cat | 1/2 | | cat→ | 2/2 | ##### Step 2. Form the product $$ P=\frac34\times1\times\frac13\times\frac12\times1=\frac18 $$ ##### Step 3. Take the reciprocal $$ 1/P=8 $$ ##### Step 4. Take the fifth root $$ \operatorname{PP}=8^{1/5}\approx1.515717 $$ **Final answer.** P(W)=1/8 and perplexity≈1.515717 using five predicted events. **If the question explicitly states N = 6:** use that supplied convention and report PP = 81/6 ≈ 1.414214. The class slide counts both boundary markers in N. Write the chosen N beside the formula; do not silently switch between 5 and 6. Also, This occurs three times and is follows it three times, so P(is|This)=3/3, not 3/4. **Question. A uniform model assigns probability 1/10 to each actual next word in a five-token test. Find its perplexity.** #### Given Five scored tokens, each with assigned probability 0.1. N=5. #### To find The sentence probability and perplexity. #### Formula and why it applies $$ P(W)=(1/10)^5,\qquad \operatorname{PP}=P(W)^{-1/5} $$ Uniform equal probabilities make the effective number of choices especially clear. #### Solution ##### Step 1. Multiply the five factors $$ P(W)=0.1^5=0.00001=10^{-5} $$ ##### Step 2. Invert $$ 1/P(W)=10^5 $$ ##### Step 3. Take the fifth root $$ \operatorname{PP}=(10^5)^{1/5}=10 $$ **Final answer.** Perplexity = 10. **Yaad rakho:** Probability high → surprise low → perplexity low. --- ## M2 · Log probabilities: replace long products with sums Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/formulas/f-smoothing--log-probabilities/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/formulas/f-smoothing/log-probabilities Formulas explained through full worked examples Probabilities are at most 1, so multiplying many of them produces very small numbers. Computers can round extremely small positive products to zero: **numerical underflow**. Log probabilities convert multiplication into addition and are more stable for long sequences. Natural logs use base e; log₂ uses base 2. Choose one base consistently. $$ \ln P(W)=\sum_{i=1}^{N}\ln p_i $$ $$ P(W)=e^{\ln P(W)},\qquad \operatorname{PP}(W)=\exp\left(-\frac1N\sum_i\ln p_i\right) $$ The log of a probability below 1 is negative. A value closer to zero means a larger probability: ln(0.5) ≈ −0.693 is larger than ln(0.1) ≈ −2.303. A zero probability has log −∞; taking logs does not solve the unseen-event problem. Smoothing handles that separate problem. **Question. The next-word probabilities are 0.5, 0.25 and 0.5. Calculate the natural log probability, ordinary probability and perplexity.** #### Given p₁=0.5; p₂=0.25; p₃=0.5; N=3. #### To find ln P(W), P(W), and PP. #### Formula and why it applies $$ \ln P=\sum_i\ln p_i,\quad P=e^{\ln P},\quad\operatorname{PP}=e^{-\ln P/N} $$ Logarithms preserve products as sums and the exponent converts the result back. #### Solution ##### Step 1. Compute individual logs $$ \ln0.5\approx-0.693147,\quad\ln0.25\approx-1.386294 $$ ##### Step 2. Add all three terms $$ \ln P=-0.693147-1.386294-0.693147\approx-2.772589 $$ ##### Step 3. Recover the probability $$ P=e^{-2.772589}\approx0.0625 $$ ##### Step 4. Find average negative log probability $$ -\ln P/3\approx2.772589/3=0.924196 $$ ##### Step 5. Exponentiate $$ \operatorname{PP}=e^{0.924196}\approx2.519842 $$ **Final answer.** ln P≈−2.772589, P=0.0625, PP≈2.519842. **Question. A model has a total natural log probability −12 over six scored tokens. Calculate its perplexity.** #### Given ln P(W)=−12; N=6. #### To find PP without forming the tiny ordinary probability. #### Formula and why it applies $$ \operatorname{PP}=\exp(-\ln P(W)/N) $$ The logarithmic formula directly uses the supplied log score. #### Solution ##### Step 1. Negate the log score $$ -\ln P=12 $$ ##### Step 2. Divide by token count $$ 12/6=2 $$ ##### Step 3. Apply exponential $$ \operatorname{PP}=e^2\approx7.389056 $$ **Final answer.** Perplexity≈7.389056. **Question. A language model's average negative log₂ probability is 3 bits per token. Find its perplexity.** #### Given Cross-entropy H=3 bits/token. Log base is 2. #### To find PP. #### Formula and why it applies $$ \operatorname{PP}=2^H $$ The inverse operation of log₂ is raising 2 to that power; use e only for natural logs. #### Solution ##### Step 1. Identify the log base Bits mean base 2. ##### Step 2. Raise the base to H $$ \operatorname{PP}=2^3=8 $$ **Final answer.** Perplexity = 8. **Yaad rakho:** ln use kiya toh e se wapas; log₂ use kiya toh 2 se wapas. --- ## M2 · Markov chains: read a transition table and multiply a path Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/formulas/f-hmm--markov-chain/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/formulas/f-hmm/markov-chain Formulas explained through full worked examples **A Markov chain** is a probabilistic model of a sequence of states. Under the first-order Markov assumption, the next state's probability depends on the current state rather than the complete previous history. The **initial probability** tells us how likely each first state is. A **transition probability** tells us how likely a move from one state to another is. $$ P(s_t\mid s_1,\ldots,s_{t-1})=P(s_t\mid s_{t-1}) $$ $$ P(s_1,\ldots,s_T)=P(s_1)\prod_{t=2}^{T}P(s_t\mid s_{t-1}) $$ For T states in a path, use one initial factor and T−1 transition factors. A row-oriented transition matrix must have each complete row sum to 1. Always read the row/column labels; some exam diagrams use a transposed arrangement. | Current → next | Rain | Dry | | --- | --- | --- | | Rain | 0.3 | 0.7 | | Dry | 0.2 | 0.8 | Initial probabilities: P(Rain)=0.4, P(Dry)=0.6. Rows describe the current state; columns describe the next state. **Question. For the Rain/Dry Markov chain with initial P(Rain)=0.4 and P(Dry)=0.6, and transitions Rain→Rain=0.3, Rain→Dry=0.7, Dry→Rain=0.2, Dry→Dry=0.8, find the probability of Dry, Dry, Rain, Rain.** #### Given | Current → next | Rain | Dry | | --- | --- | --- | | Rain | 0.3 | 0.7 | | Dry | 0.2 | 0.8 | Initial probabilities: P(Rain)=0.4, P(Dry)=0.6. Rows describe the current state; columns describe the next state. #### To find P(Dry,Dry,Rain,Rain). #### Formula and why it applies $$ P(D,D,R,R)=P(D)P(D\mid D)P(R\mid D)P(R\mid R) $$ Each step conditions on the immediately preceding state. The sequence contains four states, so it needs one initial plus three transition factors. #### Solution ##### Step 1. Start in Dry $$ P(D)=0.6 $$ ##### Step 2. Move Dry to Dry $$ 0.6\times0.8=0.48 $$ ##### Step 3. Move Dry to Rain $$ 0.48\times0.2=0.096 $$ ##### Step 4. Move Rain to Rain $$ 0.096\times0.3=0.0288 $$ **Final answer.** Path probability = 0.0288. **Question. For initial probabilities Rain=0.4, Dry=0.6 and transition rows Rain=(0.3,0.7), Dry=(0.2,0.8) in column order Rain,Dry, find the probability of Rain, Dry, Rain.** #### Given | Current → next | Rain | Dry | | --- | --- | --- | | Rain | 0.3 | 0.7 | | Dry | 0.2 | 0.8 | Initial probabilities: P(Rain)=0.4, P(Dry)=0.6. Rows describe the current state; columns describe the next state. #### To find P(Rain,Dry,Rain). #### Formula and why it applies $$ P(R,D,R)=P(R)P(D\mid R)P(R\mid D) $$ Do not replace Dry→Rain with Rain→Dry: the two directional probabilities differ. #### Solution ##### Step 1. Initial Rain $$ P(R)=0.4 $$ ##### Step 2. First transition $$ P(D\mid R)=0.7,\quad0.4(0.7)=0.28 $$ ##### Step 3. Second transition $$ P(R\mid D)=0.2,\quad0.28(0.2)=0.056 $$ **Final answer.** Path probability = 0.056. **Question. For the same fully specified chain—initial Rain=0.4, Dry=0.6; Rain→Rain=0.3, Rain→Dry=0.7, Dry→Rain=0.2, Dry→Dry=0.8—find P(Dry,Dry). Also find the conditional probability of next Dry given current Dry.** #### Given | Current → next | Rain | Dry | | --- | --- | --- | | Rain | 0.3 | 0.7 | | Dry | 0.2 | 0.8 | Initial probabilities: P(Rain)=0.4, P(Dry)=0.6. Rows describe the current state; columns describe the next state. #### To find The two-state joint probability and the one-step conditional probability. #### Formula and why it applies $$ P(D,D)=P(D)P(D\mid D) $$ The joint path includes an uncertain initial state. The conditional transition assumes the current state is already Dry. #### Solution ##### Step 1. Read the conditional transition $$ P(D\mid D)=0.8 $$ ##### Step 2. Include the initial state for the joint path $$ P(D,D)=0.6\times0.8=0.48 $$ **Final answer.** P(Dry,Dry)=0.48; P(next Dry|current Dry)=0.8. These answer different questions. **Yaad rakho:** Path: pehle start probability, phir har arrow ki probability. --- ## M2 · Estimate transition and emission probabilities from tagged data Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/formulas/f-hmm--transition-emission-estimation/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/formulas/f-hmm/transition-emission-estimation Formulas explained through full worked examples $$ P(t'\mid t)=\frac{C(t,t')}{C(t)\text{ with a following tag/end event}} $$ $$ P(w\mid t)=\frac{C(w,t)}{C(t)} $$ In a transition denominator, count the previous tag's outgoing events. In an emission denominator, count all tokens bearing that tag. With explicit end transitions the totals align conveniently. Keep the two numerator types separate: tag→tag counts are not word→tag counts. **Question. A tagged corpus contains 116454 determiner tags DT, each counted with an outgoing event. DT is followed by NN 56509 times. Estimate P(NN|DT).** #### Given C(DT)=116454; C(DT,NN)=56509. #### To find The DT→NN transition probability. #### Formula and why it applies $$ P(NN\mid DT)=C(DT,NN)/C(DT) $$ The denominator is the number of previous DT contexts because DT is the condition. #### Solution ##### Step 1. Identify the transition count DT followed by NN occurs 56509 times. ##### Step 2. Identify the conditional total There are 116454 DT contexts. ##### Step 3. Divide $$ P(NN\mid DT)=\frac{56509}{116454}\approx0.485247 $$ **Final answer.** P(NN|DT)≈0.485247. **Question. The tag VBZ occurs 21627 times, and 10073 of those tokens are the word “is”. Estimate P(is|VBZ).** #### Given C(VBZ)=21627; C(is,VBZ)=10073. #### To find The emission probability of is from VBZ. #### Formula and why it applies $$ P(is\mid VBZ)=C(is,VBZ)/C(VBZ) $$ The condition is the tag VBZ; we ask which word it emits. #### Solution ##### Step 1. Choose the word-tag count The required word is is with tag VBZ: count 10073. ##### Step 2. Choose the tag total All VBZ-tagged word tokens: count 21627. ##### Step 3. Divide $$ P(is\mid VBZ)=\frac{10073}{21627}\approx0.465760 $$ **Final answer.** P(is|VBZ)≈0.465760. This is different from P(VBZ|is). **Question. Use the tagged sentences “dogs/N chase/V cats/N” and “cats/N sleep/V”. Estimate the initial tag probabilities, all nonzero transitions including END, and emissions.** #### Given Corpus: START dogs/N chase/V cats/N END; START cats/N sleep/V END. #### To find Initial distribution, transition rows and emission rows. #### Formula and why it applies $$ \pi_t=C(START,t)/2,\quad a_{tu}=C(t,u)/C(t),\quad b_t(w)=C(w,t)/C(t) $$ The corpus explicitly tells us the tags, so we can count training frequencies directly. #### Solution ##### Step 1. Count tag tokens N occurs 3 times: dogs, cats, cats. V occurs 2 times: chase, sleep. ##### Step 2. Count sentence starts Both sentences start with N. $$ \pi_N=2/2=1,\quad \pi_V=0/2=0 $$ ##### Step 3. Count transitions | Pair | Count | | --- | --- | | N→V | 2 | | N→END | 1 | | V→N | 1 | | V→END | 1 | ##### Step 4. Normalize transition rows | Current | To N | To V | To END | | --- | --- | --- | --- | | N | 0 | 2/3 | 1/3 | | V | 1/2 | 0 | 1/2 | ##### Step 5. Count and normalize emissions | Tag | dogs | cats | chase | sleep | | --- | --- | --- | --- | --- | | N | 1/3 | 2/3 | 0 | 0 | | V | 0 | 0 | 1/2 | 1/2 | **Final answer.** π=(1,0); transitions and emissions are given in the completed tables. Each complete probability row sums to 1. **Yaad rakho:** Tag→tag table aur tag→word table ko mix mat karna. --- ## M2 · HMM path probabilities: three complete numerical examples Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/formulas/f-hmm--hot-cold-paths/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/formulas/f-hmm/hot-cold-paths Formulas explained through full worked examples The following model uses hidden weather states Hot (H) and Cold (C). Observations are numbers 1, 2 and 3. These numbers are observation labels, not quantities to multiply directly. Their emission probabilities come from the table. | Initial state | Probability | | --- | --- | | H | 0.8 | | C | 0.2 | | From → to | H | C | | --- | --- | --- | | H | 0.7 | 0.3 | | C | 0.4 | 0.6 | | Emission state | Observation 1 | Observation 2 | Observation 3 | | --- | --- | --- | --- | | H | 0.2 | 0.4 | 0.4 | | C | 0.5 | 0.4 | 0.1 | **Question. An HMM has π(H)=0.8, π(C)=0.2; H→H=0.7, H→C=0.3, C→H=0.4, C→C=0.6. Emissions for observations (1,2,3) are H:(0.2,0.4,0.4) and C:(0.5,0.4,0.1). Find the joint probability of hidden path H,C,H and observations 1,3,1.** #### Given Hidden path H→C→H. Observations 1→3→1. Model probabilities are specified in the question. There is no END factor. #### To find P(H,C,H; 1,3,1). #### Formula and why it applies $$ P=\pi_H b_H(1)a_{HC}b_C(3)a_{CH}b_H(1) $$ There are three hidden states and observations: three emissions, two transitions, and one initial probability. #### Solution ##### Step 1. Start Hot and emit 1 $$ 0.8\times0.2=0.16 $$ ##### Step 2. Transition Hot to Cold $$ 0.16\times0.3=0.048 $$ ##### Step 3. Cold emits 3 $$ 0.048\times0.1=0.0048 $$ ##### Step 4. Transition Cold to Hot $$ 0.0048\times0.4=0.00192 $$ ##### Step 5. Hot emits the last 1 $$ 0.00192\times0.2=0.000384 $$ **Final answer.** Joint probability = 0.000384 = 3.84×10⁻⁴. **Question. For π(H)=0.8, π(C)=0.2; H→H=0.7, H→C=0.3, C→H=0.4, C→C=0.6; emissions H:(0.2,0.4,0.4), C:(0.5,0.4,0.1) for labels (1,2,3), find the joint probability of path H,H,H and observations 1,3,1.** #### Given State path H→H→H, observations 1→3→1; no END factor. #### To find P(H,H,H;1,3,1). #### Formula and why it applies $$ P=\pi_H b_H(1)a_{HH}b_H(3)a_{HH}b_H(1) $$ The state remains Hot; each transition still contributes its probability 0.7. #### Solution ##### Step 1. Initial state and first emission $$ 0.8(0.2)=0.16 $$ ##### Step 2. First self-transition $$ 0.16(0.7)=0.112 $$ ##### Step 3. Emit observation 3 $$ 0.112(0.4)=0.0448 $$ ##### Step 4. Second self-transition $$ 0.0448(0.7)=0.03136 $$ ##### Step 5. Emit final observation 1 $$ 0.03136(0.2)=0.006272 $$ **Final answer.** Joint probability = 0.006272. A self-transition is not automatically probability 1. **Question. For π(H)=0.8, π(C)=0.2; transitions H:(0.7,0.3), C:(0.4,0.6) in next-state order H,C; emissions H:(0.2,0.4,0.4), C:(0.5,0.4,0.1) for observations 1,2,3, find the score of path C,C,H with observations 1,3,1.** #### Given Hidden path C→C→H. Observation sequence 1→3→1. No end-state factor. #### To find The joint path-and-observation probability. #### Formula and why it applies $$ P=\pi_C b_C(1)a_{CC}b_C(3)a_{CH}b_H(1) $$ The initial state is Cold, so use π(C), not π(H). Each emission is selected from the state at the same position. #### Solution ##### Step 1. Start Cold and emit 1 $$ 0.2(0.5)=0.1 $$ ##### Step 2. Stay Cold $$ 0.1(0.6)=0.06 $$ ##### Step 3. Emit 3 from Cold $$ 0.06(0.1)=0.006 $$ ##### Step 4. Move Cold to Hot $$ 0.006(0.4)=0.0024 $$ ##### Step 5. Emit 1 from Hot $$ 0.0024(0.2)=0.00048 $$ **Final answer.** Joint probability = 0.00048. **Yaad rakho:** Har column par: state ka emission. Columns ke beech: transition. --- ## M2 · POS HMM worked corpus: Mary, Jane, can, will and spot Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/formulas/f-hmm--tagging-corpus-case/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/formulas/f-hmm/tagging-corpus-case Formulas explained through full worked examples The class example contains words that can be either names or ordinary verbs/modal auxiliaries. The term is **modal** (M), not “model”. To avoid guessing tags from spelling, the training corpus below supplies them explicitly and lowercases words for the emission vocabulary. | Training sentence | Gold tags | | --- | --- | | mary jane can see will | N N M V N | | spot will see mary | N M V N | | will jane spot mary | M N V N | | mary will pat spot | N M V N | There are 9 noun tokens, 4 modal tokens and 4 verb tokens. Each sentence includes a start and end transition. For nouns, the next event can be another noun, modal, verb or END. | Current → next | N | M | V | END | | --- | --- | --- | --- | --- | | START | 3/4 | 1/4 | 0 | 0 | | N | 1/9 | 3/9 | 1/9 | 4/9 | | M | 1/4 | 0 | 3/4 | 0 | | V | 4/4 | 0 | 0 | 0 | | Tag → word | mary | jane | can | see | will | spot | pat | | --- | --- | --- | --- | --- | --- | --- | --- | | N | 4/9 | 2/9 | 0 | 0 | 1/9 | 2/9 | 0 | | M | 0 | 0 | 1/4 | 0 | 3/4 | 0 | 0 | | V | 0 | 0 | 0 | 2/4 | 0 | 1/4 | 1/4 | **Question. Using the fully tagged four-sentence corpus above, calculate the joint score of “will jane spot mary” assigned tags M,N,V,N, including END.** #### Given Required factors from the tables: START→M=1/4; emit will|M=3/4; M→N=1/4; jane|N=2/9; N→V=1/9; spot|V=1/4; V→N=1; mary|N=4/9; N→END=4/9. #### To find The specified tag-path score, including words and sentence end. #### Formula and why it applies $$ P=\pi_M b_M(will)a_{MN}b_N(jane)a_{NV}b_V(spot)a_{VN}b_N(mary)a_{N,END} $$ Both transitions and emissions are needed. A tag transition alone cannot tell us whether a particular word is plausible. #### Solution ##### Step 1. Start and emit will $$ \frac14\times\frac34=\frac3{16} $$ ##### Step 2. Move to N and emit jane $$ \frac3{16}\times\frac14\times\frac29=\frac1{96} $$ ##### Step 3. Move to V and emit spot $$ \frac1{96}\times\frac19\times\frac14=\frac1{3456} $$ ##### Step 4. Move to N and emit mary $$ \frac1{3456}\times1\times\frac49=\frac1{7776} $$ ##### Step 5. End the sentence $$ \frac1{7776}\times\frac49=\frac1{17496}\approx0.0000571559 $$ **Final answer.** The joint score is 1/17496 ≈ 0.0000571559. **Question. For the same tagged corpus and tables, explain the unsmoothed score of “will can spot mary” assigned tags M,V,N,N.** #### Given The proposed second word is can with hidden tag V. In the training corpus, can occurs only with M, so b\_V(can)=0. #### To find Whether the complete path score is zero and what this means. #### Formula and why it applies $$ P(\text{path,words})=(\text{other required factors})\times b_V(can) $$ A product containing one zero factor is zero even if every other factor is positive. #### Solution ##### Step 1. Find the critical emission $$ b_V(can)=C(can,V)/C(V)=0/4=0 $$ ##### Step 2. Use the full-product consequence $$ P(\text{path,words})=0 $$ ##### Step 3. Interpret correctly This unsmoothed corpus model has no evidence for can as a verb. It does not prove that a verb usage of can is impossible in English. **Final answer.** The unsmoothed path score is zero. --- ## M2 · Why exhaustive search grows exponentially Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/formulas/f-hmm--sequence-count/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/formulas/f-hmm/sequence-count Formulas explained through full worked examples If each of T observed words can receive any of N tags, there are N choices at each position. Thus the total number of complete tag sequences is N multiplied by itself T times. This counts possible assignments before grammar rules or zero probabilities eliminate any. $$ \text{Number of tag sequences}=N^T $$ **Question. A four-word sentence is tagged using three possible POS tags per word. How many complete tag sequences are possible?** #### Given N=3 possible tags per position; T=4 word positions. #### To find The number of tag sequences. #### Formula and why it applies $$ M=N^T $$ Every independent position contributes a factor of three choices. #### Solution ##### Step 1. Write one factor per position $$ M=3\times3\times3\times3 $$ ##### Step 2. Multiply $$ 3\times3=9,\quad9\times3=27,\quad27\times3=81 $$ **Final answer.** 81 possible tag sequences. **Question. A five-word sentence has two candidate tags per word. Count its tag sequences.** #### Given N=2; T=5. #### To find M. #### Formula and why it applies $$ M=N^T $$ The length is the exponent, not a multiplier. #### Solution ##### Step 1. Substitute $$ M=2^5 $$ ##### Step 2. Evaluate $$ 2\times2\times2\times2\times2=32 $$ **Final answer.** 32 possible sequences. **Question. Using 45 possible tags, compare the number of tag sequences for sentences of length 2 and 3.** #### Given N=45. First T=2, then T=3. #### To find Both counts and the growth factor. #### Formula and why it applies $$ M_2=45^2,\quad M_3=45^3 $$ Adding one position multiplies the possibilities by the number of tag choices. #### Solution ##### Step 1. Two words $$ 45^2=2025 $$ ##### Step 2. Three words $$ 45^3=2025\times45=91125 $$ ##### Step 3. Compare $$ 91125/2025=45 $$ **Final answer.** 2 words: 2025 paths. 3 words: 91125 paths. One additional word multiplies the count by 45. The slides introduce Viterbi to avoid enumerating every path. It reuses the best prefix score for each ending state. For a dense first-order HMM, it needs O(TN²) recurrence work instead of enumerating NT complete paths. **Yaad rakho:** Choices per word ki power = sentence length. --- ## M2 · Viterbi decoding: compare incoming paths, keep the best, trace back Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/formulas/f-hmm--viterbi-explained/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/formulas/f-hmm/viterbi-explained Formulas explained through full worked examples **The Viterbi algorithm** finds the most likely hidden-state sequence for a fixed observation sequence. At each position, it keeps the best path ending in each state. This is safe because, under the HMM assumptions, future extensions depend on that ending state, not the discarded details of the prefix. $$ \delta_1(j)=\pi_jb_j(o_1) $$ $$ \delta_t(j)=b_j(o_t)\max_i[\delta_{t-1}(i)a_{ij}] $$ $$ \psi_t(j)=\arg\max_i[\delta_{t-1}(i)a_{ij}] $$ | Symbol | Meaning in ordinary words | | --- | --- | | δₜ(j) | Score of the best prefix that ends in state j at position t. | | ψₜ(j) | Previous state that supplied that best prefix; keep it as a backpointer. | | max | Choose the larger candidate score; do not add. | | argmax | Remember which previous state won, rather than only its value. | Initialize: start × first emission → For each next state: compare incoming scores → Multiply winner by this state's emission → Keep score + winning predecessor → Choose best final state and trace arrows backward Do not choose the best state independently at each word. The best last state can come from a different best predecessor than a local guess. Backpointers recover one consistent complete path. If scores tie, either tied path is optimal under the model; state the tie. **Question. Use an HMM with π(H)=0.8, π(C)=0.2, transitions H:(0.7,0.3), C:(0.4,0.6) in H,C order, and emissions H:(0.2,0.4,0.4), C:(0.5,0.4,0.1) for observations 1,2,3. Find the most likely hidden path for observations 1,3,1 using Viterbi.** #### Given Initial probabilities: π(H)=0.8, π(C)=0.2. Transition matrix: H→H=0.7, H→C=0.3, C→H=0.4, C→C=0.6. Emissions for labels (1,2,3): H=(0.2,0.4,0.4), C=(0.5,0.4,0.1). There is no END-state factor. #### To find The best hidden-state sequence and its joint score. #### Formula and why it applies $$ \delta_1(j)=\pi_jb_j(o_1),\quad\delta_t(j)=b_j(o_t)\max_i[\delta_{t-1}(i)a_{ij}] $$ Viterbi keeps two best prefix scores per column, one ending H and one ending C. #### Solution ##### Step 1. Initialize at observation 1 $$ \delta_1(H)=0.8(0.2)=0.16 $$ $$ \delta_1(C)=0.2(0.5)=0.10 $$ ##### Step 2. At observation 3, calculate the H cell $$ H\to H:\ 0.16(0.7)=0.112 $$ $$ C\to H:\ 0.10(0.4)=0.040 $$ $$ \delta_2(H)=\max(0.112,0.040)(0.4)=0.0448 $$ Backpointer for this H cell: H. ##### Step 3. At observation 3, calculate the C cell $$ H\to C:\ 0.16(0.3)=0.048 $$ $$ C\to C:\ 0.10(0.6)=0.060 $$ $$ \delta_2(C)=\max(0.048,0.060)(0.1)=0.006 $$ Backpointer for this C cell: C. ##### Step 4. At final observation 1, calculate H $$ H\to H:\ 0.0448(0.7)=0.03136 $$ $$ C\to H:\ 0.006(0.4)=0.0024 $$ $$ \delta_3(H)=0.03136(0.2)=0.006272 $$ Backpointer: H. ##### Step 5. At final observation 1, calculate C $$ H\to C:\ 0.0448(0.3)=0.01344 $$ $$ C\to C:\ 0.006(0.6)=0.0036 $$ $$ \delta_3(C)=0.01344(0.5)=0.00672 $$ Backpointer: H. ##### Step 6. Select the final state 0.00672 > 0.006272, so the best final state is C. ##### Step 7. Trace back Final C came from H at position2. That H came from H at position1. Reading forward gives H,H,C. ##### Step 8. Verify the recovered complete path $$ 0.8(0.2)(0.7)(0.4)(0.3)(0.5)=0.00672 $$ **Final answer.** Best path H→H→C with joint score 0.00672. **Question. For π(H)=0.8, π(C)=0.2; transitions H:(0.7,0.3), C:(0.4,0.6); emissions H:(0.2,0.4,0.4), C:(0.5,0.4,0.1) for labels1,2,3, decode observations 3,1.** #### Given Initial probabilities: π(H)=0.8, π(C)=0.2. Transition matrix: H→H=0.7, H→C=0.3, C→H=0.4, C→C=0.6. Emissions for labels (1,2,3): H=(0.2,0.4,0.4), C=(0.5,0.4,0.1). There is no END-state factor. #### To find Best two-state hidden path and score. #### Formula and why it applies $$ \delta_1(j)=\pi_jb_j(3),\quad\delta_2(j)=b_j(1)\max_i[\delta_1(i)a_{ij}] $$ The same recurrence applies even though the observation order changed. #### Solution ##### Step 1. Initialize observation 3 $$ \delta_1(H)=0.8(0.4)=0.32,\quad\delta_1(C)=0.2(0.1)=0.02 $$ ##### Step 2. Calculate the final H $$ \max(0.32(0.7),0.02(0.4))(0.2) $$ $$ =\max(0.224,0.008)(0.2)=0.0448 $$ Winning predecessor H. ##### Step 3. Calculate the final C $$ \max(0.32(0.3),0.02(0.6))(0.5) $$ $$ =\max(0.096,0.012)(0.5)=0.048 $$ Winning predecessor H. ##### Step 4. Select and trace Final C wins with 0.048 and came from H. The path is H,C. **Final answer.** Best path H→C, score 0.048. **Question. For π(H)=0.8, π(C)=0.2; transitions H:(0.7,0.3), C:(0.4,0.6); emissions H:(0.2,0.4,0.4), C:(0.5,0.4,0.1) for labels1,2,3, decode observations 2,2.** #### Given Initial probabilities: π(H)=0.8, π(C)=0.2. Transition matrix: H→H=0.7, H→C=0.3, C→H=0.4, C→C=0.6. Emissions for labels (1,2,3): H=(0.2,0.4,0.4), C=(0.5,0.4,0.1). There is no END-state factor. #### To find Best path and score. #### Formula and why it applies $$ \delta_1(j)=\pi_jb_j(2),\quad\delta_2(j)=b_j(2)\max_i[\delta_1(i)a_{ij}] $$ Both states emit2 with probability 0.4, but their initial and transition probabilities still differ. #### Solution ##### Step 1. Initialize $$ \delta_1(H)=0.8(0.4)=0.32,\quad\delta_1(C)=0.2(0.4)=0.08 $$ ##### Step 2. Final H $$ \max(0.32(0.7),0.08(0.4))(0.4) $$ $$ =\max(0.224,0.032)(0.4)=0.0896 $$ Predecessor H. ##### Step 3. Final C $$ \max(0.32(0.3),0.08(0.6))(0.4) $$ $$ =\max(0.096,0.048)(0.4)=0.0384 $$ Predecessor H. ##### Step 4. Select and trace The final H score 0.0896 is larger. Its predecessor is H. **Final answer.** Best path H→H with score 0.0896. **Yaad rakho:** Har destination ke liye incoming paths compare; winner ka arrow yaad. --- ## M2 · A path score is not the probability of the observed sentence Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/formulas/f-hmm--joint-vs-posterior/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/formulas/f-hmm/joint-vs-posterior Formulas explained through full worked examples **Question. An HMM has π(H)=0.8, π(C)=0.2; transitions H:(0.7,0.3), C:(0.4,0.6); emissions H:(0.2,0.4,0.4), C:(0.5,0.4,0.1) for labels1,2,3. For observations 1,3, calculate the probability of the observations by summing all four two-state paths.** #### Given Initial probabilities: π(H)=0.8, π(C)=0.2. Transition matrix: H→H=0.7, H→C=0.3, C→H=0.4, C→C=0.6. Emissions for labels (1,2,3): H=(0.2,0.4,0.4), C=(0.5,0.4,0.1). There is no END-state factor. #### To find P(observations 1,3), not just the best-path score. #### Formula and why it applies $$ P(o_1,o_2)=\sum_{s_1,s_2}\pi_{s_1}b_{s_1}(o_1)a_{s_1s_2}b_{s_2}(o_2) $$ The hidden path is unspecified, so its mutually exclusive alternatives must be summed. #### Solution ##### Step 1. Score H,H $$ 0.8(0.2)(0.7)(0.4)=0.0448 $$ ##### Step 2. Score H,C $$ 0.8(0.2)(0.3)(0.1)=0.0048 $$ ##### Step 3. Score C,H $$ 0.2(0.5)(0.4)(0.4)=0.016 $$ ##### Step 4. Score C,C $$ 0.2(0.5)(0.6)(0.1)=0.006 $$ ##### Step 5. Add all paths $$ P(1,3)=0.0448+0.0048+0.016+0.006=0.0716 $$ **Final answer.** Observation probability =0.0716. The best single-path score 0.0448 is a different quantity. If asked for the posterior of path H,H given observations 1,3, normalize: 0.0448/0.0716 ≈0.625698. Viterbi chooses the best path using joint scores because the observation denominator is common to every candidate. It does not compute that denominator by taking a maximum. --- ## M2 · Lesk algorithm, from words to overlap scores Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/formulas/f-wsd--u5-lesk/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/formulas/f-wsd/u5-lesk Formulas explained through full worked examples **Lesk is a knowledge-based WSD method that chooses senses whose dictionary descriptions have the greatest overlap with contextual evidence.** Original Lesk compares definitions of neighbouring words' senses. A common simplified version compares each target-sense gloss directly with the sentence context. State which version you use. ### Simplified Lesk procedure 1. Remove the target word from the context; tokenize and normalize the remaining content words. 2. Obtain the gloss for each candidate sense and apply the same normalization. 3. Count shared words between the context and each gloss. 4. Select the sense with maximum overlap. If scores tie, report the tie or use a specified tie-break rule. $$ \operatorname{score}(s)=|C\cap G_s|,\qquad s^*=\arg\max_s\operatorname{score}(s) $$ $$ C $$ is the set of context words, $$ G_s $$ the set of words in sense $$ s $$ 's gloss, $$ \cap $$ means common elements, $$ |\cdot| $$ means count, and $$ \arg\max $$ means “the sense giving the largest score”. This version counts unique matching words once. **Question. Use simplified Lesk to disambiguate “bank” in “The river water rose near the bank.” Use C = {river, water, rose, near}; financial gloss = {institution, money, deposit}; river gloss = {land, river, water, edge}.** #### Given The normalized context and two gloss sets are given. Count exact shared words. #### To find The score of each sense and the selected meaning. #### Formula and why it applies Use $$ \operatorname{score}(s)=|C\cap G_s| $$ because the task specifies context–gloss overlap. #### Step 1. Compare the financial sense No word occurs in both {river, water, rose, near} and {institution, money, deposit}. Overlap = {}. #### Step 2. Count the financial score $$ \operatorname{score}(F)=0 $$ . #### Step 3. Compare the river sense Shared words are {river, water}. “Rose” and “near” are not in the gloss. #### Step 4. Count and select $$ \operatorname{score}(R)=2 $$ . Since $$ 2>0 $$ , choose the river-edge sense. **Answer.** Bank means land beside a river. **Question. Use simplified Lesk for “The dog barked loudly.” Treat barked as bark. Context C = {dog, loudly}; sound gloss = {sharp, cry, dog}; tree gloss = {outer, covering, tree}; ship gloss = {sailing, ship}.** #### Given Three glosses and normalized context {dog, loudly}. #### To find Select the sense of bark. #### Formula and why it applies $$ \operatorname{score}(s)=|C\cap G_s| $$ . Lemmatization lets barked match the target lemma bark; it does not create additional overlap words. #### Step 1. Sound sense Intersection = {dog}; score = 1. #### Step 2. Tree sense Intersection = {}; score = 0. #### Step 3. Ship sense Intersection = {}; score = 0. #### Step 4. Compare The sound sense has the unique maximum: 1 > 0. **Answer.** Bark is the dog's cry. **Question. Apply definition–definition Lesk to “pine cone”. Pine senses: P1 = {evergreen, tree, needle, leaf}; P2 = {waste, sorrow, illness}. Cone senses: C1 = {solid, shape, circular}; C2 = {fruit, evergreen, tree}.** #### Given Two candidate senses for each word. Tokens are already normalized; tree and trees would be reduced to the same form if necessary. #### To find The most compatible pair of senses. #### Formula and why it applies Use $$ |G_{P_i}\cap G_{C_j}| $$ for every pair because this is original-style gloss–gloss comparison. #### Step 1. P1 with C1 No shared word → 0. #### Step 2. P1 with C2 Shared {evergreen, tree} → 2. #### Step 3. P2 with each cone sense P2–C1 has no overlap → 0; P2–C2 has no overlap → 0. #### Step 4. Choose the best pair The four scores are 0, 2, 0, 0. Maximum 2 belongs to P1–C2. **Answer.** Pine is the evergreen tree; cone is its fruit-like reproductive structure. **Limitations:** exact overlap misses synonyms, short glosses may produce zeros, and different preprocessing choices change the score. Extended Lesk may include related synset definitions or longer phrase overlaps; do not silently substitute it for a simple question. Lesk: context aur dictionary mein common words gino; sabse bada overlap jeeta. --- ## M2 · Thesaurus / Walker approach, with three complete examples Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/formulas/f-wsd--u5-walker/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/formulas/f-wsd/u5-walker Formulas explained through full worked examples **The Walker approach uses thesaurus categories as semantic evidence.** Associate each candidate sense with a category. Count or weight context words that support that category. Choose the largest total. $$ \operatorname{score}(s)=\sum_{w\in C} I(w,\operatorname{category}(s)) $$ $$ I=1 $$ if the supplied category mapping says the context word supports the sense, otherwise $$ I=0 $$ . The summation symbol means add all those contributions. Weighted questions may replace 1 with a supplied weight. Category membership must be supplied or justified; it is not guessed from word frequency. **Question. Disambiguate bank using Walker scoring. Context words are money, interest, annum, fetch. The finance category supports money, interest and annum; the location category supports none of these words.** #### Given Two categories: Finance and Location. Each supported word contributes 1. #### To find The two totals and selected bank sense. #### Formula and why it applies Sum the category indicators for each sense. #### Step 1. Make a contribution table | Word | Finance | Location | | --- | --- | --- | | money | 1 | 0 | | interest | 1 | 0 | | annum | 1 | 0 | | fetch | 0 | 0 | #### Step 2. Add Finance $$ 1+1+1+0=3 $$ . #### Step 3. Add Location $$ 0+0+0+0=0 $$ . #### Step 4. Select 3 > 0, so select the financial institution. **Answer.** Finance, with score 3 versus 0. **Question. Use Walker scoring for “The fisherman caught bass beside the lake.” Context {fisherman,caught,lake}. The Fish category contains all three clues; the Music category contains none.** #### Given Fish and Music senses, binary category membership given. #### To find The correct bass sense. #### Formula and why it applies Score = sum of context-category matches. #### Step 1. Record Fish matches fisherman → 1; caught → 1; lake → 1. #### Step 2. Add Fish score $$ 1+1+1=3 $$ . #### Step 3. Add Music score $$ 0+0+0=0 $$ . #### Step 4. Choose Fish has the larger score. **Answer.** Bass refers to a fish. **Question. Score “The player used a bat in the match.” Context {player,match}. Sport supports both words with weights 2 and 1; Animal supports neither. Use the supplied weighted Walker scheme.** #### Given Sport weights {player:2, match:1}; Animal weights {player:0, match:0}. #### To find Weighted sense scores. #### Formula and why it applies Use the sum of supplied weights, not an unweighted count, because the question explicitly gives weights. #### Step 1. Sport contribution of player Add 2. #### Step 2. Sport contribution of match Add 1. Sport total = 2 + 1 = 3. #### Step 3. Animal total 0 + 0 = 0. #### Step 4. Compare 3 > 0 selects the sporting implement. **Answer.** Bat is the sports implement. Walker: dictionary ke exact words nahi, context kis topic/category ko support karta hai woh count. --- ## M2 · Multinomial Naive Bayes: the exact solving method Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/formulas/f-wsd--u5-bayes-steps/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/formulas/f-wsd/u5-bayes-steps Formulas explained through full worked examples Naive Bayes selects the sense with the largest prior × likelihood product. The context evidence denominator is identical across candidate senses, so it can be omitted for comparison. $$ \operatorname{score}(s)=P(s)\prod_j P(w_j\mid s). $$ For add-one multinomial smoothing: $$ P(w\mid s)=\frac{C(w,s)+1}{N_s+V}. $$ $$ C(w,s) $$ counts occurrences of word $$ w $$ in training contexts of sense $$ s $$ ; $$ N_s $$ is the total number of word occurrences in those contexts; $$ V $$ is the shared vocabulary size. It is not the number of training sentences. Repeated query words contribute repeated factors. ### Always keep these counts separate Class priors use **number of examples**. Word likelihoods in this multinomial model use **number of word occurrences**. The vocabulary counts **unique words across all classes**. 1. List the class examples and calculate priors. 2. Write one shared vocabulary and count its size. 3. Count total word occurrences in each class. 4. Count each query word in each class. 5. Apply the given smoothing rule. 6. Multiply each prior by all its query likelihoods. 7. Compare scores; divide each score by their sum only if a normalized posterior is requested. $$ P(s\mid C)=\frac{\operatorname{score}(s)}{\sum_{s'}\operatorname{score}(s')}. $$ **Why smoothing is needed:** an unseen clue gives $$ 0/N_s=0 $$ ; one zero makes the whole product zero. Add-one smoothing redistributes probability across every vocabulary word and prevents these zero factors. Use it only when requested, taught for the task, or state the assumption. **Different model warning:** a Bernoulli Naive Bayes question counts presence/absence per document and may include absence factors. Do not use its denominator in a multinomial word-count question. Pehle sentences se prior; phir words se likelihood; dono counts mix mat kar. --- ## M2 · Mam's bank numerical—every count and every multiplication Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/formulas/f-wsd--u5-bayes-mam/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/formulas/f-wsd/u5-bayes-mam Formulas explained through full worked examples **Question. Classify the sense of “bank” using the training table below. The sentence is: “As the stream flowed gently past the shore, we set up camp by the bank of the river, discussing the possibility of borrowing a loan to buy a boat for future trips.” Use only the highlighted feature words stream, shore and loan, and add-one multinomial smoothing.** #### Given | Training row | Context words | Sense | | --- | --- | --- | | 1 | money, deposit, loan | Finance | | 2 | river, water, shore | Riverbank | | 3 | account, savings, investment | Finance | | 4 | boat, river, stream | Riverbank | | 5 | branch, deposit, check | Finance | | 6 | fishing, shore, river | Riverbank | Query feature set: {stream, shore, loan}. Other sentence words are not included as features in this supplied example. #### To find The most probable bank sense and normalized posteriors. #### Formula and why it applies Naive Bayes selects the sense with the largest prior × likelihood product. The context evidence denominator is identical across candidate senses, so it can be omitted for comparison. $$ \operatorname{score}(s)=P(s)\prod_j P(w_j\mid s). $$ For add-one multinomial smoothing: $$ P(w\mid s)=\frac{C(w,s)+1}{N_s+V}. $$ $$ C(w,s) $$ counts occurrences of word $$ w $$ in training contexts of sense $$ s $$ ; $$ N_s $$ is the total number of word occurrences in those contexts; $$ V $$ is the shared vocabulary size. It is not the number of training sentences. Repeated query words contribute repeated factors. #### Step 1. Calculate priors Finance appears in 3 of the 6 rows; Riverbank appears in 3 of 6. $$ P(F)=\frac36=\frac12,\qquad P(R)=\frac36=\frac12. $$ #### Step 2. Construct the vocabulary {money, deposit, loan, river, water, shore, account, savings, investment, boat, stream, fishing, branch, check}. Thus V=14. Repeated deposit or river is counted once in vocabulary. #### Step 3. Count word occurrences per class There are 3 finance rows × 3 words = 9 word occurrences. There are 3 river rows × 3 words = 9. Thus $$ N_F=N_R=9 $$ , and each smoothed denominator is $$ 9+14=23 $$ . #### Step 4. Count the three query words | Word | Finance count | Riverbank count | | --- | --- | --- | | stream | 0 | 1 | | shore | 0 | 2 | | loan | 1 | 0 | #### Step 5. Calculate finance likelihoods $$ P(\text{stream}\mid F)=\frac{0+1}{23}=\frac1{23} $$ $$ P(\text{shore}\mid F)=\frac{0+1}{23}=\frac1{23} $$ $$ P(\text{loan}\mid F)=\frac{1+1}{23}=\frac2{23} $$ #### Step 6. Multiply the finance score $$ \operatorname{score}(F)=\frac12\times\frac1{23}\times\frac1{23}\times\frac2{23}=\frac1{23^3}=\frac1{12167}\approx0.00008219. $$ #### Step 7. Calculate river likelihoods $$ P(\text{stream}\mid R)=\frac{1+1}{23}=\frac2{23} $$ $$ P(\text{shore}\mid R)=\frac{2+1}{23}=\frac3{23} $$ $$ P(\text{loan}\mid R)=\frac{0+1}{23}=\frac1{23} $$ #### Step 8. Multiply the river score $$ \operatorname{score}(R)=\frac12\times\frac2{23}\times\frac3{23}\times\frac1{23}=\frac3{12167}\approx0.00024657. $$ #### Step 9. Compare scores 3/12167 is larger than 1/12167. Riverbank wins even though “loan” is financial evidence: the combined clues matter. #### Step 10. Normalize if a posterior is requested $$ P(F\mid C)=\frac{1/12167}{4/12167}=\frac14=0.25 $$ $$ P(R\mid C)=\frac{3/12167}{4/12167}=\frac34=0.75. $$ **Answer.** Bank is the riverbank sense. Scores are 1/12167 and 3/12167; normalized model posteriors are 0.25 and 0.75. **Without smoothing:** stream and shore are absent in Finance; loan is absent in Riverbank. Both unsmoothed scores become zero, so there is no valid winning class from those products. Also, unnormalized scores should not be labelled as final posterior probabilities. Zero word? Add-one: count+1 upar, total words+vocabulary neeche. Last mein score bada wala sense. --- ## M2 · More WSD numericals: given likelihoods, unequal priors and repeated clues Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/formulas/f-wsd--u5-bayes-practice/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/formulas/f-wsd/u5-bayes-practice Formulas explained through full worked examples **Question. For bass, let P(Fish)=0.6 and P(Music)=0.4. Given Fish likelihoods P(caught|Fish)=0.5, P(lake|Fish)=0.4 and Music likelihoods P(caught|Music)=0.05, P(lake|Music)=0.1, classify context {caught,lake} and normalize.** #### Given Priors and likelihoods are provided directly; no smoothing or training-count estimation is requested. #### To find The winning sense and its posterior. #### Formula and why it applies $$ \operatorname{score}(s)=P(s)P(\text{caught}\mid s)P(\text{lake}\mid s) $$ , because the two features are treated as conditionally independent. #### Step 1. Fish multiplication $$ 0.6\times0.5=0.3 $$ , then $$ 0.3\times0.4=0.12 $$ . #### Step 2. Music multiplication $$ 0.4\times0.05=0.02 $$ , then $$ 0.02\times0.1=0.002 $$ . #### Step 3. Total evidence $$ 0.12+0.002=0.122 $$ . #### Step 4. Normalize $$ P(Fish\mid C)=0.12/0.122\approx0.983607 $$ $$ P(Music\mid C)=0.002/0.122\approx0.016393. $$ **Answer.** Fish, with model posterior approximately 0.983607. **Question. Disambiguate bank from the repeated context tokens {loan, loan, river}. Priors are Finance=2/3 and River=1/3. Vocabulary V=4. Finance has N=8 total tokens, loan count 3 and river count 0; River has N=4, loan count 0 and river count 2. Apply add-one smoothing.** #### Given Multinomial counts are supplied. The two appearances of loan must both contribute. #### To find Scores and normalized Finance posterior. #### Formula and why it applies Naive Bayes selects the sense with the largest prior × likelihood product. The context evidence denominator is identical across candidate senses, so it can be omitted for comparison. $$ \operatorname{score}(s)=P(s)\prod_j P(w_j\mid s). $$ For add-one multinomial smoothing: $$ P(w\mid s)=\frac{C(w,s)+1}{N_s+V}. $$ $$ C(w,s) $$ counts occurrences of word $$ w $$ in training contexts of sense $$ s $$ ; $$ N_s $$ is the total number of word occurrences in those contexts; $$ V $$ is the shared vocabulary size. It is not the number of training sentences. Repeated query words contribute repeated factors. #### Step 1. Finance denominator $$ 8+4=12 $$ . $$ P(loan\mid F)=(3+1)/12=1/3 $$ ; $$ P(river\mid F)=(0+1)/12=1/12 $$ . #### Step 2. River denominator $$ 4+4=8 $$ . $$ P(loan\mid R)=(0+1)/8=1/8 $$ ; $$ P(river\mid R)=(2+1)/8=3/8 $$ . #### Step 3. Finance product $$ \operatorname{score}(F)=\frac23\times\frac13\times\frac13\times\frac1{12}=\frac1{162}\approx0.00617284. $$ #### Step 4. River product $$ \operatorname{score}(R)=\frac13\times\frac18\times\frac18\times\frac38=\frac1{512}\approx0.001953125. $$ #### Step 5. Compare and normalize $$ P(F\mid C)=\frac{1/162}{1/162+1/512}=\frac{512}{674}=\frac{256}{337}\approx0.759644. $$ **Answer.** Finance wins. Repeated loan contributes a squared likelihood; do not reduce the query to a unique-word set. ### Answer-writing order in an exam Write the class priors, vocabulary size, class token totals, a likelihood table, both score products and the final class. Keep fractions until the comparison where possible. If the question only asks classification, normalization is optional; if it asks posterior probabilities, normalization is required. Repeated word aaye toh factor bhi repeat. Sirf word-set bana diya toh multinomial answer badal jayega. --- ## M2 · Q1B · Chunking and chinking Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/papers/final-2025--final-2025-q1b/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/papers/final-2025/final-2025-q1b M2-relevant questions from an actual TEE paper · original marks retained for longer practice Question. Explain the concepts of chunking and chinking in Natural Language Processing (NLP). Discuss how Part-of-Speech (POS) tagging supports these processes, and illustrate your explanation with a suitable example. [5 marks] ### Answer **Chunking** groups adjacent tagged words into shallow phrases, such as noun phrases, without building a complete sentence parse. **Chinking** removes a specified sequence of tags or words from a chunk, splitting a larger candidate chunk when required. ### POS tags provide the pattern A POS tag describes a grammatical category: DT = determiner, JJ = adjective, NN = singular noun, VBD = past-tense verb and IN = preposition. A noun-phrase rule can be written as `NP: {
?*+}`: an optional determiner, zero or more adjectives, followed by one or more nouns. ### Worked example | Word | The | small | cat | sat | on | the | mat | | --- | --- | --- | --- | --- | --- | --- | --- | | Tag | DT | JJ | NN | VBD | IN | DT | NN | Applying the NP rule gives **[The small cat] sat on [the mat]**. The first chunk contains DT–JJ–NN; the second contains DT–NN. “sat on” is not included because it does not match the noun-phrase rule. For chinking, begin with one broad chunk: **[The small cat sat on the mat]**. Remove the VBD and IN sequence “sat on”, for example with `}+{`. The result is **[The small cat] sat on [the mat]**. POS tagging therefore supplies grammatical labels; chunking uses labels to select phrases, while chinking uses labels to exclude unwanted material. Incorrect tags can lead to incorrect chunks. **Yaad rakho:** Chunking = phrase ko jodo; chinking = beech ka unwanted part nikalo. --- ## M2 · Q1C · Hyponymy, hypernymy and WSD Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/papers/final-2025--final-2025-q1c/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/papers/final-2025/final-2025-q1c M2-relevant questions from an actual TEE paper · original marks retained for longer practice Question. Define hyponymy and hypernymy with suitable linguistic examples. How can hypernymy be applied in Word Sense Disambiguation (WSD) to resolve the correct meaning of polysemous words? Provide an example? [5 marks] ### Answer **Hyponymy** is the “is-a-kind-of” relation from a specific concept to a broader concept. **Hypernymy** is the inverse relation from the broader concept to the specific one. “Rose” is a hyponym of “flower”; “flower” is a hypernym of “rose”. rose / tulip → flower → plant The arrow points towards a broader category. Hyponymy is not a part–whole relation: a wheel is part of a car, but it is not a kind of car. ### Using the hierarchy for WSD WSD selects the intended sense of a word from its context. Obtain candidate senses from a lexical resource such as WordNet, inspect their hypernym chains, and favour the chain most compatible with nearby words. 1. Sentence: “The mouse clicked the icon.” 2. Candidate 1: mouse → rodent → mammal → animal. Candidate 2: mouse → pointing device → computer peripheral. 3. The context words “clicked” and “icon” belong to the computer-use domain. They support the device hierarchy, rather than the animal hierarchy. 4. Select the computer-pointing-device sense. In “The mouse ate cheese,” the animal hierarchy is more compatible. Hypernym evidence supplies semantic generalisation; it is a useful signal rather than a guarantee, because short or unusual contexts may still be ambiguous. **Yaad rakho:** Hyponym specific hai; hypernym uska bada category hai. --- ## M2 · Q2B · Laplace-smoothed bigram probability Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/papers/final-2025--final-2025-q2b/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/papers/final-2025/final-2025-q2b M2-relevant questions from an actual TEE paper · original marks retained for longer practice Question. Consider the following training corpus: 1. artists paint murals in city 2. musicians play songs in city 3. painters paint murals on walls 4. signs are near city Using a bigram language model with Laplace smoothing, evaluate the probability of the test sentence: “artists paint murals near city”. Assume the and for each sentence. [10 marks] ### Answer ### Given - artists paint murals in city - musicians play songs in city - painters paint murals on walls - signs are near city Test: artists paint murals near city. Add one start token and one end token to every training and test sentence. ### To find The Laplace-smoothed bigram probability of the complete test sentence, including its end token. ### Formula: which one and why $$ P_{+1}(w_i\mid w_{i-1})=\frac{C(w_{i-1},w_i)+1}{C(w_{i-1})+V} $$ C(previous,current) counts that adjacent pair in the training corpus. C(previous) counts its occurrences as a history. V is the vocabulary size. Add-one smoothing is necessary because an unseen pair must not force the whole sentence probability to zero. $$ P(w_1,\ldots,w_n,\langle/s\rangle\mid\langle s\rangle)=\prod_{i=1}^{n+1}P(w_i\mid w_{i-1}) $$ ### Solution ### Step 1. Fix the token and vocabulary convention Lowercase words and remove the final punctuation. The supplied course key counts both boundary tokens in V. Following that convention: are, artists, city, in, murals, musicians, near, on, paint, painters, play, signs, songs, walls, , gives **V = 16**. The start marker supplies history; it is not an extra probability factor before the first word. ### Step 2. List every required transition The complete test chain is → artists → paint → murals → near → city → . ### Step 3. Count first, then add one | Required bigram | History count | Pair count | Smoothed probability | | --- | --- | --- | --- | | → artists | 4 | 1 | (1+1)/(4+16) = 1/10 | | artists → paint | 1 | 1 | (1+1)/(1+16) = 2/17 | | paint → murals | 2 | 2 | (2+1)/(2+16) = 1/6 | | murals → near | 2 | 0 | (0+1)/(2+16) = 1/18 | | near → city | 1 | 1 | (1+1)/(1+16) = 2/17 | | city → | 3 | 3 | (3+1)/(3+16) = 4/19 | Count only adjacent words, never words that merely appear in the same sentence. The unseen pairs have count zero, so their numerator becomes one. ### Step 4. Multiply all six probabilities $$ P=\frac{2}{20}\times\frac{2}{17}\times\frac{3}{18}\times\frac{1}{18}\times\frac{2}{17}\times\frac{4}{19} $$ Start with subtotal 1. At each row, multiply the previous subtotal by exactly one next factor. Simplify the fraction after multiplication, but do not round it. | Factor being included | Previous subtotal × next factor = new subtotal | | --- | --- | | 1. → artists | $$ 1\times\frac{1}{10}=\frac{1}{10} $$ | | 2. artists → paint | $$ \frac{1}{10}\times\frac{2}{17}=\frac{1}{85} $$ | | 3. paint → murals | $$ \frac{1}{85}\times\frac{1}{6}=\frac{1}{510} $$ | | 4. murals → near | $$ \frac{1}{510}\times\frac{1}{18}=\frac{1}{9180} $$ | | 5. near → city | $$ \frac{1}{9180}\times\frac{2}{17}=\frac{1}{78030} $$ | | 6. city → | $$ \frac{1}{78030}\times\frac{4}{19}=\frac{2}{741285} $$ | The last subtotal is the complete product. Only now convert the fraction to a decimal. $$ P=\frac{2}{741285}\approx 2.69801763\times10^{-6} $$ ### Step 5. Interpret the result This is a joint sentence probability, so a small value is normal. Do not add transition probabilities; each word must occur in sequence, so the chain multiplies them. ### Final answer **P ≈ 2.69801763 × 10-6** with the stated course vocabulary convention. **Yaad rakho:** Pair count mein 1 jodo, previous-word count mein V; phir saare fractions multiply karo. **Convention note:** Some implementations exclude from the predicted vocabulary. Their V is one smaller and their numerical probability differs. In an exam, state V explicitly and follow the convention specified in the question or class solution. --- ## M2 · Q4B · HMM emission, transition and sequence Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/papers/final-2025--final-2025-q4b/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/papers/final-2025/final-2025-q4b M2-relevant questions from an actual TEE paper · original marks retained for longer practice Question. Consider the following POS tagged corpus. Apply Hidden Markov Model (HMM) approach to: 1. Cat/Noun chase/Verb mice/Noun 2. The/Det dog/Noun runs/Verb 3. A/Det cat/Noun eats/Verb cheese/Noun i. Construct the emission probability matrix. ii. Construct the transmission probability matrix. iii. Calculate the probability of sequence of events: The/Det cat/Noun eats/Verb mice/Noun. Note: Include start and end characters for all the sentences. [10 marks] ### Answer ### Given | Sentence | Tagged sequence | | --- | --- | | 1 | cat/Noun chase/Verb mice/Noun | | 2 | the/Det dog/Noun runs/Verb | | 3 | a/Det cat/Noun eats/Verb cheese/Noun | Test words: the cat eats mice. Given tag sequence: Det → Noun → Verb → Noun. Include Start and End for all sentences. Lowercase the corpus so sentence-initial Cat and cat are one word. ### To find Emission probabilities, transition probabilities and the joint probability of the specified tag-and-word sequence. ### Formula: which one and why $$ P(w\mid t)=\frac{C(t,w)}{C(t)} $$ An emission probability asks: among tokens carrying this tag t, what fraction are the word w? $$ P(t_j\mid t_i)=\frac{C(t_i,t_j)}{\sum_k C(t_i,t_k)} $$ A transition probability asks: among transitions leaving tag tᵢ, what fraction go to tⱼ? Include End in the outgoing counts. The paper says “transmission”; the standard HMM term is transition. $$ P(\mathbf w,\mathbf t)=P(t_1\mid\mathrm{Start})\prod_{i=1}^{n}P(w_i\mid t_i)\prod_{i=2}^{n}P(t_i\mid t_{i-1})P(\mathrm{End}\mid t_n) $$ ### Solution ### Step 1. Count tokens for each tag Det has 2 tokens; Noun has 5 tokens; Verb has 3 tokens. The repeated noun cat occurs twice with Noun; every other displayed noun occurs once. The three verb types each occur once. ### Step 2. Build the emission matrix | Tag / word | a | cat | chase | cheese | dog | eats | mice | runs | the | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | Det | 1/2 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1/2 | | Noun | 0 | 2/5 | 0 | 1/5 | 1/5 | 0 | 1/5 | 0 | 0 | | Verb | 0 | 0 | 1/3 | 0 | 0 | 1/3 | 0 | 1/3 | 0 | Each row sums to one. A zero means no occurrence under that tag in this unsmoothed training corpus; no add-one smoothing was requested. ### Step 3. Write the three tag chains Start → Noun → Verb → Noun → End Start → Det → Noun → Verb → End Start → Det → Noun → Verb → Noun → End ### Step 4. Count outgoing transitions | From | To Det | To Noun | To Verb | To End | Outgoing total | | --- | --- | --- | --- | --- | --- | | Start | 2 | 1 | 0 | 0 | 3 | | Det | 0 | 2 | 0 | 0 | 2 | | Noun | 0 | 0 | 3 | 2 | 5 | | Verb | 0 | 2 | 0 | 1 | 3 | ### Step 5. Divide each row by its outgoing total | From / To | Det | Noun | Verb | End | | --- | --- | --- | --- | --- | | Start | 2/3 | 1/3 | 0 | 0 | | Det | 0 | 1 | 0 | 0 | | Noun | 0 | 0 | 3/5 | 2/5 | | Verb | 0 | 2/3 | 0 | 1/3 | End is terminal, so it has no estimated outgoing row. The four nonterminal rows each sum to one. ### Step 6. Choose the factors for the given sequence | Event in order | Probability | | --- | --- | | Start → Det | 2/3 | | Det emits the | 1/2 | | Det → Noun | 1 | | Noun emits cat | 2/5 | | Noun → Verb | 3/5 | | Verb emits eats | 1/3 | | Verb → Noun | 2/3 | | Noun emits mice | 1/5 | | Noun → End | 2/5 | ### Step 7. Multiply, including the end transition $$ P=\frac23\times\frac12\times1\times\frac25\times\frac35\times\frac13\times\frac23\times\frac15\times\frac25 $$ Start with subtotal 1. At each row, multiply the previous subtotal by exactly one next factor. Simplify the fraction after multiplication, but do not round it. | Factor being included | Previous subtotal × next factor = new subtotal | | --- | --- | | 1. Start → Det | $$ 1\times\frac{2}{3}=\frac{2}{3} $$ | | 2. Det emits the | $$ \frac{2}{3}\times\frac{1}{2}=\frac{1}{3} $$ | | 3. Det → Noun | $$ \frac{1}{3}\times1=\frac{1}{3} $$ | | 4. Noun emits cat | $$ \frac{1}{3}\times\frac{2}{5}=\frac{2}{15} $$ | | 5. Noun → Verb | $$ \frac{2}{15}\times\frac{3}{5}=\frac{2}{25} $$ | | 6. Verb emits eats | $$ \frac{2}{25}\times\frac{1}{3}=\frac{2}{75} $$ | | 7. Verb → Noun | $$ \frac{2}{75}\times\frac{2}{3}=\frac{4}{225} $$ | | 8. Noun emits mice | $$ \frac{4}{225}\times\frac{1}{5}=\frac{4}{1125} $$ | | 9. Noun → End | $$ \frac{4}{1125}\times\frac{2}{5}=\frac{8}{5625} $$ | The last subtotal is the complete product. Only now convert the fraction to a decimal. $$ P=\frac8{5625}\approx0.0014222222 $$ ### Final answer **Joint sequence probability = 8/5625 ≈ 0.0014222222.** This is the probability of the specified words and tags together. It is not the probability of the word sequence summed over every possible tag path. **Yaad rakho:** Tag se tag = transition; tag se word = emission. Dono alternate multiply, end ko mat bhoolo. **Boundary reminder:** The initial factor is P(Det | Start) = 2/3, based on three sentence starts, and the final factor is P(End | Noun) = 2/5. The overall fraction of Det tokens, 2/10, is not the start probability. Using it and omitting End, as in the handwritten key’s last line, computes a different quantity. --- ## M2 · Q5B · Naïve Bayes word-sense classification Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/papers/final-2025--final-2025-q5b/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/papers/final-2025/final-2025-q5b M2-relevant questions from an actual TEE paper · original marks retained for longer practice Question. Consider the sentence: “The bat hit the ball hard on the cricket field.” Two possible senses of “bat”: Sense 1: Bat (Sports equipment – cricket/baseball); Sense 2: Bat (Nocturnal flying mammal). | Sentence ID | Context words | Sense | | --- | --- | --- | | 1 | ball, cricket, score, stadium | Sports | | 2 | night, cave, wings, fly | Animal | | 3 | player, run, match, bat | Sports | | 4 | nocturnal, vampire, insect | Animal | | 5 | team, field, bowl, bat | Sports | | 6 | dark, flying, cave, night | Animal | a) Using the Naïve Bayes algorithm, classify the sense of “bat” in the given sentence. Apply Laplace Smoothing wherever applicable. b) Discuss the significance of Laplace Smoothing in Naïve Bayes Classification. [10 marks] ### Answer ### Given There are three Sports examples and three Animal examples. Treat training context lists as bag-of-words token counts. Exclude the ambiguous target “bat” from the test context. Use the fixed training vocabulary; ignore out-of-vocabulary test words hit and hard. The usable test features are ball, cricket and field. ### To find The posterior-score comparison for Sports and Animal, and the role of smoothing. ### Formula: which one and why $$ \mathrm{score}(s)=P(s)\prod_{w\in\mathrm{context}}P(w\mid s) $$ $$ P(w\mid s)=\frac{C(w,s)+1}{N_s+V} $$ P(s) is the class prior, C(w,s) counts occurrences of context word w in class s, Nₛ is the total number of context tokens in that class and V is the shared vocabulary size. Conditional independence lets us multiply word probabilities. The common denominator P(context) is unnecessary when only comparing classes. ### Solution ### Step 1. Calculate the priors $$ P(S)=\frac36=0.5,\qquad P(A)=\frac36=0.5 $$ ### Step 2. Count class tokens and vocabulary Sports has 4 + 4 + 4 = 12 tokens. Animal has 4 + 3 + 4 = 11 tokens. The shared training vocabulary has 20 distinct words: ball, cricket, score, stadium, player, run, match, bat, team, field, bowl, night, cave, wings, fly, nocturnal, vampire, insect, dark, flying. “bat”, “night” and “cave” repeat but count only once each towards V. ### Step 3. Look up each usable context word | Feature | Sports count | Animal count | P(word | Sports) | P(word | Animal) | | --- | --- | --- | --- | --- | | ball | 1 | 0 | (1+1)/(12+20) = 2/32 | (0+1)/(11+20) = 1/31 | | cricket | 1 | 0 | 2/32 | 1/31 | | field | 1 | 0 | 2/32 | 1/31 | ### Step 4. Multiply the Sports score $$ \mathrm{score}(S)=0.5\times\frac2{32}\times\frac2{32}\times\frac2{32}=0.0001220703125 $$ ### Step 5. Multiply the Animal score $$ \mathrm{score}(A)=0.5\times\frac1{31}\times\frac1{31}\times\frac1{31}\approx0.00001678359236 $$ ### Step 6. Compare the scores 0.0001220703125 is greater than 0.00001678359236. Therefore select Sports. Optional normalization gives P(Sports | retained features) ≈ 0.87913; normalization is not needed for the decision. ### Final answer The intended sense is **bat as sports equipment**. Ball, cricket and field favour that sense. **Yaad rakho:** Pehle class prior, phir har context-word likelihood multiply; bigger score wali sense choose karo. ### Why Laplace smoothing matters Without smoothing, all three test features have zero count in Animal. One zero factor makes the entire class score zero, even if other evidence could support that class. Add-one smoothing assigns nonzero probability to every word in the defined vocabulary and redistributes probability mass. It does not mean every unseen word outside the vocabulary can be inserted without changing the vocabulary model. ### If following the supplied paper key exactly The key also multiplies probabilities for hit and hard using the same denominators 32 and 31. That convention produces Sports = 1.19209290 × 10⁻⁷ and Animal ≈ 1.74647163 × 10⁻⁸, again selecting Sports. Its treatment of out-of-vocabulary words is not a normalized fixed-vocabulary model. State your feature convention clearly; the complete calculation above uses a consistent fixed training vocabulary. --- ## M2 · Q1B · Content and function words Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/papers/reexam-2026--reexam-2026-q1b/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/papers/reexam-2026/reexam-2026-q1b M2-relevant questions from an actual TEE paper · original marks retained for longer practice Question. i. Define function words and content words. ii. Which category (function words and content words) belong to open class and closed class? Justify your answer. iii. For the sentence “The small cat is sleeping under the tree”, identify the function words and content words. [5 marks] ### Answer **Content words** carry much of a sentence’s lexical meaning, usually including nouns, main verbs, adjectives and many adverbs. **Function words** primarily express grammatical relationships, including determiners, prepositions, conjunctions, pronouns and auxiliary verbs. | Category | Typical class | Reason | | --- | --- | --- | | Content words | Open class | New nouns, verbs and adjectives enter the language readily, such as new technology names | | Function words | Closed class | The grammatical inventory is relatively stable; new determiners or pronouns are much less frequently added | | Word | Role in this sentence | Category | | --- | --- | --- | | The | Determiner | Function | | small | Adjective | Content | | cat | Noun | Content | | is | Auxiliary supporting progressive tense | Function | | sleeping | Main verb | Content | | under | Preposition | Function | | the | Determiner | Function | | tree | Noun | Content | **Final answer:** content words = small, cat, sleeping, tree; function words = The, is, under, the. “is” is auxiliary here; a word’s role must be judged in context. “Closed” means relatively resistant to new members, not that change is logically impossible. **Yaad rakho:** Content words scene batate hain; function words grammar ka connection banate hain. --- ## M2 · Q2B · Laplace smoothing on a second corpus Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/papers/reexam-2026--reexam-2026-q2b/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/papers/reexam-2026/reexam-2026-q2b M2-relevant questions from an actual TEE paper · original marks retained for longer practice Question. Consider the following training corpus: 1. cats chase mice in the garden 2. dogs bark loudly at night 3. birds sing in the morning 4. children play games in the park Using a bigram language model with Laplace smoothing, evaluate the probability of the test sentence: “cats bark in the park.” Assume and as start and end of the sentence. [10 marks] ### Answer ### Given - cats chase mice in the garden - dogs bark loudly at night - birds sing in the morning - children play games in the park Test: cats bark in the park. Add one start token and one end token to every training and test sentence. ### To find The Laplace-smoothed bigram probability of the complete test sentence, including its end token. ### Formula: which one and why $$ P_{+1}(w_i\mid w_{i-1})=\frac{C(w_{i-1},w_i)+1}{C(w_{i-1})+V} $$ C(previous,current) counts that adjacent pair in the training corpus. C(previous) counts its occurrences as a history. V is the vocabulary size. Add-one smoothing is necessary because an unseen pair must not force the whole sentence probability to zero. $$ P(w_1,\ldots,w_n,\langle/s\rangle\mid\langle s\rangle)=\prod_{i=1}^{n+1}P(w_i\mid w_{i-1}) $$ ### Solution ### Step 1. Fix the token and vocabulary convention Lowercase words and remove the final punctuation. The supplied course key counts both boundary tokens in V. Following that convention: at, bark, birds, cats, chase, children, dogs, games, garden, in, loudly, mice, morning, night, park, play, sing, the, , gives **V = 20**. The start marker supplies history; it is not an extra probability factor before the first word. ### Step 2. List every required transition The complete test chain is → cats → bark → in → the → park → . ### Step 3. Count first, then add one | Required bigram | History count | Pair count | Smoothed probability | | --- | --- | --- | --- | | → cats | 4 | 1 | (1+1)/(4+20) = 1/12 | | cats → bark | 1 | 0 | (0+1)/(1+20) = 1/21 | | bark → in | 1 | 0 | (0+1)/(1+20) = 1/21 | | in → the | 3 | 3 | (3+1)/(3+20) = 4/23 | | the → park | 3 | 1 | (1+1)/(3+20) = 2/23 | | park → | 1 | 1 | (1+1)/(1+20) = 2/21 | Count only adjacent words, never words that merely appear in the same sentence. The unseen pairs have count zero, so their numerator becomes one. ### Step 4. Multiply all six probabilities $$ P=\frac{2}{24}\times\frac{1}{21}\times\frac{1}{21}\times\frac{4}{23}\times\frac{2}{23}\times\frac{2}{21} $$ Start with subtotal 1. At each row, multiply the previous subtotal by exactly one next factor. Simplify the fraction after multiplication, but do not round it. | Factor being included | Previous subtotal × next factor = new subtotal | | --- | --- | | 1. → cats | $$ 1\times\frac{1}{12}=\frac{1}{12} $$ | | 2. cats → bark | $$ \frac{1}{12}\times\frac{1}{21}=\frac{1}{252} $$ | | 3. bark → in | $$ \frac{1}{252}\times\frac{1}{21}=\frac{1}{5292} $$ | | 4. in → the | $$ \frac{1}{5292}\times\frac{4}{23}=\frac{1}{30429} $$ | | 5. the → park | $$ \frac{1}{30429}\times\frac{2}{23}=\frac{2}{699867} $$ | | 6. park → | $$ \frac{2}{699867}\times\frac{2}{21}=\frac{4}{14697207} $$ | The last subtotal is the complete product. Only now convert the fraction to a decimal. $$ P=\frac{4}{14697207}\approx 2.72160554\times10^{-7} $$ ### Step 5. Interpret the result This is a joint sentence probability, so a small value is normal. Do not add transition probabilities; each word must occur in sequence, so the chain multiplies them. ### Final answer **P ≈ 2.72160554 × 10-7** with the stated course vocabulary convention. **Yaad rakho:** Pair count mein 1 jodo, previous-word count mein V; phir saare fractions multiply karo. **Convention note:** Some implementations exclude from the predicted vocabulary. Their V is one smaller and their numerical probability differs. In an exam, state V explicitly and follow the convention specified in the question or class solution. --- ## M2 · Q3A · Top-down parsing Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/papers/reexam-2026--reexam-2026-q3a/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/papers/reexam-2026/reexam-2026-q3a M2-relevant questions from an actual TEE paper · original marks retained for longer practice Question. In the context of natural language processing, explain the concept of top-down and bottom-up parsing approaches. Apply top-down syntax parsing approach to the sentence “The tall boy saw a girl in the park.” and draw a parse tree using the CFG grammar rules: S → NP VP; NP → Det Adj N | Det N; VP → V NP | VP PP; PP → P NP. Lexicon: Det → {the | a}; Adj → {tall}; N → {boy | girl | park}; V → {saw}; P → {in}. [10 marks] ### Answer | Aspect | Top-down | Bottom-up | | --- | --- | --- | | Start | The start symbol S | The input words | | Operation | Expand a nonterminal using grammar productions | Combine words and constituents using productions in reverse | | Success | Leaves match the complete input | All input is reduced to S | | Possible difficulty | Naive recursive descent can loop on left recursion or need backtracking | A naive shift-reduce parser may face shift/reduce conflicts | ### Given The sentence contains the tokens the / tall / boy / saw / a / girl / in / the / park. Lowercase the initial “The” to match the lexicon. The full stop is punctuation outside the listed grammar. ### To find A derivation beginning at S and a parse tree matching the whole sentence. ### Rule: which one and why A context-free production replaces one nonterminal with its right-hand sequence. A top-down derivation begins at S and expands the desired structure. The supplied grammar attaches the PP through VP → VP PP; it does not provide NP → NP PP. ### Step 1. Expand the sentence S ⇒ NP VP. Choose NP → Det Adj N for “the tall boy”. ### Step 2. Match the subject words Det ⇒ the, Adj ⇒ tall, N ⇒ boy. The remaining material must be generated by VP. ### Step 3. Allow a verb phrase followed by a PP Expand the outer VP using VP → VP PP. Then expand the inner VP using VP → V NP. This finite derivation selects the recursive rule once. ### Step 4. Match the verb and object V ⇒ saw. Choose NP → Det N, then Det ⇒ a and N ⇒ girl. ### Step 5. Expand the prepositional phrase PP ⇒ P NP. P ⇒ in. NP ⇒ Det N, with Det ⇒ the and N ⇒ park. ### Step 6. Check the complete leaf order The leaves read “the tall boy saw a girl in the park”, exactly the normalized input. The PP “in the park” attaches to the VP under this supplied grammar. SNPVPDetAdjNVPPPVNPPNPDetNDetNThetallboysawagirlinthepark Top-down parse tree showing the PP attached to VP. ### Complete derivation S ⇒ NP VP ⇒ Det Adj N VP ⇒ the tall boy VP ⇒ the tall boy VP PP ⇒ the tall boy V NP PP ⇒ the tall boy saw Det N PP ⇒ the tall boy saw a girl P NP ⇒ the tall boy saw a girl in Det N ⇒ the tall boy saw a girl in the park. **Implementation caution:** VP → VP PP is left-recursive. A naive recursive-descent implementation may loop; it must transform or otherwise handle the grammar. A valid finite top-down derivation, as shown above, still exists. **Yaad rakho:** Rule jo diya hai wahi use karo: yahan “in the park” VP ka part hai. --- ## M2 · Q3B · Lesk word-sense disambiguation Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/papers/reexam-2026--reexam-2026-q3b/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/papers/reexam-2026/reexam-2026-q3b M2-relevant questions from an actual TEE paper · original marks retained for longer practice Question. Consider the sentence: “The bank can ensure your deposits are safe.” The word “bank” has the following two possible senses (WordNet-style glosses): Bank₁: Sloping land especially beside a body of water. Bank₂: A financial institution that accepts deposits and channels the money into lending activities. i. Apply the LESK algorithm to determine the correct sense of the word “bank” in the given sentence. Show the steps clearly. ii. Briefly discuss limitations of the LESK algorithm in handling ambiguous words in real-world contexts. [10 marks] ### Answer ### Given Context sentence and the two supplied dictionary glosses. Use a simplified Lesk method: lowercase, remove function words, exclude the target bank and compare distinct content-word overlap. Do not add synonyms that the question does not supply. ### To find Which candidate gloss has the greatest overlap with the local context. ### Formula: which one and why $$ \mathrm{score}(s)=|C\cap G_s|,\qquad \hat s=\arg\max_s\mathrm{score}(s) $$ C is the set of context content words. Gₛ is the content-word set of the gloss for sense s. The intersection contains words appearing in both; its size is the score. This is an overlap count, not a probability. ### Solution ### Step 1. Prepare the context Remove The, can, your, are and the target bank. A reasonable content set is C = {ensure, deposits, safe}. If “can” is retained under another fixed stop-list it matches neither gloss and does not change the result. ### Step 2. Prepare Bank₁’s gloss G₁ = {sloping, land, especially, beside, body, water}. There is no shared word with {ensure, deposits, safe}. ### Step 3. Calculate Bank₁’s score $$ C\cap G_1=\varnothing,\qquad \mathrm{score}(1)=0 $$ ### Step 4. Prepare Bank₂’s gloss and count overlap G₂ = {financial, institution, accepts, deposits, channels, money, lending, activities}. The shared word is deposits. ### Step 5. Calculate Bank₂’s score and select $$ C\cap G_2=\{\mathrm{deposits}\},\qquad \mathrm{score}(2)=1 $$ 1 > 0, so choose the financial-institution sense. ### Final answer **Bank₂: financial institution.** The explicit overlap “deposits” supports this choice. **Yaad rakho:** Lesk: context aur har meaning ki definition mein common words count karo; highest overlap choose karo. ### ii. Limitations - Short contexts or short glosses may produce no overlaps or ties. - Exact word matching misses related words and synonyms, such as deposit versus savings, unless an extension is defined. - Irrelevant frequent words can create misleading overlaps if preprocessing is poor. - Dictionary glosses may omit a domain-specific sense or fail to express the context’s wording. - Simplified Lesk uses limited local information and may miss discourse or real-world knowledge. Lemmatization, richer glosses and semantic similarity can improve variants, but the exact variant and tie-breaking rule should be stated. --- ## M2 · Q4B · A second HMM sequence Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/papers/reexam-2026--reexam-2026-q4b/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/papers/reexam-2026/reexam-2026-q4b M2-relevant questions from an actual TEE paper · original marks retained for longer practice Question. Consider the following POS tagged corpus. Apply Hidden Markov Model (HMM) approach to: 1. Dog/Noun eats/Verb bread/Noun 2. The/Det cat/Noun sleeps/Verb 3. A/Det dog/Noun eats/Verb fish/Noun i. Construct the emission probability matrix. ii. Construct the transmission probability matrix. iii. Calculate the probability of the given sequence of events: The/Det dog/Noun eats/Verb bread/Noun. Note: Include start and end characters for all the sentences. [10 marks] ### Answer The repeated verb eats has a different emission count from the final paper; recalculate it rather than copying the previous numerical result. ### Given | Sentence | Tagged sequence | | --- | --- | | 1 | dog/Noun eats/Verb bread/Noun | | 2 | the/Det cat/Noun sleeps/Verb | | 3 | a/Det dog/Noun eats/Verb fish/Noun | Lowercase the corpus. Test sequence: the/Det dog/Noun eats/Verb bread/Noun. Add Start and End markers. ### To find Emission and transition matrices, then the probability of the supplied tagged sequence. ### Formula: which one and why $$ P(w\mid t)=\frac{C(t,w)}{C(t)} $$ $$ P(t_j\mid t_i)=\frac{C(t_i,t_j)}{\sum_kC(t_i,t_k)} $$ Emission divides the word’s tag-specific count by the total tokens with that tag. Transition divides the tag-pair count by all transitions leaving the previous tag, including End. $$ P(\mathbf w,\mathbf t)=P(t_1\mid\mathrm{Start})\prod_iP(w_i\mid t_i)\prod_{i=2}^{n}P(t_i\mid t_{i-1})P(\mathrm{End}\mid t_n) $$ ### Solution ### Step 1. Count tag totals and word frequencies Det: the=1, a=1, total2. Noun: dog=2, bread=1, cat=1, fish=1, total5. Verb: eats=2, sleeps=1, total3. ### Step 2. Construct emission matrix | Tag / word | a | bread | cat | dog | eats | fish | sleeps | the | | --- | --- | --- | --- | --- | --- | --- | --- | --- | | Det | 1/2 | 0 | 0 | 0 | 0 | 0 | 0 | 1/2 | | Noun | 0 | 1/5 | 1/5 | 2/5 | 0 | 1/5 | 0 | 0 | | Verb | 0 | 0 | 0 | 0 | 2/3 | 0 | 1/3 | 0 | ### Step 3. Count transitions from the three chains Start → Noun → Verb → Noun → End Start → Det → Noun → Verb → End Start → Det → Noun → Verb → Noun → End | From | Det count | Noun count | Verb count | End count | Total | | --- | --- | --- | --- | --- | --- | | Start | 2 | 1 | 0 | 0 | 3 | | Det | 0 | 2 | 0 | 0 | 2 | | Noun | 0 | 0 | 3 | 2 | 5 | | Verb | 0 | 2 | 0 | 1 | 3 | ### Step 4. Divide outgoing counts | From / to | Det | Noun | Verb | End | | --- | --- | --- | --- | --- | | Start | 2/3 | 1/3 | 0 | 0 | | Det | 0 | 1 | 0 | 0 | | Noun | 0 | 0 | 3/5 | 2/5 | | Verb | 0 | 2/3 | 0 | 1/3 | All nonterminal rows sum to one. No smoothing is requested. ### Step 5. Write every sequence factor | Event | Probability | | --- | --- | | Start → Det | 2/3 | | Det emits the | 1/2 | | Det → Noun | 1 | | Noun emits dog | 2/5 | | Noun → Verb | 3/5 | | Verb emits eats | 2/3 | | Verb → Noun | 2/3 | | Noun emits bread | 1/5 | | Noun → End | 2/5 | ### Step 6. Multiply all factors $$ P=\frac23\times\frac12\times1\times\frac25\times\frac35\times\frac23\times\frac23\times\frac15\times\frac25 $$ Start with subtotal 1. At each row, multiply the previous subtotal by exactly one next factor. Simplify the fraction after multiplication, but do not round it. | Factor being included | Previous subtotal × next factor = new subtotal | | --- | --- | | 1. Start → Det | $$ 1\times\frac{2}{3}=\frac{2}{3} $$ | | 2. Det emits the | $$ \frac{2}{3}\times\frac{1}{2}=\frac{1}{3} $$ | | 3. Det → Noun | $$ \frac{1}{3}\times1=\frac{1}{3} $$ | | 4. Noun emits dog | $$ \frac{1}{3}\times\frac{2}{5}=\frac{2}{15} $$ | | 5. Noun → Verb | $$ \frac{2}{15}\times\frac{3}{5}=\frac{2}{25} $$ | | 6. Verb emits eats | $$ \frac{2}{25}\times\frac{2}{3}=\frac{4}{75} $$ | | 7. Verb → Noun | $$ \frac{4}{75}\times\frac{2}{3}=\frac{8}{225} $$ | | 8. Noun emits bread | $$ \frac{8}{225}\times\frac{1}{5}=\frac{8}{1125} $$ | | 9. Noun → End | $$ \frac{8}{1125}\times\frac{2}{5}=\frac{16}{5625} $$ | The last subtotal is the complete product. Only now convert the fraction to a decimal. $$ P=\frac{16}{5625}\approx0.0028444444 $$ ### Final answer **P = 16/5625 ≈ 0.0028444444** for these words and specified tags jointly. The two occurrences of eats give P(eats | Verb) = 2/3. **Yaad rakho:** Same tag structure ho sakta hai, lekin word emission counts dobara count karna. **Boundary reminder:** Include P(Det | Start) = 2/3 and P(End | Noun) = 2/5. The handwritten key’s final line instead inserts the overall Det fraction 2/10 and omits End; the complete boundary-aware probability requested here is 0.0028444444. --- ## M2 · Paper instructions Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/mocks/mock-a--instructions/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/mocks/mock-a/instructions M2 · confirmed 2 + 2 + any two of 3 + 3 + 3 · worked practice **Practice paper · 10 marks · 60 minutes.** Q1 and Q2 are compulsory (2 marks each). Attempt any two of Q3, Q4 and Q5 (3 marks each). All five solutions are included below their questions. These are newly authored practice questions, not predictions of the actual paper. Use about 8 minutes each for Q1–Q2, 16 minutes each for the two chosen 3-mark questions, and the remaining 12 minutes to read, choose and check. Write the formula and substitutions even if a final calculation gets stuck. --- ## M2 · Q1 (2 marks) Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/mocks/mock-a--q1/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/mocks/mock-a/q1 M2 · confirmed 2 + 2 + any two of 3 + 3 + 3 · worked practice ### Q1 For “we study language models”, list the unigrams, bigrams and trigrams. Do not add boundary markers. (2 marks) **Given** Four tokens in the stated order. **To find** All contiguous n-grams for n=1,2,3. ### Formula and why it applies $$ \#\text{ n-grams}=T-n+1 $$ T is the number of tokens. Slide a window of n consecutive tokens, one position at a time. ### Solution #### Step 1. Unigrams, window size 1 (we), (study), (language), (models); count=4. #### Step 2. Bigrams, window size 2 (we study), (study language), (language models); count=4−2+1=3. #### Step 3. Trigrams, window size 3 (we study language), (study language models); count=4−3+1=2. 4 unigrams, 3 bigrams and 2 trigrams, listed above. --- ## M2 · Q2 (2 marks) Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/mocks/mock-a--q2/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/mocks/mock-a/q2 M2 · confirmed 2 + 2 + any two of 3 + 3 + 3 · worked practice ### Q2 A corpus has C(read)=10, C(read books)=6 and C(read papers)=4. Find both bigram conditional probabilities. (2 marks) **Given** read has 10 continuation opportunities; 6 are books and 4 are papers. **To find** P(books | read) and P(papers | read). ### Formula and why it applies $$ P(w\mid h)=\frac{C(hw)}{C(h)} $$ h is the preceding word. Divide each pair count by the count of its shared history. ### Solution #### Step 1. Probability of books $$ P(\text{books}\mid\text{read})=\frac6{10}=0.6 $$ #### Step 2. Probability of papers $$ P(\text{papers}\mid\text{read})=\frac4{10}=0.4 $$ #### Step 3. Check the complete row $$ 0.6+0.4=1 $$ P(books | read)=0.6 and P(papers | read)=0.4. --- ## M2 · Q3 (3 marks) Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/mocks/mock-a--q3/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/mocks/mock-a/q3 M2 · confirmed 2 + 2 + any two of 3 + 3 + 3 · worked practice ### Q3 Training corpus: “I like tea” and “I like coffee”. Lowercase all words and add and to each sentence. Using add-one bigram smoothing and a vocabulary containing both boundary symbols, find P(“i like tea”). (3 marks) **Given** Padded corpus: i like tea ; i like coffee . V={, i, like, tea, coffee, }, |V|=6. **To find** Product of the four smoothed transition probabilities, including the end transition. ### Formula and why it applies $$ P_{+1}(w\mid h)=\frac{C(hw)+1}{C(h)+|V|} $$ Every possible vocabulary continuation gets one pseudocount. Both boundaries are counted in V because the question explicitly requires the course convention. ### Solution #### Step 1. Count histories and pairs | Transition | Pair count | History count | | --- | --- | --- | | → i | 2 | 2 | | i → like | 2 | 2 | | like → tea | 1 | 2 | | tea → | 1 | 1 | #### Step 2. Compute first probability $$ P(i\mid\langle s\rangle)=\frac{2+1}{2+6}=\frac38 $$ #### Step 3. Compute middle probabilities $$ P(\text{like}\mid i)=\frac38 $$ $$ P(\text{tea}\mid\text{like})=\frac{1+1}{2+6}=\frac14 $$ #### Step 4. Compute end probability $$ P(\langle/ s\rangle\mid\text{tea})=\frac{1+1}{1+6}=\frac27 $$ #### Step 5. Multiply $$ P=\frac38\times\frac38\times\frac14\times\frac27=\frac9{896}\approx0.010044643 $$ Sentence probability = 9/896 ≈ 0.010044643. --- ## M2 · Q4 (3 marks) Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/mocks/mock-a--q4/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/mocks/mock-a/q4 M2 · confirmed 2 + 2 + any two of 3 + 3 + 3 · worked practice ### Q4 An HMM has P(N|start)=0.6, P(V|N)=0.5, P(end|V)=0.4, P(dogs|N)=0.5 and P(run|V)=0.8. Find the joint probability of words “dogs run” with tag path N,V, including the end transition. (3 marks) **Given** The five required transition/emission factors are supplied. **To find** P(dogs run, N V, end). ### Formula and why it applies $$ P(w,t)=P(t_1\mid start)P(w_1\mid t_1)P(t_2\mid t_1)P(w_2\mid t_2)P(end\mid t_2) $$ Transitions move between tags. Emissions generate words from tags. Include both types of factor. ### Solution #### Step 1. Place the factors along the path start → N: 0.6 → emit dogs: 0.5 → N → V: 0.5 → emit run: 0.8 → V → end: 0.4 #### Step 2. First word contribution $$ 0.6\times0.5=0.3 $$ #### Step 3. Enter V and emit run $$ 0.3\times0.5=0.15 $$ $$ 0.15\times0.8=0.12 $$ #### Step 4. End the sentence $$ 0.12\times0.4=0.048 $$ Joint path-and-word probability = 0.048. This is not a normalised posterior over tag paths. --- ## M2 · Q5 (3 marks) Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/mocks/mock-a--q5/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/mocks/mock-a/q5 M2 · confirmed 2 + 2 + any two of 3 + 3 + 3 · worked practice ### Q5 For a two-sense Naive Bayes WSD model, P(F)=P(R)=0.5. After smoothing, P(loan|F)=0.4, P(money|F)=0.3, P(loan|R)=0.1 and P(money|R)=0.1. Classify “bank” using context {loan, money} and obtain the winning sense’s posterior. (3 marks) **Given** F=financial sense; R=river sense. Two exhaustive senses; conditional independence of the context words is assumed. **To find** Two scores, selected sense and normalised posterior. ### Formula and why it applies $$ s(c)=P(c)P(\text{loan}\mid c)P(\text{money}\mid c) $$ $$ P(c\mid x)=\frac{s(c)}{s(F)+s(R)} $$ Multiply the prior by the two likelihoods, then normalise because a posterior is requested. ### Solution #### Step 1. Financial score $$ s(F)=0.5\times0.4\times0.3=0.06 $$ #### Step 2. River score $$ s(R)=0.5\times0.1\times0.1=0.005 $$ #### Step 3. Total evidence score $$ 0.06+0.005=0.065 $$ #### Step 4. Normalise winner $$ P(F\mid x)=\frac{0.06}{0.065}=\frac{12}{13}\approx0.923077 $$ Financial sense; posterior ≈0.923077. --- ## M2 · Paper instructions Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/mocks/mock-b--instructions/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/mocks/mock-b/instructions M2 · confirmed 2 + 2 + any two of 3 + 3 + 3 · worked practice **Practice paper · 10 marks · 60 minutes.** Q1 and Q2 are compulsory (2 marks each). Attempt any two of Q3, Q4 and Q5 (3 marks each). All five solutions are included below their questions. These are newly authored practice questions, not predictions of the actual paper. Use about 8 minutes each for Q1–Q2, 16 minutes each for the two chosen 3-mark questions, and the remaining 12 minutes to read, choose and check. Write the formula and substitutions even if a final calculation gets stuck. --- ## M2 · Q1 (2 marks) Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/mocks/mock-b--q1/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/mocks/mock-b/q1 M2 · confirmed 2 + 2 + any two of 3 + 3 + 3 · worked practice ### Q1 A language model gives a test sequence probability of 0.0016 across four predicted tokens. Calculate perplexity. (2 marks) **Given** P=0.0016; N=4. **To find** PP. ### Formula and why it applies $$ PP=P^{-1/N}=\left(\frac1P\right)^{1/N} $$ Use the number of predicted events, not an unrelated training token count. ### Solution #### Step 1. Reciprocal $$ \frac1{0.0016}=625 $$ #### Step 2. Fourth root $$ PP=625^{1/4}=5 $$ Perplexity=5. --- ## M2 · Q2 (2 marks) Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/mocks/mock-b--q2/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/mocks/mock-b/q2 M2 · confirmed 2 + 2 + any two of 3 + 3 + 3 · worked practice ### Q2 A first-order Markov model starts in state A with probability 0.7. P(B|A)=0.4 and P(A|B)=0.3. Compute the path probability A→B→A, with no end-state factor. (2 marks) **Given** Start A=0.7; A→B=0.4; B→A=0.3. **To find** Probability of this state path. ### Formula and why it applies $$ P(A,B,A)=P(A)P(B\mid A)P(A\mid B) $$ First-order dependence uses only the previous state for each transition. ### Solution #### Step 1. First transition contribution $$ 0.7\times0.4=0.28 $$ #### Step 2. Second transition contribution $$ 0.28\times0.3=0.084 $$ Path probability=0.084. --- ## M2 · Q3 (3 marks) Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/mocks/mock-b--q3/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/mocks/mock-b/q3 M2 · confirmed 2 + 2 + any two of 3 + 3 + 3 · worked practice ### Q3 For a fixed vocabulary of five possible continuations, C(very)=7 and C(very bright)=0. Calculate the unsmoothed and Laplace-smoothed probabilities of bright after very. (3 marks) **Given** History count=7; pair count=0; |V|=5. **To find** MLE and add-one probabilities. ### Formula and why it applies $$ P_{MLE}(w\mid h)=\frac{C(hw)}{C(h)} $$ $$ P_{+1}(w\mid h)=\frac{C(hw)+1}{C(h)+|V|} $$ MLE uses only observations. Laplace reserves probability for all known vocabulary outcomes, including unseen pairs. ### Solution #### Step 1. Unsmoothed estimate $$ P_{MLE}(\text{bright}\mid\text{very})=\frac07=0 $$ #### Step 2. Add one to the pair count $$ 0+1=1 $$ #### Step 3. Add vocabulary size to the history count $$ 7+5=12 $$ #### Step 4. Divide $$ P_{+1}(\text{bright}\mid\text{very})=\frac1{12}\approx0.083333 $$ MLE=0; Laplace=1/12≈0.083333. The zero count means unseen in this corpus, not impossible language. --- ## M2 · Q4 (3 marks) Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/mocks/mock-b--q4/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/mocks/mock-b/q4 M2 · confirmed 2 + 2 + any two of 3 + 3 + 3 · worked practice ### Q4 A two-state HMM has initial probabilities P(N)=0.6, P(V)=0.4. Transition rows (N→N,N→V)=(0.3,0.7) and (V→N,V→V)=(0.8,0.2). Emissions are P(fish|N)=0.5, P(fish|V)=0.4, P(swim|N)=0.1 and P(swim|V)=0.6. Find the best tag path for “fish swim”, without end transitions. (3 marks) **Given** Two states N,V; all start, transition and relevant emission probabilities are supplied. **To find** Largest joint score and its tag sequence. ### Formula and why it applies $$ \delta_1(t)=P(t)P(w_1\mid t) $$ $$ \delta_2(t)=P(w_2\mid t)\max_s[\delta_1(s)P(t\mid s)] $$ Viterbi keeps the best path ending at each tag and stores its predecessor. Enumerating four paths gives the same result for this tiny example. ### Solution #### Step 1. Initialise fish $$ \delta_1(N)=0.6\times0.5=0.3 $$ $$ \delta_1(V)=0.4\times0.4=0.16 $$ #### Step 2. Score both paths ending in N $$ NN:0.3\times0.3\times0.1=0.009 $$ $$ VN:0.16\times0.8\times0.1=0.0128 $$ Keep VN at N; predecessor V. #### Step 3. Score both paths ending in V $$ NV:0.3\times0.7\times0.6=0.126 $$ $$ VV:0.16\times0.2\times0.6=0.0192 $$ Keep NV at V; predecessor N. #### Step 4. Choose and backtrack 0.126 > 0.0128. End at V and follow its stored predecessor N. Best sequence: N,V, with joint score 0.126. --- ## M2 · Q5 (3 marks) Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/mocks/mock-b--q5/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/mocks/mock-b/q5 M2 · confirmed 2 + 2 + any two of 3 + 3 + 3 · worked practice ### Q5 Apply simplified Lesk to “bank” with preprocessed context {river, water, flooded}. Gloss R is {sloping, land, river, water}; gloss F is {financial, institution, money, loan}. (3 marks) **Given** Two already preprocessed gloss sets and one context set. **To find** Overlap scores and selected sense. ### Formula and why it applies $$ \text{score}(s)=|\text{context}\cap\text{gloss}(s)| $$ Simplified Lesk chooses the sense whose gloss shares the largest number of context words, using the specified set-overlap convention. ### Solution #### Step 1. Intersect the river gloss {river, water, flooded} ∩ {sloping, land, river, water} = {river, water}. #### Step 2. Count the river overlap $$ score(R)=2 $$ #### Step 3. Intersect and count the financial gloss No shared words; intersection is empty, so score(F)=0. #### Step 4. Compare 2 > 0, hence select the river-bank sense. River-bank sense, with overlap 2 versus 0. --- ## M2 · Paper instructions Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/mocks/mock-c--instructions/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/mocks/mock-c/instructions M2 · confirmed 2 + 2 + any two of 3 + 3 + 3 · worked practice **Practice paper · 10 marks · 60 minutes.** Q1 and Q2 are compulsory (2 marks each). Attempt any two of Q3, Q4 and Q5 (3 marks each). All five solutions are included below their questions. These are newly authored practice questions, not predictions of the actual paper. Use about 8 minutes each for Q1–Q2, 16 minutes each for the two chosen 3-mark questions, and the remaining 12 minutes to read, choose and check. Write the formula and substitutions even if a final calculation gets stuck. --- ## M2 · Q1 (2 marks) Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/mocks/mock-c--q1/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/mocks/mock-c/q1 M2 · confirmed 2 + 2 + any two of 3 + 3 + 3 · worked practice ### Q1 A corpus contains five occurrences tagged Noun. Three of them are “cat”. It contains four Verb tokens, one of which is “cat”. Calculate P(cat|Noun) and P(cat|Verb). (2 marks) **Given** C(N)=5; C(cat,N)=3; C(V)=4; C(cat,V)=1. **To find** The two emission probabilities. ### Formula and why it applies $$ P(w\mid t)=\frac{C(w,t)}{C(t)} $$ An emission conditions on the tag, so the denominator counts tokens of that tag. ### Solution #### Step 1. Noun emission $$ P(\text{cat}\mid N)=\frac35=0.6 $$ #### Step 2. Verb emission $$ P(\text{cat}\mid V)=\frac14=0.25 $$ P(cat|N)=0.6 and P(cat|V)=0.25. --- ## M2 · Q2 (2 marks) Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/mocks/mock-c--q2/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/mocks/mock-c/q2 M2 · confirmed 2 + 2 + any two of 3 + 3 + 3 · worked practice ### Q2 Each of five words may receive any of three POS tags. How many possible tag sequences exist before applying constraints? (2 marks) **Given** T=5 token positions; K=3 choices at each position. **To find** Total tag sequences. ### Formula and why it applies $$ K^T $$ Choose one tag independently for each position when counting candidates; multiply the number of options five times. ### Solution #### Step 1. Expand the power $$ 3^5=3\times3\times3\times3\times3 $$ #### Step 2. Multiply $$ 3\times3=9,\quad9\times3=27 $$ $$ 27\times3=81,\quad81\times3=243 $$ 243 tag sequences. --- ## M2 · Q3 (3 marks) Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/mocks/mock-c--q3/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/mocks/mock-c/q3 M2 · confirmed 2 + 2 + any two of 3 + 3 + 3 · worked practice ### Q3 For trigram prediction, C(i like)=5, C(i like tea)=3 and C(i like coffee)=2. Find the MLE probability of each continuation. Then find add-one P(tea|i like) using a continuation vocabulary of size 4. (3 marks) **Given** Two-word history i like occurs 5 times; continuations tea=3, coffee=2; |V|=4. **To find** Two MLE values and the smoothed tea probability. ### Formula and why it applies $$ P(w_i\mid w_{i-2},w_{i-1})=\frac{C(w_{i-2},w_{i-1},w_i)}{C(w_{i-2},w_{i-1})} $$ $$ P_{+1}(w\mid h)=\frac{C(hw)+1}{C(h)+|V|} $$ A trigram conditions on two preceding words. The same add-one count rule applies to that longer history. ### Solution #### Step 1. Tea MLE $$ P(\text{tea}\mid i,\text{like})=\frac35=0.6 $$ #### Step 2. Coffee MLE $$ P(\text{coffee}\mid i,\text{like})=\frac25=0.4 $$ #### Step 3. Smooth tea $$ P_{+1}(\text{tea}\mid i,\text{like})=\frac{3+1}{5+4}=\frac49\approx0.444444 $$ MLE tea=0.6; coffee=0.4; smoothed tea=4/9≈0.444444. --- ## M2 · Q4 (3 marks) Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/mocks/mock-c--q4/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/mocks/mock-c/q4 M2 · confirmed 2 + 2 + any two of 3 + 3 + 3 · worked practice ### Q4 Training sentences with tags are: “the/D cat/N sleeps/V” and “a/D dog/N runs/V”. Estimate start/end-inclusive transition probabilities and the joint probability of “the cat sleeps” with path D,N,V. (3 marks) **Given** Two paths start→D→N→V→end. Four distinct words the,a,cat,dog and two verbs sleeps,runs. Use unsmoothed estimates and lowercase tokens. **To find** Transitions, required emissions and full path score. ### Formula and why it applies $$ P(t_j\mid t_i)=\frac{C(t_i\to t_j)}{\sum_k C(t_i\to t_k)} $$ $$ P(w\mid t)=\frac{C(w,t)}{C(t)} $$ Transition denominators count outgoing transitions from the previous state. Emission denominators count all tokens tagged with the emitting state. ### Solution #### Step 1. Count transitions start→D, D→N, N→V and V→end each occur twice. Each state has just one observed next state. #### Step 2. Divide to get probabilities $$ P(D\mid start)=P(N\mid D)=P(V\mid N)=P(end\mid V)=\frac22=1 $$ #### Step 3. Estimate emissions $$ P(\text{the}\mid D)=\frac12 $$ $$ P(\text{cat}\mid N)=\frac12 $$ $$ P(\text{sleeps}\mid V)=\frac12 $$ #### Step 4. Multiply in word order $$ P=1\times\frac12\times1\times\frac12\times1\times\frac12\times1=\frac18=0.125 $$ Joint probability=0.125. Each of the four observed transition probabilities is 1. --- ## M2 · Q5 (3 marks) Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/mocks/mock-c--q5/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/mocks/mock-c/q5 M2 · confirmed 2 + 2 + any two of 3 + 3 + 3 · worked practice ### Q5 In a WSD training set, financial bank and river bank each have two labelled contexts. Token totals are N(F)=6 and N(R)=6. Vocabulary size is 8. Counts are C(loan,F)=2, C(loan,R)=0, C(water,F)=0, C(water,R)=3. Use add-one Naive Bayes to classify context {loan,water}. (3 marks) **Given** P(F)=P(R)=2/4=0.5; |V|=8; both class token totals=6. **To find** Smoothed scores and winning sense. ### Formula and why it applies $$ P(w\mid c)=\frac{C(w,c)+1}{N_c+|V|} $$ $$ s(c)=P(c)\prod_wP(w\mid c) $$ Class priors use labelled-context counts; word likelihood denominators use class token totals. Do not exchange those two totals. ### Solution #### Step 1. Find the denominator $$ 6+8=14 $$ #### Step 2. Financial likelihoods $$ P(loan\mid F)=\frac3{14},\quad P(water\mid F)=\frac1{14} $$ #### Step 3. River likelihoods $$ P(loan\mid R)=\frac1{14},\quad P(water\mid R)=\frac4{14} $$ #### Step 4. Compute financial score $$ s(F)=\frac12\times\frac3{14}\times\frac1{14}=\frac3{392}\approx0.007653061 $$ #### Step 5. Compute river score $$ s(R)=\frac12\times\frac1{14}\times\frac4{14}=\frac4{392}\approx0.010204082 $$ #### Step 6. Choose 4/392 > 3/392; the water evidence outweighs the loan evidence under these counts. River sense wins. If normalised, P(R|context)=4/7≈0.571429. --- ## M2 · Last-day reading order Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/revision/m2-revision--order/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/revision/m2-revision/order M2 · quick revision after completing the lessons Start with the families most likely to require arithmetic: **smoothed bigrams, HMM probabilities and Naive Bayes WSD**. Then revise perplexity, n-gram counts and Lesk. Parsing, POS approaches, WordNet and lexical relations remain in the confirmed portion; do not remove them merely because the expected paper is mostly numerical. | Time block | Read and work | What your written solution should contain | | --- | --- | --- | | First 60–90 min | N-grams, MLE and Laplace examples | Padded corpus → vocabulary → count table → each fraction → full product. | | Next 60–90 min | HMM transition/emission and path examples | Tag totals → transition counts → matrices → alternating transition/emission factors. | | Next 60–90 min | WSD and Lesk examples | Class priors → vocabulary → class totals → smoothed word likelihoods → score comparison. | | Next 45–60 min | Perplexity, POS, parsing, WordNet, NER | Definitions, one meaningful example, a tree or table where needed. | | Final 60 min | One M2 mock under exam time | Both 2-mark questions and any two 3-mark questions; compare every step with the answers. | ### The writing format to repeat Given → To find → Formula + meaning → Numbered calculations → Answer If the question supplies the corpus, draw the count table before using the calculator. A long product should be built from fractions that you have already justified. Keep fractions or stored calculator values until the end. --- ## M2 · Recognise the question and start correctly Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/revision/m2-revision--method/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/revision/m2-revision/method M2 · quick revision after completing the lessons | Question clue | Use this method | First thing to write | | --- | --- | --- | | “next word”, “sentence probability” | N-gram model | List the needed word transitions. | | “unseen pair”, “Laplace” | Add-one smoothing | State V and count each history. | | “perplexity” | Reciprocal Nth root | State P(test sequence) and N. | | “transition/emission matrix” | HMM count estimation | Separate tag-to-tag counts from tag-to-word counts. | | “given tag sequence” | HMM joint path | Initial × emission × transition × emission … | | “best tag sequence” | Viterbi or enumerate tiny paths | Initialise one score per tag; retain predecessor maxima. | | “gloss overlap” | Simplified Lesk | Preprocess context and glosses consistently. | | “classify word sense with counts” | Naive Bayes WSD | Priors from examples, likelihoods from tokens. | | “parse tree” | Given CFG + chosen parser | Show grammar expansions or stack reductions. | ### Errors that change the answer 1. Dividing a conditional count by the whole corpus instead of its history count. 2. Adding 1 to the numerator without adding |V| to the denominator. 3. Forgetting the end transition. 4. Counting repeated words as new vocabulary types. 5. Using P(tag|word) in place of P(word|tag). 6. Calling an unnormalised score a posterior. 7. Silently mixing case, lemmatisation or boundary conventions. ### Short theory answer format For 2 marks: give the exact definition and one specific example or distinction. For 3 marks: add the mechanism or two ordered steps; include a small diagram/table where it carries information. The actual allocation is examiner-dependent, so these are writing guides, not an official marking scheme. --- ## M2 · What this NLP guide covers Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/about/scope--sources/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/about/scope/sources NLP · M2 and TEE · edition 3 October 2026 **M2 portion confirmed by you:** Unit 3 from PDF slide 62 to 160; Unit 4 complete (128 slides); Unit 5 slides 1–57. Slide 62 of Unit 3 starts N-grams. Slide 57 of Unit 5 completes the Naive Bayes WSD example. Vector-space models start on slide 58 and belong to TEE here. **M2 pattern confirmed by you:** both 2-mark questions compulsory; any two of the three 3-mark questions. Total attempted 10. Course policy gives one hour. Three newly authored, fully solved M2 practice papers use that pattern. **TEE:** all seven units of the official AY 2026–27 syllabus. The new ZIP supplies faculty slide packs for Units 1–5 plus a regular-expressions handout. Units 6–7 are syllabus-based explanations supplemented with the standard references below; their faculty slide packs were not supplied. The official syllabus includes GloVe and spelling correction, and both are retained even though the policy’s lecture table abbreviates them. **Previous papers found:** final exam dated 2 December 2025 and re-exam dated 9 February 2026, with answer keys, inside Semester V.zip. Programme copies were deduplicated. Every supplied question subpart is mapped to the current syllabus; no question was excluded as out of syllabus. Only wholly M2-eligible subparts appear in the M2 paper tab; mixed questions remain in the complete TEE papers. The current syllabus lists course code 702AI0C037. The supplied policy and old papers use 702AI0C009; the policy also retains an older programme/semester header. Topic coverage follows the current official syllabus and your exact M2 cutoff, rather than treating the old header as a different course. ### Conventions that must stay visible Course bigram answer keys count both start and end symbols in vocabulary size. Worked PYQs state and follow that convention; other exercises state their own vocabulary explicitly. A start symbol is conditioning context, so alternative textbook conventions may exclude it from prediction vocabulary and produce a different smoothed number. HMM path scores and Naive Bayes class scores are labelled as unnormalised when applicable. Any normalisation, case-folding or lemmatisation assumption is stated before counting. ### Reading references - Your uploaded course policy, official Semester V syllabus, five NLP slide packs and regular-expressions handout. - Your uploaded final and re-exam questions and answer keys (AY 2025–26). - [Jurafsky & Martin · Speech and Language Processing](https://web.stanford.edu/~jurafsky/slp3/) — terminology and standard methods. - [NLTK Book](https://www.nltk.org/book/) — corpus, tagging, parsing and information-extraction concepts. - [Tetreault · A Corpus-Based Evaluation of Centering and Pronoun Resolution](https://aclanthology.org/J01-4003.pdf) — precise Hobbs traversal. Examples are teaching problems unless explicitly identified by their paper and question number. Practice-paper marks indicate intended answer length, not an official forecast. Reading alone cannot guarantee a score; writing the solutions independently is what makes the procedures usable under exam time. ### TEE depth review · 10 October 2026 The core TEE route retains the faculty topics CYK/CKY, grammar conversion, interpolation and Stupid Backoff. Add-k and HMM forward/posterior calculations are available separately under Optional extensions; they are not part of the core reading or formula sequence. The two TEE timed-practice entries reproduce the supplied actual papers and are not additional unseen mock papers. Method references: [Jurafsky and Martin: Hidden Markov Models](https://web.stanford.edu/~jurafsky/slp3/A.pdf); [Jurafsky and Martin: Constituency Parsing](https://web.stanford.edu/~jurafsky/slp3/E.pdf). New numerical inputs and worked traces are teaching examples. --- ## M2 · Video companions in your NotebookLM Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/about/scope--videos/ Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/about/scope/videos NLP · M2 and TEE · edition 3 October 2026 Your M2 notebook contains the exact three slide ranges and checked numerical teaching examples. The requested video topics are N-grams and Laplace smoothing, HMM/Viterbi, and Naive Bayes WSD. Use the written solutions here for the complete calculations. [Open your NLP M2 notebook →](https://notebook.google.com/notebook/a270ef1e-c285-4d31-9db1-302090c5f8b1) The notebook is in your signed-in Google account. Video generation and availability are shown in its Studio panel. Generated narration is a supplementary explanation; the numerical results in this guide have been independently checked. ## Complete video catalog updated\_utc : 2026-10-10T11:17:01.698545+00:00 notebooks : m2 : tee : lesson\_map : m2 : foundations/language : - nlp-m2-01-foundations-counts-probability foundations/notation : - nlp-m2-01-foundations-counts-probability foundations/probability : - nlp-m2-01-foundations-counts-probability m2-language-models/words-counts-probability : - nlp-m2-01-foundations-counts-probability foundations/bayes : - nlp-m2-02-bayes-generative-discriminative m2-pos-tagging/generative-discriminative : - nlp-m2-02-bayes-generative-discriminative m2-language-models/n-grams-construction : - nlp-m2-03-ngrams-language-models m2-language-models/collocations-and-lm : - nlp-m2-03-ngrams-language-models m2-language-models/chain-rule-markov : - nlp-m2-03-ngrams-language-models m2-count-to-probability/unigram-estimation : - nlp-m2-03-ngrams-language-models m2-count-to-probability/bigram-count-table : - nlp-m2-04-bigram-trigram-tables m2-count-to-probability/trigram-estimation : - nlp-m2-04-bigram-trigram-tables m2-count-to-probability/class-count-table-questions : - nlp-m2-04-bigram-trigram-tables m2-evaluation-smoothing/laplace-basics : - nlp-m2-05-laplace-smoothing-discount m2-evaluation-smoothing/smoothed-sentence : - nlp-m2-05-laplace-smoothing-discount m2-evaluation-smoothing/adjusted-count-discount : - nlp-m2-05-laplace-smoothing-discount m2-evaluation-smoothing/evaluation-perplexity : - nlp-m2-06-perplexity-logs-calculator m2-evaluation-smoothing/log-probabilities : - nlp-m2-06-perplexity-logs-calculator m2-evaluation-smoothing/class-perplexity-characters : - nlp-m2-06-perplexity-logs-calculator foundations/calculator : - nlp-m2-06-perplexity-logs-calculator m2-pos-tagging/word-classes-pos : - nlp-m2-07-pos-tags-tagger-methods m2-pos-tagging/tagger-methods : - nlp-m2-07-pos-tags-tagger-methods m2-hmm/markov-chain : - nlp-m2-08-markov-hmm-components m2-hmm/hmm-components : - nlp-m2-08-markov-hmm-components m2-hmm/transition-emission-estimation : - nlp-m2-08-markov-hmm-components m2-hmm/hot-cold-paths : - nlp-m2-09-hmm-joint-paths m2-hmm/tagging-corpus-case : - nlp-m2-09-hmm-joint-paths m2-hmm-decoding/joint-vs-posterior : - nlp-m2-09-hmm-joint-paths m2-hmm-decoding/sequence-count : - nlp-m2-10-viterbi-decoding m2-hmm-decoding/viterbi-explained : - nlp-m2-10-viterbi-decoding m2-chunks-grammar/chunking-chinking : - nlp-m2-11-chunking-chinking m2-chunks-grammar/cfg-foundation : - nlp-m2-12-cfg-top-down-parsing m2-chunks-grammar/structural-ambiguity : - nlp-m2-12-cfg-top-down-parsing m2-parsing/top-down-theory : - nlp-m2-12-cfg-top-down-parsing m2-parsing/top-down-examples : - nlp-m2-12-cfg-top-down-parsing m2-parsing/shift-reduce-theory : - nlp-m2-13-bottom-up-shift-reduce m2-parsing/shift-reduce-examples : - nlp-m2-13-bottom-up-shift-reduce m2-ner/ner-definition : - nlp-m2-14-named-entity-recognition m2-ner/ner-approaches : - nlp-m2-14-named-entity-recognition m2-ner/ner-worked-examples : - nlp-m2-14-named-entity-recognition u5-meaning/u5-words : - nlp-m2-15-lexical-meaning-wordnet u5-meaning/u5-relations : - nlp-m2-15-lexical-meaning-wordnet u5-meaning/u5-wordnet : - nlp-m2-15-lexical-meaning-wordnet u5-wsd/u5-wsd-basics : - nlp-m2-16-lesk-wsd u5-wsd/u5-lesk : - nlp-m2-16-lesk-wsd u5-wsd/u5-walker : - nlp-m2-17-walker-graph-wsd u5-wsd/u5-graph : - nlp-m2-17-walker-graph-wsd u5-naive-bayes/u5-bayes-meaning : - nlp-m2-18-naive-bayes-wsd u5-naive-bayes/u5-bayes-steps : - nlp-m2-18-naive-bayes-wsd u5-naive-bayes/u5-bayes-mam : - nlp-m2-18-naive-bayes-wsd u5-naive-bayes/u5-bayes-practice : - nlp-m2-18-naive-bayes-wsd tee : foundations/language : - nlp-m2-01-foundations-counts-probability foundations/notation : - nlp-m2-01-foundations-counts-probability foundations/probability : - nlp-m2-01-foundations-counts-probability m2-language-models/words-counts-probability : - nlp-m2-01-foundations-counts-probability foundations/bayes : - nlp-m2-02-bayes-generative-discriminative m2-pos-tagging/generative-discriminative : - nlp-m2-02-bayes-generative-discriminative m2-language-models/n-grams-construction : - nlp-m2-03-ngrams-language-models m2-language-models/collocations-and-lm : - nlp-m2-03-ngrams-language-models m2-language-models/chain-rule-markov : - nlp-m2-03-ngrams-language-models m2-count-to-probability/unigram-estimation : - nlp-m2-03-ngrams-language-models m2-count-to-probability/bigram-count-table : - nlp-m2-04-bigram-trigram-tables m2-count-to-probability/trigram-estimation : - nlp-m2-04-bigram-trigram-tables m2-count-to-probability/class-count-table-questions : - nlp-m2-04-bigram-trigram-tables m2-evaluation-smoothing/laplace-basics : - nlp-m2-05-laplace-smoothing-discount m2-evaluation-smoothing/smoothed-sentence : - nlp-m2-05-laplace-smoothing-discount m2-evaluation-smoothing/adjusted-count-discount : - nlp-m2-05-laplace-smoothing-discount m2-evaluation-smoothing/evaluation-perplexity : - nlp-m2-06-perplexity-logs-calculator m2-evaluation-smoothing/log-probabilities : - nlp-m2-06-perplexity-logs-calculator m2-evaluation-smoothing/class-perplexity-characters : - nlp-m2-06-perplexity-logs-calculator foundations/calculator : - nlp-m2-06-perplexity-logs-calculator m2-pos-tagging/word-classes-pos : - nlp-m2-07-pos-tags-tagger-methods m2-pos-tagging/tagger-methods : - nlp-m2-07-pos-tags-tagger-methods m2-hmm/markov-chain : - nlp-m2-08-markov-hmm-components m2-hmm/hmm-components : - nlp-m2-08-markov-hmm-components m2-hmm/transition-emission-estimation : - nlp-m2-08-markov-hmm-components m2-hmm/hot-cold-paths : - nlp-m2-09-hmm-joint-paths m2-hmm/tagging-corpus-case : - nlp-m2-09-hmm-joint-paths m2-hmm-decoding/joint-vs-posterior : - nlp-m2-09-hmm-joint-paths m2-hmm-decoding/sequence-count : - nlp-m2-10-viterbi-decoding m2-hmm-decoding/viterbi-explained : - nlp-m2-10-viterbi-decoding m2-chunks-grammar/chunking-chinking : - nlp-m2-11-chunking-chinking m2-chunks-grammar/cfg-foundation : - nlp-m2-12-cfg-top-down-parsing m2-chunks-grammar/structural-ambiguity : - nlp-m2-12-cfg-top-down-parsing m2-parsing/top-down-theory : - nlp-m2-12-cfg-top-down-parsing m2-parsing/top-down-examples : - nlp-m2-12-cfg-top-down-parsing m2-parsing/shift-reduce-theory : - nlp-m2-13-bottom-up-shift-reduce m2-parsing/shift-reduce-examples : - nlp-m2-13-bottom-up-shift-reduce m2-ner/ner-definition : - nlp-m2-14-named-entity-recognition m2-ner/ner-approaches : - nlp-m2-14-named-entity-recognition m2-ner/ner-worked-examples : - nlp-m2-14-named-entity-recognition u5-meaning/u5-words : - nlp-m2-15-lexical-meaning-wordnet u5-meaning/u5-relations : - nlp-m2-15-lexical-meaning-wordnet u5-meaning/u5-wordnet : - nlp-m2-15-lexical-meaning-wordnet u5-wsd/u5-wsd-basics : - nlp-m2-16-lesk-wsd u5-wsd/u5-lesk : - nlp-m2-16-lesk-wsd u5-wsd/u5-walker : - nlp-m2-17-walker-graph-wsd u5-wsd/u5-graph : - nlp-m2-17-walker-graph-wsd u5-naive-bayes/u5-bayes-meaning : - nlp-m2-18-naive-bayes-wsd u5-naive-bayes/u5-bayes-steps : - nlp-m2-18-naive-bayes-wsd u5-naive-bayes/u5-bayes-mam : - nlp-m2-18-naive-bayes-wsd u5-naive-bayes/u5-bayes-practice : - nlp-m2-18-naive-bayes-wsd u1-introduction/u1-nlp : - tee-01 u1-introduction/u1-language : - tee-01 u1-introduction/u1-approaches : - tee-01 u1-introduction/u1-nlu-nlg : - tee-02 u1-introduction/u1-pipeline : - tee-02 u1-introduction/u1-challenges : - tee-02 u2-preprocess/u2-tokenize : - tee-03 u2-preprocess/u2-normalization : - tee-03 u2-preprocess/u2-stem-lemma : - tee-03 u2-preprocess/u2-ttr : - tee-03 u2-representation/u2-vocab : - tee-04 u2-representation/u2-label-onehot : - tee-04 u2-representation/u2-bow : - tee-04 u2-representation/u2-tfidf : - tee-05 u2-similarity/u2-cosine : - tee-05 u2-representation/u2-slide-combined : - tee-05 u2-similarity/u2-jaccard : - tee-06 u2-similarity/u2-distances : - tee-06 u2-similarity/u2-choice : - tee-06 u3-corpora/u3-corpus : - tee-07 u3-corpora/u3-types : - tee-07 u3-corpora/u3-access : - tee-07 u3-corpora/u3-resources : - tee-07 u3-corpora/u3-acquisition : - tee-07 u3-corpora/u3-links : - tee-07 u3-regex/regex-language : - tee-08 u3-regex/regex-functions : - tee-08 u3-regex/regex-worked : - tee-08 u3-regex/regex-reference : - tee-08 u5-vectors/u5-distribution : - tee-09 u5-vectors/u5-representation : - tee-09 u5-vectors/u5-sentence : - tee-09 u5-vectors/u5-word2vec : - tee-10 u5-glove/u5-glove-theory : - tee-11 u5-glove/u5-glove-loss : - tee-11 u6/reference : - tee-12 u6/resolution : - tee-12 u6/hobbs : - tee-13 u7/classification : - tee-14 u7/sentiment : - tee-14 u7/evaluation : - tee-14 u7/spelling : - tee-15 u7/translation : - tee-16 u7/qa : - tee-17 u7/extraction : - tee-18 u7/retrieval : - tee-18 written\_only : m2 tee : u3-smoothing-extensions/add-k : This worked lesson was added after the video plan. No matching video has been generated or reviewed. u3-smoothing-extensions/interpolation : This worked lesson was added after the video plan. No matching video has been generated or reviewed. u3-smoothing-extensions/stupid-backoff : This worked lesson was added after the video plan. No matching video has been generated or reviewed. u4-forward-cky/forward-probability : This worked lesson was added after the video plan. No matching video has been generated or reviewed. u4-forward-cky/cnf : This worked lesson was added after the video plan. No matching video has been generated or reviewed. u4-forward-cky/cky : This worked lesson was added after the video plan. No matching video has been generated or reviewed. service\_limit : status : blocked reset\_local : 2026-10-03T22:06:00+05:30 second\_account\_reset\_local : 2026-10-03T22:51:00+05:30 remaining\_not\_submitted : 12 ### M2 01 — NLP basics, counts and conditional probability id : nlp-m2-01-foundations-counts-probability title : M2 01 — NLP basics, counts and conditional probability title\_confirmed : false topic\_title : NLP basics, counts and conditional probability mode : m2 kind : focused status : queued duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### Bayes Rule & Models id : nlp-m2-02-bayes-generative-discriminative title : Bayes Rule & Models title\_confirmed : true topic\_title : Bayes rule and generative versus discriminative models mode : m2 kind : focused status : available duration : 10:51 reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 03: N-grams & Unigrams id : nlp-m2-03-ngrams-language-models title : NLP M2 03: N-grams & Unigrams title\_confirmed : true topic\_title : N-grams, chain rule and unigram models mode : m2 kind : focused status : available duration : 9:52 reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### Bigram & Trigram Probs id : nlp-m2-04-bigram-trigram-tables title : Bigram & Trigram Probs title\_confirmed : true topic\_title : Build bigram and trigram probabilities from a corpus mode : m2 kind : focused status : available duration : 10:05 reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### M2 05: Laplace Smoothing id : nlp-m2-05-laplace-smoothing-discount title : M2 05: Laplace Smoothing title\_confirmed : true topic\_title : Laplace smoothing, unseen sequences and adjusted counts mode : m2 kind : focused status : available duration : 11:11 reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### M2 06 — Perplexity, logarithms and calculator working id : nlp-m2-06-perplexity-logs-calculator title : M2 06 — Perplexity, logarithms and calculator working title\_confirmed : false topic\_title : Perplexity, logarithms and calculator working mode : m2 kind : focused status : queued duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — POS tagging and Brill | Reviewed explanation and worked rule id : nlp-m2-07-pos-tags-tagger-methods title : NLP M2 — POS tagging and Brill | Reviewed explanation and worked rule title\_confirmed : true topic\_title : POS tags, ambiguity and tagging methods mode : m2 kind : focused status : available duration : 9:06 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Tagger methods, Brill net gain and the full conditional rule; clarified tagging and imperative explanations. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### M2 08: Markov Paths & HMMs id : nlp-m2-08-markov-hmm-components title : M2 08: Markov Paths & HMMs title\_confirmed : true topic\_title : Markov paths, HMM components and training counts mode : m2 kind : focused status : available duration : 9:45 reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### HMM Path Probabilities id : nlp-m2-09-hmm-joint-paths title : HMM Path Probabilities title\_confirmed : true topic\_title : Complete HMM path scores and observation probabilities mode : m2 kind : focused status : available duration : 8:34 reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### M2 10 Viterbi Decoding id : nlp-m2-10-viterbi-decoding title : M2 10 Viterbi Decoding title\_confirmed : true topic\_title : Viterbi decoding with backpointers mode : m2 kind : focused status : available duration : 9:34 reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — Chunking and Chinking | Reviewed rules and worked examples id : nlp-m2-11-chunking-chinking title : NLP M2 — Chunking and Chinking | Reviewed rules and worked examples title\_confirmed : true topic\_title : Chunking and chinking from POS-tagged words mode : m2 kind : focused status : available duration : 8:22 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Correct POS labels including JJ = adjective, full NP rule application and chinking boundaries. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — CFG and top-down parsing | Reviewed derivations and trees id : nlp-m2-12-cfg-top-down-parsing title : NLP M2 — CFG and top-down parsing | Reviewed derivations and trees title\_confirmed : true topic\_title : CFG, ambiguity and top-down parsing mode : m2 kind : focused status : available duration : 8:54 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Complete grammars, derivations and valid parse trees; corrected the acceptance condition. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### M2 13 — Shift-reduce parsing and backtracking id : nlp-m2-13-bottom-up-shift-reduce title : M2 13 — Shift-reduce parsing and backtracking title\_confirmed : false topic\_title : Shift-reduce parsing and backtracking mode : m2 kind : focused status : queued duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### M2 14 — Named entities, boundaries and recognition approaches id : nlp-m2-14-named-entity-recognition title : M2 14 — Named entities, boundaries and recognition approaches title\_confirmed : false topic\_title : Named entities, boundaries and recognition approaches mode : m2 kind : focused status : queued duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### M2 15 — Lexical meaning, semantic relations and WordNet id : nlp-m2-15-lexical-meaning-wordnet title : M2 15 — Lexical meaning, semantic relations and WordNet title\_confirmed : false topic\_title : Lexical meaning, semantic relations and WordNet mode : m2 kind : focused status : queued duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### M2 16 WSD and Lesk Scoring id : nlp-m2-16-lesk-wsd title : M2 16 WSD and Lesk Scoring title\_confirmed : true topic\_title : Word-sense disambiguation and Lesk overlap scoring mode : m2 kind : focused status : available duration : 10:17 reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### M2 17 — Walker category scoring and graph-based WSD id : nlp-m2-17-walker-graph-wsd title : M2 17 — Walker category scoring and graph-based WSD title\_confirmed : false topic\_title : Walker category scoring and graph-based WSD mode : m2 kind : focused status : queued duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### Naive Bayes WSD Numerical id : nlp-m2-18-naive-bayes-wsd title : Naive Bayes WSD Numerical title\_confirmed : true topic\_title : Naive Bayes WSD: a complete corpus numerical mode : m2 kind : focused status : available duration : 11:35 reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### TEE 01 — NLP foundations: language units, tasks and learning approaches id : tee-01 title : TEE 01 — NLP foundations: language units, tasks and learning approaches title\_confirmed : false topic\_title : NLP foundations: language units, tasks and learning approaches mode : tee kind : focused status : queued duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLU, NLG, the NLP pipeline and language challenges id : tee-02 title : NLU, NLG, the NLP pipeline and language challenges title\_confirmed : false topic\_title : NLU, NLG, the NLP pipeline and language challenges mode : tee kind : focused status : not\_generated duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### Preprocessing and morphology: tokenize, normalize, stem, lemmatize, count TTR id : tee-03 title : Preprocessing and morphology: tokenize, normalize, stem, lemmatize, count TTR title\_confirmed : false topic\_title : Preprocessing and morphology: tokenize, normalize, stem, lemmatize, count TTR mode : tee kind : focused status : not\_generated duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### Vocabulary, label IDs, one-hot encoding and bag of words id : tee-04 title : Vocabulary, label IDs, one-hot encoding and bag of words title\_confirmed : false topic\_title : Vocabulary, label IDs, one-hot encoding and bag of words mode : tee kind : focused status : not\_generated duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### TEE 05 — TF-IDF and cosine: from the corpus to the final similarity id : tee-05 title : TEE 05 — TF-IDF and cosine: from the corpus to the final similarity title\_confirmed : false topic\_title : TF-IDF and cosine: from the corpus to the final similarity mode : tee kind : focused status : queued duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### TEE 06 — Jaccard and distances: intersection, union, Manhattan, Euclidean, Minkowski id : tee-06 title : TEE 06 — Jaccard and distances: intersection, union, Manhattan, Euclidean, Minkowski title\_confirmed : false topic\_title : Jaccard and distances: intersection, union, Manhattan, Euclidean, Minkowski mode : tee kind : focused status : queued duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### Corpora and lexical resources: where NLP evidence comes from id : tee-07 title : Corpora and lexical resources: where NLP evidence comes from title\_confirmed : false topic\_title : Corpora and lexical resources: where NLP evidence comes from mode : tee kind : focused status : not\_generated duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### Regular expressions: build and explain exact text patterns id : tee-08 title : Regular expressions: build and explain exact text patterns title\_confirmed : false topic\_title : Regular expressions: build and explain exact text patterns mode : tee kind : focused status : not\_generated duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### TEE 09 — Distributional semantics, co-occurrence matrices and sentence vectors id : tee-09 title : TEE 09 — Distributional semantics, co-occurrence matrices and sentence vectors title\_confirmed : false topic\_title : Distributional semantics, co-occurrence matrices and sentence vectors mode : tee kind : focused status : queued duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### TEE 10 — Word2Vec: CBOW and skip-gram, training pairs and softmax numericals id : tee-10 title : TEE 10 — Word2Vec: CBOW and skip-gram, training pairs and softmax numericals title\_confirmed : false topic\_title : Word2Vec: CBOW and skip-gram, training pairs and softmax numericals mode : tee kind : focused status : queued duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### GloVe: global counts, probability ratios and weighted loss id : tee-11 title : GloVe: global counts, probability ratios and weighted loss title\_confirmed : false topic\_title : GloVe: global counts, probability ratios and weighted loss mode : tee kind : focused status : not\_generated duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### Reference and coreference: mentions, antecedents and pronoun resolution id : tee-12 title : Reference and coreference: mentions, antecedents and pronoun resolution title\_confirmed : false topic\_title : Reference and coreference: mentions, antecedents and pronoun resolution mode : tee kind : focused status : not\_generated duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### Hobbs algorithm: parse-tree traversal with three complete traces id : tee-13 title : Hobbs algorithm: parse-tree traversal with three complete traces title\_confirmed : false topic\_title : Hobbs algorithm: parse-tree traversal with three complete traces mode : tee kind : focused status : not\_generated duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### Text classification, sentiment and evaluation numericals id : tee-14 title : Text classification, sentiment and evaluation numericals title\_confirmed : false topic\_title : Text classification, sentiment and evaluation numericals mode : tee kind : focused status : not\_generated duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### TEE 15 — Spelling correction: full edit-distance tables and noisy-channel ranking id : tee-15 title : TEE 15 — Spelling correction: full edit-distance tables and noisy-channel ranking title\_confirmed : false topic\_title : Spelling correction: full edit-distance tables and noisy-channel ranking mode : tee kind : focused status : queued duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### Machine translation: meaning preservation and worked processing id : tee-16 title : Machine translation: meaning preservation and worked processing title\_confirmed : false topic\_title : Machine translation: meaning preservation and worked processing mode : tee kind : focused status : not\_generated duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### Question answering: identify, retrieve and justify the answer id : tee-17 title : Question answering: identify, retrieve and justify the answer title\_confirmed : false topic\_title : Question answering: identify, retrieve and justify the answer mode : tee kind : focused status : not\_generated duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### Information extraction and retrieval: records, indexes and precision/recall id : tee-18 title : Information extraction and retrieval: records, indexes and precision/recall title\_confirmed : false topic\_title : Information extraction and retrieval: records, indexes and precision/recall mode : tee kind : focused status : not\_generated duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — N-grams and Laplace | Reviewed worked numericals id : overview-ngrams-laplace title : NLP M2 — N-grams and Laplace | Reviewed worked numericals title\_confirmed : true topic\_title : NLP M2: N-Grams & Laplace mode : m2 kind : supplemental status : available duration : 7:41 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Complete counts and four Laplace factors; corrected formulas, conditional-factor wording and rounding. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids : - nlp-m2-03-ngrams-language-models - nlp-m2-04-bigram-trigram-tables - nlp-m2-05-laplace-smoothing-discount These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — HMM and Viterbi | Reviewed worked numericals id : overview-hmm-viterbi title : NLP M2 — HMM and Viterbi | Reviewed worked numericals title\_confirmed : true topic\_title : HMM Prob & Viterbi mode : m2 kind : supplemental status : available duration : 8:30 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Complete HMM factors, all Viterbi candidate scores and backpointers; clarified zero emissions. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids : - nlp-m2-08-markov-hmm-components - nlp-m2-09-hmm-joint-paths - nlp-m2-10-viterbi-decoding These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — Word-sense disambiguation | Reviewed Lesk and Bayes numericals id : overview-wsd title : NLP M2 — Word-sense disambiguation | Reviewed Lesk and Bayes numericals title\_confirmed : true topic\_title : WSD: Detective Math Toolkit mode : m2 kind : supplemental status : available duration : 6:11 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Lesk variants, conditional Bayes definitions, smoothing and both worked posteriors; corrected spoken claims. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids : - nlp-m2-16-lesk-wsd - nlp-m2-17-walker-graph-wsd - nlp-m2-18-naive-bayes-wsd These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — Probability and Bayes | Reviewed step-by-step numericals id : overview-probability-bayes title : NLP M2 — Probability and Bayes | Reviewed step-by-step numericals title\_confirmed : true topic\_title : NLP M2 — Probability and Bayes | Reviewed step-by-step numericals mode : m2 kind : supplemental status : available duration : 8:53 reviewed : true notebook\_url : youtube\_url corrected\_visuals : true upload\_pending\_correction : true correction\_note : Complete HMM and Naive Bayes calculations; clarified joint scores versus model posteriors. Full transcript and detected slides checked; short voice corrections included. The corrected YouTube upload is still processing. related\_topic\_ids : - nlp-m2-01-foundations-counts-probability - nlp-m2-02-bayes-generative-discriminative These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — Perplexity | Reviewed formulas and solved numericals id : overview-perplexity-logs title : NLP M2 — Perplexity | Reviewed formulas and solved numericals title\_confirmed : true topic\_title : NLP M2 — Perplexity | Reviewed formulas and solved numericals mode : m2 kind : supplemental status : available duration : 9:35 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Readable formulas and all worked examples; corrected comparison conditions and context claims. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids : - nlp-m2-06-perplexity-logs-calculator These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · What NLP is doing Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/foundations--language/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/foundations/language Shared language and probability foundations · TEE **Natural Language Processing (NLP)** is the study of computational methods for analysing, interpreting and producing human language. An NLP system receives text or speech and produces an output such as a tag, a meaning, a translation or a probability. A central question in NLP is: **how do counts from a training corpus become probabilities, and how do those probabilities help choose a sentence, a tag or a word sense?** You will learn the vocabulary first, then do the arithmetic with complete working. Training textexamples already observed → Countshow often things occur → Probabilitiescounts divided by totals → Decisioncompare possible answers | Term | Meaning | Concrete example | | --- | --- | --- | | Corpus | A collection of language data. The plural is corpora. | Three sentences saved together form a tiny corpus. | | Document | One text item within a collection. | One review or one article. | | Token | One occurrence produced by tokenization. | “cats like cats” has 3 word tokens. | | Type | A distinct token value. | The same text has 2 types: cats and like. | | Vocabulary | The set of token types recognised by the model. | V = {cats, like}; vocabulary size is 2. | | Training | Estimating rules or numbers using examples. | Count how often like follows cats. | | Test | Applying the learned model to a new example. | Calculate a probability for a sentence not used to build the counts. | ### Three levels you will repeatedly see | Level | Question it answers | Example | | --- | --- | --- | | Word sequence | Which word is likely next? | After “drink”, is “water” likely? | | Syntax | What role or structure does the word have? | “book” is a noun in “a book” and a verb in “book tickets”. | | Semantics | Which meaning is intended? | “bank” near “deposit” means a financial institution. | **Yaad rakho:** Word ka order = language model; word ka role = POS; word ka matlab = sense. ### Your TEE route Read these foundations, then continue through Units 1–7 in order. TEE includes preprocessing and vectors, corpora and regular expressions, language models, syntax, semantics, discourse and applications. The dedicated formula lessons and eight recovered previous papers follow the same methods. Original ambiguities are explained beside the affected answers. Latest TEE pattern:100 marks in three hours; Q1 compulsory plus any four of Q2–Q7. Each full question carries20 marks. Historical December2023 ends at Q6; its choices are labelled separately. The M2 tab remains restricted to your confirmed midterm portion. [Open the complete paper-by-paper map, original marks and full answers →](https://deep-learning-study-room.vercel.app/read/nlp/tee/about/pyq-analysis--all-papers/) ### Prepare the question forms seen in the latest papers first The 2 December 2025 final and 9 February 2026 re-exam both combine theory with applied work. Use their complete questions in [Solved papers](https://deep-learning-study-room.vercel.app/read/nlp/tee/index/papers/); the two Timed papers are those actual papers, not new predictions. - **Repeated calculations:** add-one bigram sentence probability (Q2b in both); HMM emission/transition tables and the joint probability of a specified path (Q4b in both); co-occurrence vectors, cosine and a plot (final Q3b / re-exam Q5b). - **Other observed forms:** TF–IDF matrix (final Q5a); BoW and pairwise cosines (re-exam Q4a); Naive Bayes WSD (final Q5b); Lesk with supplied glosses (re-exam Q3b). - **Theory and diagrams remain essential:** definitions, comparisons, preprocessing, corpus uses, CFG trees, Hobbs traversal, embeddings and application workflows. An explanation question is not automatically a demand for a long calculation. Other faculty topics remain in the full syllabus route, but their authored exercises do not establish past-paper marks or frequency. [Optional extensions](https://deep-learning-study-room.vercel.app/read/nlp/tee/about-chapter/optional-extensions/) can wait until the actual paper methods are secure. ### M2 01 — NLP basics, counts and conditional probability id : nlp-m2-01-foundations-counts-probability title : M2 01 — NLP basics, counts and conditional probability title\_confirmed : false topic\_title : NLP basics, counts and conditional probability mode : m2 kind : focused status : queued duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — Probability and Bayes | Reviewed step-by-step numericals id : overview-probability-bayes title : NLP M2 — Probability and Bayes | Reviewed step-by-step numericals title\_confirmed : true topic\_title : NLP M2 — Probability and Bayes | Reviewed step-by-step numericals mode : m2 kind : supplemental status : available duration : 8:53 reviewed : true notebook\_url : youtube\_url corrected\_visuals : true upload\_pending\_correction : true correction\_note : Complete HMM and Naive Bayes calculations; clarified joint scores versus model posteriors. Full transcript and detected slides checked; short voice corrections included. The corrected YouTube upload is still processing. related\_topic\_ids : - nlp-m2-01-foundations-counts-probability - nlp-m2-02-bayes-generative-discriminative These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Tokens, counts and symbols Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/foundations--notation/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/foundations/notation Shared language and probability foundations · TEE A formula is a short way of stating a calculation. First name the quantities; only then insert the numbers. In this guide, C means a count and P means a probability. | Notation | Read it as | What to do | | --- | --- | --- | | $$ C(\text{cat})=3 $$ | Count of cat is three | Count every occurrence, including repeats. | | $$ P(\text{cat}) $$ | Probability of cat | A number from 0 to 1. | | $$ P(\text{cat}\mid\text{the}) $$ | Probability of cat given the | Restrict attention to occasions where the history is “the”. | | $$ \sum_i x_i $$ | Sum of the x values | Add the terms: x₁ + x₂ + … | | $$ \prod_i x_i $$ | Product of the x values | Multiply the terms: x₁ × x₂ × … | | $$ \arg\max_c s(c) $$ | The choice c with the largest score | Return a label, not the numeric score. | | $$ \max_c s(c) $$ | Largest score | Return a number. | ### Example 1 · tokens and types For the corpus “we study nlp” and “we like nlp”, find the token count, vocabulary and count of “we”. **Given** Two lowercase sentences. Ignore punctuation; do not add boundary markers. **To find** Token count N, vocabulary V, vocabulary size |V| and C(we). ### Formula and why it applies A token is an occurrence; a type is a distinct word. Add sentence lengths to count tokens; remove duplicates only when finding the vocabulary. ### Solution #### Step 1. List every occurrence we, study, nlp, we, like, nlp. #### Step 2. Count tokens $$ N=3+3=6 $$ #### Step 3. Keep one copy of each type V = {we, study, nlp, like}; |V| = 4. #### Step 4. Count the requested word “we” appears once in each sentence, hence C(we) = 2. N = 6; |V| = 4; C(we) = 2. **Yaad rakho:** Token mein repeat gin; type mein duplicate hata. ### Example 2 · sum and product Given probabilities 0.5, 0.2 and 0.1, compute their sum and their product. **Given** Three numbers: 0.5, 0.2, 0.1. **To find** Sum and product; recognise that these are different operations. ### Formula and why it applies $$ \sum_{i=1}^{3}p_i=p_1+p_2+p_3 $$ $$ \prod_{i=1}^{3}p_i=p_1p_2p_3 $$ Add for a sum. Multiply for a product. A sentence-chain calculation normally uses the product of its conditional probabilities. ### Solution #### Step 1. Add the first two numbers $$ 0.5+0.2=0.7 $$ #### Step 2. Complete the sum $$ 0.7+0.1=0.8 $$ #### Step 3. Multiply the first two numbers $$ 0.5\times0.2=0.1 $$ #### Step 4. Complete the product $$ 0.1\times0.1=0.01 $$ Sum = 0.8; product = 0.01. ### M2 01 — NLP basics, counts and conditional probability id : nlp-m2-01-foundations-counts-probability title : M2 01 — NLP basics, counts and conditional probability title\_confirmed : false topic\_title : NLP basics, counts and conditional probability mode : m2 kind : focused status : queued duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — Probability and Bayes | Reviewed step-by-step numericals id : overview-probability-bayes title : NLP M2 — Probability and Bayes | Reviewed step-by-step numericals title\_confirmed : true topic\_title : NLP M2 — Probability and Bayes | Reviewed step-by-step numericals mode : m2 kind : supplemental status : available duration : 8:53 reviewed : true notebook\_url : youtube\_url corrected\_visuals : true upload\_pending\_correction : true correction\_note : Complete HMM and Naive Bayes calculations; clarified joint scores versus model posteriors. Full transcript and detected slides checked; short voice corrections included. The corrected YouTube upload is still processing. related\_topic\_ids : - nlp-m2-01-foundations-counts-probability - nlp-m2-02-bayes-generative-discriminative These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Probability, without skipped steps Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/foundations--probability/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/foundations/probability Shared language and probability foundations · TEE **Empirical probability** estimates how likely an event is from its observed frequency. Divide the number of relevant observations by the total number of observations in the selected group. For a conditional probability, the denominator is the count of the condition, not the size of the entire corpus. $$ P(A)=\frac{C(A)}{N} $$ $$ P(B\mid A)=\frac{C(A,B)}{C(A)} $$ Here A is the conditioning event, B is the event being predicted, C(A,B) counts observations containing both in the required relationship, and N is the total number of observations. “Given A” tells you which denominator to use. Select the condition“given A” → Count matching outcomesC(A,B) → Divide by condition countC(A) ### Example 1 · word probability A corpus contains 20 tokens; “rain” occurs 5 times. Find the unigram probability of “rain”. **Given** N = 20 and C(rain) = 5. **To find** P(rain). ### Formula and why it applies $$ P(w)=\frac{C(w)}{N} $$ Unigram probability ignores neighbouring words. Every token in the corpus is one observation. ### Solution #### Step 1. Insert the count and total $$ P(\text{rain})=\frac{5}{20} $$ #### Step 2. Divide $$ \frac{5}{20}=0.25 $$ P(rain) = 0.25, or 25%. ### Example 2 · conditional probability In a corpus, “drink” is followed by another word 8 times. It is followed by “water” 6 times. Find P(water | drink). **Given** C(drink as a history) = 8; C(drink, water) = 6. **To find** Probability that the next word is water, given drink. ### Formula and why it applies $$ P(\text{water}\mid\text{drink})=\frac{C(\text{drink water})}{C(\text{drink})} $$ The condition is drink, so use the 8 drink-histories as the denominator. ### Solution #### Step 1. Identify the favourable continuations Six of the eight continuations are water. #### Step 2. Divide $$ P(\text{water}\mid\text{drink})=\frac68=0.75 $$ P(water | drink) = 0.75. **Yaad rakho:** Bar ke right mein jo hai, denominator uska count. ### Example 3 · multiply a sequence A model gives P(a | start) = 0.4, P(cat | a) = 0.5 and P(end | cat) = 0.2. Find the probability of the complete sentence “a cat”. **Given** Start → a = 0.4; a → cat = 0.5; cat → end = 0.2. **To find** Probability of the three-transition path. ### Formula and why it applies $$ P(\text{a cat, end})=P(\text{a}\mid\text{start})P(\text{cat}\mid\text{a})P(\text{end}\mid\text{cat}) $$ Every transition must happen in sequence, so multiply the model’s conditional probabilities. ### Solution #### Step 1. Write the three factors in order $$ P=0.4\times0.5\times0.2 $$ #### Step 2. Multiply the first pair $$ 0.4\times0.5=0.2 $$ #### Step 3. Multiply by the last factor $$ 0.2\times0.2=0.04 $$ Complete-sentence probability = 0.04. ### Example 4 · compare labels Two tag candidates have scores noun = 0.036 and verb = 0.048. Which tag does argmax return? If these are the only candidates, find the normalised noun probability. **Given** s(noun) = 0.036 and s(verb) = 0.048. **To find** Winning label and normalised noun probability. ### Formula and why it applies $$ \hat c=\arg\max_c s(c) $$ $$ P(c)=\frac{s(c)}{\sum_j s(j)} $$ A score need not sum to 1. Dividing each score by the sum gives a distribution over the candidates. ### Solution #### Step 1. Compare scores 0.048 > 0.036, so verb wins. #### Step 2. Find their total $$ 0.036+0.048=0.084 $$ #### Step 3. Divide the noun score by the total $$ P(\text{noun})=\frac{0.036}{0.084}=\frac37\approx0.428571 $$ argmax returns “verb”. The normalised noun probability is approximately 0.428571. ### M2 01 — NLP basics, counts and conditional probability id : nlp-m2-01-foundations-counts-probability title : M2 01 — NLP basics, counts and conditional probability title\_confirmed : false topic\_title : NLP basics, counts and conditional probability mode : m2 kind : focused status : queued duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — Probability and Bayes | Reviewed step-by-step numericals id : overview-probability-bayes title : NLP M2 — Probability and Bayes | Reviewed step-by-step numericals title\_confirmed : true topic\_title : NLP M2 — Probability and Bayes | Reviewed step-by-step numericals mode : m2 kind : supplemental status : available duration : 8:53 reviewed : true notebook\_url : youtube\_url corrected\_visuals : true upload\_pending\_correction : true correction\_note : Complete HMM and Naive Bayes calculations; clarified joint scores versus model posteriors. Full transcript and detected slides checked; short voice corrections included. The corrected YouTube upload is still processing. related\_topic\_ids : - nlp-m2-01-foundations-counts-probability - nlp-m2-02-bayes-generative-discriminative These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Bayes’ rule: prior, evidence and answer Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/foundations--bayes/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/foundations/bayes Shared language and probability foundations · TEE **Bayes’ rule** reverses a conditional probability. A word-sense classifier asks “which sense is most likely given these context words?” Training data more easily supplies “how likely are these words when this sense is used?” Bayes connects the two. $$ P(c\mid x)=\frac{P(x\mid c)P(c)}{P(x)} $$ | Symbol | Meaning | | --- | --- | | c | A candidate class, tag or word sense. | | x | The observed evidence, such as a context word. | | P(c) | Prior: how common the class is before seeing the evidence. | | P(x | c) | Likelihood: how compatible the evidence is with that class. | | P(c | x) | Posterior: probability of the class after observing the evidence. | | P(x) | Evidence normaliser, shared by all classes for the same observation. | For ranking classes, calculate **prior × likelihood**. The common denominator does not affect which class is largest. For an actual posterior probability, divide by the sum of all class scores. ### A complete Bayes calculation A word has two senses: financial bank and river bank. Priors are 0.6 and 0.4. The probability of observing “loan” is 0.5 under the financial sense and 0.1 under the river sense. Find the most likely sense and its posterior. **Given** P(F) = 0.6; P(R) = 0.4; P(loan | F) = 0.5; P(loan | R) = 0.1. These two senses are exhaustive. **To find** Scores and P(F | loan). ### Formula and why it applies $$ s(c)=P(c)P(\text{loan}\mid c) $$ $$ P(F\mid\text{loan})=\frac{s(F)}{s(F)+s(R)} $$ Multiply prior and likelihood to reverse the conditioning. Then normalise because a posterior was requested. ### Solution #### Step 1. Score the financial sense $$ s(F)=0.6\times0.5=0.30 $$ #### Step 2. Score the river sense $$ s(R)=0.4\times0.1=0.04 $$ #### Step 3. Add the scores $$ P(\text{loan})=0.30+0.04=0.34 $$ #### Step 4. Normalise the financial score $$ P(F\mid\text{loan})=\frac{0.30}{0.34}=\frac{15}{17}\approx0.882353 $$ Financial bank wins; its posterior probability is about 0.882353. **Yaad rakho:** Pehle se kitna common × evidence kitna suit karta = score. ### Bayes Rule & Models id : nlp-m2-02-bayes-generative-discriminative title : Bayes Rule & Models title\_confirmed : true topic\_title : Bayes rule and generative versus discriminative models mode : m2 kind : focused status : available duration : 10:51 reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — Probability and Bayes | Reviewed step-by-step numericals id : overview-probability-bayes title : NLP M2 — Probability and Bayes | Reviewed step-by-step numericals title\_confirmed : true topic\_title : NLP M2 — Probability and Bayes | Reviewed step-by-step numericals mode : m2 kind : supplemental status : available duration : 8:53 reviewed : true notebook\_url : youtube\_url corrected\_visuals : true upload\_pending\_correction : true correction\_note : Complete HMM and Naive Bayes calculations; clarified joint scores versus model posteriors. Full transcript and detected slides checked; short voice corrections included. The corrected YouTube upload is still processing. related\_topic\_ids : - nlp-m2-01-foundations-counts-probability - nlp-m2-02-bayes-generative-discriminative These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Small probabilities, logs and calculator use Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/foundations--calculator/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/foundations/calculator Shared language and probability foundations · TEE Repeated multiplication of probabilities produces small numbers. A calculator may show 0.0000125 as **1.25 × 10⁻⁵**; they are the same value. A logarithm turns a product into a sum. Log probabilities are often negative because probabilities between 0 and 1 have negative logs. $$ \log(ab)=\log a+\log b $$ $$ \log_b x=y\quad\Longleftrightarrow\quad b^y=x $$ Use the log base specified in the question. Natural log is ln (base e); ordinary calculator log is usually base 10. Perplexity is simplest to evaluate using the direct power formula. Do not mix bases in the same calculation. ### Calculator example · roots in perplexity A model assigns probability 1/16 to a test sequence with 4 predicted tokens. Calculate perplexity. **Given** P = 1/16; N = 4 predicted tokens. **To find** Perplexity PP. ### Formula and why it applies $$ PP=P^{-1/N}=\left(\frac1P\right)^{1/N} $$ Perplexity takes the reciprocal probability and then its Nth root. N must match the number of predicted events in the product. ### Solution #### Step 1. Take the reciprocal $$ \frac{1}{1/16}=16 $$ #### Step 2. Take the fourth root $$ PP=16^{1/4}=2 $$ #### Step 3. Interpret A perplexity of 2 means an effective average uncertainty comparable to two equally likely alternatives. It is not 2% error. PP = 2. ### Calculator entry order Enter the full product with parentheses. For the Nth root enter **(1 ÷ P)^(1 ÷ N)**. Keep at least 6 significant digits while working; round the final answer. If your calculator offers Ans, reuse the stored value instead of typing a heavily rounded result. **Yaad rakho:** Probability ke factors multiply; log mein add; perplexity mein ulta karke root. ### M2 06 — Perplexity, logarithms and calculator working id : nlp-m2-06-perplexity-logs-calculator title : M2 06 — Perplexity, logarithms and calculator working title\_confirmed : false topic\_title : Perplexity, logarithms and calculator working mode : m2 kind : focused status : queued duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — Perplexity | Reviewed formulas and solved numericals id : overview-perplexity-logs title : NLP M2 — Perplexity | Reviewed formulas and solved numericals title\_confirmed : true topic\_title : NLP M2 — Perplexity | Reviewed formulas and solved numericals mode : m2 kind : supplemental status : available duration : 9:35 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Readable formulas and all worked examples; corrected comparison conditions and context claims. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids : - nlp-m2-06-perplexity-logs-calculator These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · What natural language processing actually does Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/u1-introduction--u1-nlp/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/u1-introduction/u1-nlp TEE · Unit 1 full **Natural language processing (NLP) is the field concerned with computational methods for analysing, interpreting and generating human language.** Natural languages such as English, Hindi and Marathi evolved through human use; their meanings are not specified by the rigid rules of a programming language. An NLP system may receive text, speech converted to text, or another linguistic representation. Its output depends on the task: a category, a tag for every word, a translated sentence, a ranked document list or a generated answer. It does not always “understand” in the human sense; its performance must be evaluated for the specific task. | Task | Input | Output | | --- | --- | --- | | Sentiment analysis | “The service was excellent.” | Positive sentiment | | Text classification | An email | Spam / not spam | | POS tagging | “Birds fly.” | Birds/NOUN fly/VERB | | Named entity recognition | “Asha joined Google in Mumbai.” | Asha: person; Google: organisation; Mumbai: location | | Machine translation | A Hindi sentence | A corresponding English sentence | | Information retrieval | A search query | Ranked relevant documents | | Question answering | Question plus knowledge/context | An answer | | Summarization | A long article | A shorter text preserving key information | ### How NLP relates to AI, ML and DL **AI** is the broader study of systems performing tasks associated with intelligent behaviour. **Machine learning** learns patterns from data. **Deep learning** uses multiple trainable neural layers. NLP is an application/research field that can use hand-written rules, statistical ML or deep learning. Therefore, not every NLP system is a neural network. Linguistics supplies knowledge about language structure. Statistics supplies uncertainty models. Computer science supplies algorithms, storage and efficient implementation. A practical NLP system combines these ideas. NLP = human language ko computational task mein badlo: label, meaning, search ya generated sentence. ### TEE 01 — NLP foundations: language units, tasks and learning approaches id : tee-01 title : TEE 01 — NLP foundations: language units, tasks and learning approaches title\_confirmed : false topic\_title : NLP foundations: language units, tasks and learning approaches mode : tee kind : focused status : queued duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Building blocks and levels of linguistic analysis Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/u1-introduction--u1-language/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/u1-introduction/u1-language TEE · Unit 1 full **A phoneme** is a sound unit that can distinguish meanings in a language, such as the /p/ and /b/ contrast in “pat” and “bat”. A phoneme is a sound, not necessarily one alphabet letter. **A morpheme** is the smallest meaning-bearing grammatical unit: cats = cat + plural -s; unhappy = un- + happy. **A lexeme** is an abstract vocabulary item grouping its inflected forms: write, writes, wrote and writing belong to WRITE. A **phrase** is a group of words functioning together; a **clause** normally contains a predicate and its associated participants; a **sentence** expresses a complete grammatical unit. | Level | What it analyses | Concrete example | | --- | --- | --- | | Phonetics / phonology | Physical sounds / sound patterns in a language | Distinguish bat from pat | | Morphology / lexical analysis | Word structure and lexical items | un + help + ful → unhelpful | | Syntax | Grammatical arrangement and relationships | The dog chased the cat: subject, verb, object | | Semantics | Meaning of words and composed expressions | The dog is the chaser; the cat is chased | | Discourse | Connections across sentences | “Riya arrived. She smiled.” → She refers to Riya | | Pragmatics | Intended meaning in the situation | “It is cold here” may request closing the window | word forms → grammatical structure → sentence meaning → cross-sentence context → speaker intention ### Three worked interpretations **“The cats are sleeping.”** Morphology identifies cat + s as plural. Syntax connects “cats” to “are sleeping”. Semantics represents animals engaged in sleeping. **“Aman bought a laptop. It was expensive.”** Sentence-level syntax alone does not identify “it”. Discourse connects “it” to the laptop across the sentence boundary. **“Wonderful, another power cut!”** The word wonderful is positive literally, but situational context can indicate sarcasm and frustration. This is a pragmatic interpretation. The analysis levels are conceptual descriptions. Modern systems may learn several jointly rather than executing a compulsory separate module for every level. Morphology = word ke andar; syntax = sentence ka structure; semantics = meaning; discourse = sentences ka link; pragmatics = intention. ### TEE 01 — NLP foundations: language units, tasks and learning approaches id : tee-01 title : TEE 01 — NLP foundations: language units, tasks and learning approaches title\_confirmed : false topic\_title : NLP foundations: language units, tasks and learning approaches mode : tee kind : focused status : queued duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Rule-based, statistical and deep-learning approaches Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/u1-introduction--u1-approaches/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/u1-introduction/u1-approaches TEE · Unit 1 full **A rule-based system** applies explicit patterns, dictionaries or linguistic rules written by people. Example: mark an email as a candidate company message when its sender matches a known domain. Rules are transparent but can be brittle under paraphrasing. **A statistical/ML system** estimates useful patterns from examples. A supervised classifier learns from texts with known labels. Unsupervised learning finds structure without those labels, such as clusters of news articles. **A deep-learning system** learns layered representations and can model context using neural architectures. | Criterion | Rule based | Statistical ML | Deep learning | | --- | --- | --- | --- | | Main knowledge source | Expert-written rules | Features and training examples | Data and learned representations | | Example | Regex + sentiment lexicon | TF–IDF + Naive Bayes/SVM | Transformer text classifier | | Typical benefit | Interpretable, little training data | Effective with suitable features | Learns complex context | | Typical limitation | Rule maintenance; exceptions | Feature dependence; data mismatch | Compute/data needs; harder interpretation | ### Understanding the learning terms A **training example** is one input instance. A **label** is its desired output. A **feature** is an input property used by the model. **Training** estimates parameters; **inference** applies fixed learned parameters to new input. Separate training, validation and test sets prevent evaluation from merely measuring memorization of examples already seen. **Supervised example:** review → positive/negative, using labelled reviews. **Unsupervised example:** cluster documents by word usage without topic labels. **Reinforcement learning:** choose actions using feedback or reward over interactions; it is not the same as training an ordinary classifier from labelled text. Rules: hum likhte hain. ML: examples se pattern. DL: representation bhi layers se seekhta hai. ### TEE 01 — NLP foundations: language units, tasks and learning approaches id : tee-01 title : TEE 01 — NLP foundations: language units, tasks and learning approaches title\_confirmed : false topic\_title : NLP foundations: language units, tasks and learning approaches mode : tee kind : focused status : queued duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · NLU versus NLG, explained through one assistant Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/u1-introduction--u1-nlu-nlg/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/u1-introduction/u1-nlu-nlg TEE · Unit 1 full **Natural language understanding (NLU)** maps language into an interpretation useful for a task: intents, entities, semantic relations or another meaning representation. **Natural language generation (NLG)** produces human-language output from data, a representation or another source. “Book a ticket to Pune tomorrow.” → NLU: intent=book; destination=Pune; date=tomorrow → Application: search available journeys → NLG: “Here are the available journeys to Pune.” | Feature | NLU | NLG | | --- | --- | --- | | Direction | Language → interpretation | Data/representation → language | | Main concern | Meaning, intent, ambiguity | Content choice, organization, fluent wording | | Example | Find destination in a request | Write the booking confirmation | | Common failure | Wrong destination or sense | Unclear, unsupported or ungrammatical output | ### NLG stages in a classical pipeline 1. **Content determination:** choose facts to communicate. 2. **Data interpretation:** identify relationships or trends. 3. **Document planning:** organize facts in a useful order. 4. **Sentence aggregation:** combine related facts without repetition. 5. **Lexical and grammatical realization:** choose words and construct grammatical sentences. 6. **Presentation:** format the generated message for its audience. For a weather report, facts {city=Pune, temperature=28°C, rain=yes} can become “Pune will have rain with a temperature of 28°C.” NLU would instead extract facts or intent from an incoming language message. NLU = language se meaning andar. NLG = data se language bahar. ### NLU, NLG, the NLP pipeline and language challenges id : tee-02 title : NLU, NLG, the NLP pipeline and language challenges title\_confirmed : false topic\_title : NLU, NLG, the NLP pipeline and language challenges mode : tee kind : focused status : not\_generated duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Generic NLP pipeline, evaluation and deployment Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/u1-introduction--u1-pipeline/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/u1-introduction/u1-pipeline TEE · Unit 1 full **An NLP pipeline is an organized sequence of operations that transforms raw language into a task output and checks whether that output is useful.** Not every task needs every cleaning operation. Data acquisition → Preprocessing → Feature representation → Model → Evaluation → Deployment and monitoring 1. **Data acquisition:** collect relevant text and, when needed, labels. 2. **Cleaning and preprocessing:** handle markup, tokenize, normalize and decide which information to retain. 3. **Feature engineering:** represent text numerically using BoW, TF–IDF or embeddings. 4. **Modelling:** apply rules or train a statistical/neural model. 5. **Evaluation:** test on held-out data using task-appropriate measures. 6. **Deployment:** make the system available, log suitable feedback and monitor errors or data changes. ### Concrete spam-filter example Collect labelled emails → separate train/test data → tokenize text while preserving meaningful terms → fit vocabulary and IDF on training data → train classifier → evaluate spam detection on the held-out set → deploy an email service filter → monitor legitimate emails wrongly blocked. **Intrinsic evaluation** measures model/task quality, such as precision, recall or tagging accuracy. **Extrinsic evaluation** measures effect in the wider application, such as whether users find answers faster. A higher internal metric does not automatically guarantee a better user outcome. **Data leakage:** learning vocabulary statistics, labels or preprocessing decisions from the test set can make reported performance too optimistic. Fit learned transformations on training data, then apply them to validation/test data. Collect → clean → numbers → model → test → use. Test data ko training ki information mat banao. ### NLU, NLG, the NLP pipeline and language challenges id : tee-02 title : NLU, NLG, the NLP pipeline and language challenges title\_confirmed : false topic\_title : NLU, NLG, the NLP pipeline and language challenges mode : tee kind : focused status : not\_generated duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Ambiguity and other challenges with precise examples Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/u1-introduction--u1-challenges/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/u1-introduction/u1-challenges TEE · Unit 1 full **Ambiguity** means an expression supports more than one interpretation. Good exam answers name the ambiguity and explain both readings. | Challenge | Example | Why a system can fail | | --- | --- | --- | | Lexical/sense ambiguity | bank: finance or river edge | One written word has several senses | | Syntactic ambiguity | I saw the man with a telescope. | The telescope may modify saw or the man | | Reference ambiguity | Riya told Neha that she won. | She may refer to either person without more context | | Sarcasm | Great, another delay. | Literal positive word conveys frustration | | Negation | The film was not good. | Dropping not reverses meaning | | Domain shift | “Positive” in a medical report | Meaning/use differs by domain | | Code mixing | Kal meeting reschedule karna. | Several languages and spelling conventions | | Noisy text | tmrw, repeated letters, typos | Vocabulary/token boundaries become uncertain | | Low-resource setting | Few labelled examples in a language | Models lack representative training evidence | | Bias and privacy | Skewed data or personal details | Errors and harms can persist in deployed systems | **Homophones** sound alike but may differ in spelling (pair/pear). **Homographs** share spelling but can differ in meaning or pronunciation (lead the team / lead metal). These are different dimensions from syntactic ambiguity. Long-distance dependencies, cultural knowledge, idioms, morphology and multilingual writing systems also complicate processing. For “kick the bucket”, literal word-by-word interpretation can miss the idiomatic meaning. For “The book that the students recommended is useful”, the verb is agrees with book across an intervening phrase. Exam mein challenge ka naam + ek sentence + dono meanings/actual failure likho. ### NLU, NLG, the NLP pipeline and language challenges id : tee-02 title : NLU, NLG, the NLP pipeline and language challenges title\_confirmed : false topic\_title : NLU, NLG, the NLP pipeline and language challenges mode : tee kind : focused status : not\_generated duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Tokenization: sentence, word, character and subword Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/u2-preprocess--u2-tokenize/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/u2-preprocess/u2-tokenize TEE · Unit 2 PDF pages 1–50 **Tokenization divides text into units called tokens for further processing.** A token may be a word, punctuation mark, character or subword, depending on the tokenizer. **Sentence segmentation** identifies sentence boundaries. Tokenization is not necessarily just splitting on spaces. | Method | Example | Trade-off | | --- | --- | --- | | Word | “cats sleep” → [cats,sleep] | Easy to interpret; unfamiliar words can be unseen tokens | | Character | cat → [c,a,t] | Handles spelling units; creates longer sequences | | Subword | unhappiness → pieces such as [un,happi,ness] in an illustrative vocabulary | Balances vocabulary size and rare-word representation | | Sentence | “Birds fly. Fish swim.” → two sentences | Abbreviations and punctuation complicate boundaries | Actual subword splits depend on the learned vocabulary; the example is illustrative, not a universal tokenizer output. Word segmentation is especially important where spaces do not reliably separate words or in compound-rich languages. **Question. Tokenize “Dr. Rao can't come today.” into words and punctuation under this rule: preserve Dr. and can't as whole tokens, and split only the final full stop.** #### Given The tokenization rule is explicitly supplied. #### To find The resulting token sequence. #### Formula and why it applies Apply the stated boundary rule; do not assume every period or apostrophe is a split point. #### Step 1. Handle abbreviation Dr. remains one token. #### Step 2. Handle contraction can't remains one token. #### Step 3. Write sequence [Dr., Rao, can't, come, today, .] **Answer.** 6 tokens under the specified rule. Another tokenizer may legitimately produce a different sequence. ### Three difficult cases **New York** may be two tokens but one named entity. **3.14** should not normally end a sentence at the decimal point. **don't** may remain whole or split into do + n't, depending on the tokenizer. Always state the convention when calculating counts. Token = chosen unit. Space split ko universal rule mat samajh. ### Preprocessing and morphology: tokenize, normalize, stem, lemmatize, count TTR id : tee-03 title : Preprocessing and morphology: tokenize, normalize, stem, lemmatize, count TTR title\_confirmed : false topic\_title : Preprocessing and morphology: tokenize, normalize, stem, lemmatize, count TTR mode : tee kind : focused status : not\_generated duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Cleaning, case folding, punctuation and stopwords Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/u2-preprocess--u2-normalization/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/u2-preprocess/u2-normalization TEE · Unit 2 PDF pages 1–50 **Normalization reduces superficial variation so comparable forms can be processed consistently.** Common operations include lowercasing, Unicode normalization, appropriate spelling correction, contraction handling, and optional stemming or lemmatization. **Stopwords** are frequent words selected for removal for a particular task, such as the or a. Removing them may reduce feature size, but removing not, no or nor can destroy sentiment meaning. Punctuation can convey sentence boundaries, emotion or mathematical structure. Lowercasing can erase differences such as US versus us. **Question. Preprocess “The CATS are not sleeping!!!” by lowercasing, removing punctuation, removing only stopwords {the,are}, and lemmatizing plural cats→cat and sleeping→sleep.** #### Given Operations and stopword set are fixed. Do not remove not. #### To find The final token list. #### Formula and why it applies Apply the requested operations in order, keeping a visible intermediate result after each transformation. #### Step 1. Lowercase the cats are not sleeping!!! #### Step 2. Tokenize and remove punctuation [the,cats,are,not,sleeping] #### Step 3. Remove specified stopwords Remove the and are → [cats,not,sleeping]. #### Step 4. Lemmatize cats→cat; sleeping→sleep → [cat,not,sleep]. **Answer.** [cat,not,sleep]. The negative meaning is retained. ### Task decides preprocessing For a simple keyword-retrieval model, removing selected function words can help. For machine translation, question answering, grammar analysis or transformer input, aggressive removal may discard necessary syntax or violate the model's expected input format. Emoji can carry sentiment; convert or retain them when relevant instead of always deleting them. Clean karna = useful signal bachake noise hatao. Har word/punctuation delete karna cleaning nahi. ### Preprocessing and morphology: tokenize, normalize, stem, lemmatize, count TTR id : tee-03 title : Preprocessing and morphology: tokenize, normalize, stem, lemmatize, count TTR title\_confirmed : false topic\_title : Preprocessing and morphology: tokenize, normalize, stem, lemmatize, count TTR mode : tee kind : focused status : not\_generated duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Stemming versus lemmatization Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/u2-preprocess--u2-stem-lemma/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/u2-preprocess/u2-stem-lemma TEE · Unit 2 PDF pages 1–50 **Stemming** applies string-based rules, often stripping suffixes, to reduce related forms to a stem. The result need not be a valid dictionary word. **Lemmatization** uses vocabulary and morphological analysis, often with POS information, to return a dictionary lemma. | Criterion | Stemming | Lemmatization | | --- | --- | --- | | Basis | Surface form/suffix rules | Lexicon and morphological/POS analysis | | Output | May be nonword, e.g. studi | Dictionary form, e.g. study | | Context/POS | Usually little or none | Often important | | Irregular forms | May miss went→go | Can map went→go with verb analysis | | Cost | Generally simpler | Usually more linguistic processing | Porter applies a sequence of suffix-removal rules subject to conditions; Snowball provides related language-specific stemming algorithms; Lancaster is often more aggressive; regex stemming uses explicit patterns; Lovins is another established stemming approach. Exact output depends on the algorithm and implementation. ### Three worked reductions 1. **studies, studying:** an English stemmer may yield studi/study depending on the rule sequence; verb lemmatization aims at study. State the exact algorithm if exact outputs are asked. 2. **better:** adjective-aware lemmatization can map it to good; naïve suffix removal cannot infer that irregular relation. 3. **saw:** as a past-tense verb it can become see; as a cutting-tool noun it remains saw. POS and context matter. **Over-stemming** merges forms that should remain distinct; **under-stemming** fails to merge related forms. Normalization can increase recall in search while losing useful distinctions, so it is a task-dependent choice. Stem = rules se cut; lemma = dictionary form, meaning/POS ka dhyan. ### Preprocessing and morphology: tokenize, normalize, stem, lemmatize, count TTR id : tee-03 title : Preprocessing and morphology: tokenize, normalize, stem, lemmatize, count TTR title\_confirmed : false topic\_title : Preprocessing and morphology: tokenize, normalize, stem, lemmatize, count TTR mode : tee kind : focused status : not\_generated duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Type–token ratio: three fully worked sums Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/u2-preprocess--u2-ttr/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/u2-preprocess/u2-ttr TEE · Unit 2 PDF pages 1–50 **A token is one occurrence; a type is a distinct token form under the chosen normalization.** In “cat cat dog”, there are three tokens and two types. Type–token ratio (TTR) measures observed lexical diversity. $$ \operatorname{TTR}=\frac{\text{number of distinct types}}{\text{number of tokens}}. $$ A larger TTR means more distinct words relative to text length in that sample. It is strongly affected by text length and preprocessing, so comparisons should use comparable samples. **Question. Find TTR for “the cat saw the cat” after lowercasing and ignoring punctuation. Keep all words.** #### Given Tokens [the,cat,saw,the,cat]. #### To find Type count, token count and TTR. #### Formula and why it applies $$ TTR=V/N $$ , where $$ V $$ counts unique forms and $$ N $$ counts all occurrences. #### Step 1. Count tokens There are 5 positions → N=5. #### Step 2. List types {the,cat,saw} → V=3. #### Step 3. Divide $$ TTR=3/5=0.6 $$ . **Answer.** TTR=0.6, or 60%. **Question. Find TTR for “Run run RUN” before and after lowercasing.** #### Given Before lowercasing, case distinguishes forms. There are 3 occurrences in either case. #### To find Two TTR values. #### Formula and why it applies $$ TTR=V/N $$ . Recount types after normalization. #### Step 1. Before case folding {Run,run,RUN} has 3 types; N=3. #### Step 2. First ratio $$ 3/3=1 $$ . #### Step 3. After case folding Tokens [run,run,run]; types {run} → V=1. #### Step 4. Second ratio $$ 1/3\approx0.333333 $$ . **Answer.** Before=1; after≈0.333333. **Question. A text has 80 tokens and 20 types. Another has 50 tokens and 25 types. Find and compare their TTRs.** #### Given Text A: N=80,V=20. Text B: N=50,V=25. #### To find Both ratios. #### Formula and why it applies $$ TTR=V/N $$ . #### Step 1. Text A $$ 20/80=0.25 $$ . #### Step 2. Text B $$ 25/50=0.5 $$ . #### Step 3. Interpret B has a larger observed type proportion, but unequal text lengths prevent treating this alone as an unbiased measure of richer language. **Answer.** A=0.25, B=0.5. Tokens = sab occurrences; types = unique names; TTR = unique ÷ total. ### Preprocessing and morphology: tokenize, normalize, stem, lemmatize, count TTR id : tee-03 title : Preprocessing and morphology: tokenize, normalize, stem, lemmatize, count TTR title\_confirmed : false topic\_title : Preprocessing and morphology: tokenize, normalize, stem, lemmatize, count TTR mode : tee kind : focused status : not\_generated duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Vocabulary and vectors—what each coordinate means Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/u2-representation--u2-vocab/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/u2-representation/u2-vocab TEE · Unit 2 PDF pages 51–92 **A vocabulary** is the ordered collection of distinct terms allowed by a representation. Its order defines what every coordinate means. A **vector** is simply an ordered number-list. A **document–term matrix** puts one document in each row and one vocabulary term in each column. With vocabulary [cat,dog,fish], the count vector [2,0,1] means cat appears twice, dog zero times and fish once. With [dog,cat,fish], the same text becomes [0,2,1]. A vector without its vocabulary order is incomplete. | Document | cat | dog | fish | | --- | --- | --- | --- | | cat cat fish | 2 | 0 | 1 | | dog fish | 0 | 1 | 1 | **Nominal categories** have no inherent order, such as cities. **Ordinal categories** do have an order, such as low < medium < high. Assigning arbitrary word IDs does not create meaningful numerical distances between words. Vector ka har box ek fixed word ka hai. Pehle column order fix, phir numbers. ### Vocabulary, label IDs, one-hot encoding and bag of words id : tee-04 title : Vocabulary, label IDs, one-hot encoding and bag of words title\_confirmed : false topic\_title : Vocabulary, label IDs, one-hot encoding and bag of words mode : tee kind : focused status : not\_generated duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Label and one-hot encoding: three complete examples Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/u2-representation--u2-label-onehot/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/u2-representation/u2-label-onehot TEE · Unit 2 PDF pages 51–92 **Label encoding** assigns each category or word a unique integer ID. **One-hot encoding** assigns a length- $$ V $$ vector with one 1 at the word's vocabulary position and zeros elsewhere. $$ \operatorname{onehot}(w)_j=\begin{cases}1,&j=\operatorname{index}(w)\\0,&\text{otherwise.}\end{cases} $$ The index is an ID, not a meaning score. One-hot represents one token/category; a multiword document generally needs multiple token vectors or an aggregation scheme. **Question. Encode “dog cat dog” using vocabulary [cat,dog,fish], IDs starting at 0. Give label IDs and a one-hot vector for every token.** #### Given cat→0, dog→1, fish→2. Vector columns have the same order. #### To find The ID sequence and three one-hot rows. #### Formula and why it applies Replace each word by its ID for labels; set its vocabulary position to 1 for one-hot. #### Step 1. Label IDs dog→1, cat→0, dog→1 → [1,0,1]. #### Step 2. First dog [0,1,0]. #### Step 3. Cat [1,0,0]. #### Step 4. Second dog [0,1,0]. Repeated tokens keep the same representation. **Answer.** IDs [1,0,1]; one-hot rows [0,1,0], [1,0,0], [0,1,0]. **Question. Encode the city sequence Mumbai, Delhi, Mumbai, Pune using vocabulary [Delhi,Mumbai,Pune] and label IDs starting at 1.** #### Given Delhi→1, Mumbai→2, Pune→3. #### To find IDs and one-hot rows. #### Formula and why it applies One-hot uses three coordinates regardless of whether label IDs begin at 0 or 1. #### Step 1. IDs [2,1,2,3]. #### Step 2. Mumbai rows Each Mumbai → [0,1,0]. #### Step 3. Delhi and Pune Delhi→[1,0,0]; Pune→[0,0,1]. **Answer.** Rows [0,1,0], [1,0,0], [0,1,0], [0,0,1]. **Question. One-hot encode categories [red,green,blue] for a linear regression with an intercept. Then give dummy coding after dropping blue.** #### Given Full column order [red,green,blue]. Blue is the baseline. #### To find Full and reduced codes. #### Formula and why it applies Full one-hot rows sum to 1. With an intercept this creates a linear dependency; dropping one category column gives k−1 independent dummy indicators. #### Step 1. Full encoding | Category | red | green | blue | | --- | --- | --- | --- | | red | 1 | 0 | 0 | | green | 0 | 1 | 0 | | blue | 0 | 0 | 1 | #### Step 2. Drop blue column | Category | red | green | | --- | --- | --- | | red | 1 | 0 | | green | 0 | 1 | | blue | 0 | 0 | #### Step 3. Interpret zero row [0,0] now identifies the baseline blue, not missing data. **Answer.** Full width 3; reduced dummy width 2. Dropping a column is model-dependent, not a requirement for every neural embedding input. **Limitations:** one-hot is large and sparse and does not express semantic similarity. Unseen words need a defined unknown-token or vocabulary-update policy. Label IDs are compact but should not be treated as an ordered scalar feature for nominal words. Label = ID number. One-hot = us ID ke box mein 1, baaki 0. ### Vocabulary, label IDs, one-hot encoding and bag of words id : tee-04 title : Vocabulary, label IDs, one-hot encoding and bag of words title\_confirmed : false topic\_title : Vocabulary, label IDs, one-hot encoding and bag of words mode : tee kind : focused status : not\_generated duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Bag of words and bag of n-grams Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/u2-representation--u2-bow/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/u2-representation/u2-bow TEE · Unit 2 PDF pages 51–92 **Bag of words (BoW) represents a document by the counts of vocabulary terms, discarding their positions.** A binary BoW variant stores only presence/absence; a count BoW stores frequencies. State which is used. $$ x_{d,j}=C(v_j,d). $$ $$ x_{d,j} $$ is document $$ d $$ 's coordinate for vocabulary term $$ v_j $$ . $$ C(v_j,d) $$ is its occurrence count. A **bag of n-grams** uses short contiguous token sequences as features. Bigrams can distinguish “not good” from “good”, but increasing n enlarges the vocabulary and creates more rare features. **Question. Build count and binary BoW for D1=“cat cat dog” and D2=“dog fish”. Use vocabulary [cat,dog,fish] and no stopword removal.** #### Given Two tokenized documents, fixed vocabulary order. #### To find Count vectors and binary vectors. #### Formula and why it applies Count BoW uses $$ C(v_j,d) $$ . Binary BoW is 1 when that count is positive and 0 otherwise. #### Step 1. D1 counts cat=2, dog=1, fish=0 → [2,1,0]. #### Step 2. D2 counts cat=0, dog=1, fish=1 → [0,1,1]. #### Step 3. Convert to presence D1 [1,1,0]; D2 [0,1,1]. **Answer.** Count rows [2,1,0], [0,1,1]; binary rows [1,1,0], [0,1,1]. **Question. After lowercasing and removing {is,a,means}, obtain BoW vectors for D1=“Continuous learning is learning lifetime”, D2=“Continuous learning means lifetime learning”, D3=“Learning is a good practice”. Use [continuous,good,learning,lifetime,practice].** #### Given Three documents and exact stopword set from the supplied slides. #### To find The document–term count matrix. #### Formula and why it applies Remove only the listed stopwords, then count every remaining token in the fixed column order. #### Step 1. Clean D1 [continuous,learning,learning,lifetime] → [1,0,2,1,0]. #### Step 2. Clean D2 [continuous,learning,lifetime,learning] → [1,0,2,1,0]. #### Step 3. Clean D3 [learning,good,practice] → [0,1,1,0,1]. #### Step 4. Compare D1 and D2 They have identical BoW vectors despite different original wording/order. **Answer.** Rows [1,0,2,1,0], [1,0,2,1,0], [0,1,1,0,1]. **Question. Build a bag of bigrams for “not good but not bad” with vocabulary [not good,good but,but not,not bad]. Do not add sentence boundary tokens.** #### Given Five tokens; use adjacent pairs, stride 1. #### To find The bigram count vector. #### Formula and why it applies A sequence of $$ L $$ tokens has $$ L-1 $$ adjacent bigrams when no boundaries are added. #### Step 1. Enumerate pairs (not,good), (good,but), (but,not), (not,bad). #### Step 2. Count each vocabulary bigram Each appears once. #### Step 3. Write vector [1,1,1,1]. **Answer.** Four bigram features, each with count 1. BoW: order bhool, word-count rakh. Bag of bigrams: neighbouring word-pair count. ### Vocabulary, label IDs, one-hot encoding and bag of words id : tee-04 title : Vocabulary, label IDs, one-hot encoding and bag of words title\_confirmed : false topic\_title : Vocabulary, label IDs, one-hot encoding and bag of words mode : tee kind : focused status : not\_generated duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · TF–IDF from the exact slide convention Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/u2-representation--u2-tfidf/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/u2-representation/u2-tfidf TEE · Unit 2 PDF pages 51–92 **TF–IDF weights a term highly when it is frequent in a particular document but uncommon across the corpus.** The supplied Unit 2 slides use normalized term frequency and base-10 inverse document frequency: $$ \operatorname{TF}(t,d)=\frac{C(t,d)}{N_d} $$ $$ \operatorname{IDF}(t)=\log_{10}\frac{D}{\operatorname{df}(t)} $$ $$ \operatorname{TFIDF}(t,d)=\operatorname{TF}(t,d)\operatorname{IDF}(t). $$ $$ N_d $$ is the document's token count after the specified preprocessing; $$ D $$ is number of documents; $$ \operatorname{df}(t) $$ is the number of documents containing the term at least once. A word repeated ten times in one document contributes one to document frequency. **Why the log?** It compresses the ratio's scale. If a term occurs in all documents, $$ D/df=1 $$ , so IDF=0 under this unsmoothed convention. Other libraries use smoothed IDF, different TF or normalization. Use the question's formula; do not silently substitute a library default. **Question. A corpus contains 10 documents. The word model appears in 2 documents and occurs 3 times in a 12-token document. Compute TF, IDF and TF–IDF using the slide convention.** #### Given D=10, df(model)=2, C(model,d)=3, Nd=12; log base 10. #### To find The TF–IDF weight. #### Formula and why it applies $$ TF=C/N_d $$ , $$ IDF=\log_{10}(D/df) $$ , then multiply. #### Step 1. TF $$ 3/12=0.25 $$ . #### Step 2. Document ratio $$ 10/2=5 $$ . #### Step 3. IDF $$ \log_{10}5\approx0.698970 $$ . #### Step 4. Weight $$ 0.25\times0.698970\approx0.174743 $$ . **Answer.** TF=0.25, IDF≈0.698970, TF–IDF≈0.174743. **Question. A term occurs 4 times in a 20-token document and is present in all 8 corpus documents. Calculate its TF–IDF.** #### Given C=4, Nd=20, D=8, df=8. #### To find Its weight under unsmoothed base-10 IDF. #### Formula and why it applies $$ TF=C/N_d $$ , $$ IDF=\log_{10}(D/df) $$ . #### Step 1. TF $$ 4/20=0.2 $$ . #### Step 2. IDF $$ \log_{10}(8/8)=\log_{10}1=0 $$ . #### Step 3. Multiply $$ 0.2\times0=0 $$ . **Answer.** Weight 0: this term does not distinguish documents under this convention. **Question. Find TF–IDF vectors for D1=“cat cat dog”, D2=“dog fish”, D3=“cat fish fish”, using [cat,dog,fish], normalized TF and log10 IDF.** #### Given No further preprocessing. Each term occurs in exactly two of the three documents. #### To find All three vectors. #### Formula and why it applies First count df by documents, then compute each document's TF, then multiply by its term's IDF. #### Step 1. Common IDF $$ D=3 $$ , $$ df(cat)=df(dog)=df(fish)=2 $$ . Each IDF $$ =\log_{10}(3/2)\approx0.176091 $$ . #### Step 2. D1 term frequencies Length 3 → $$ [2/3,1/3,0] $$ . #### Step 3. D1 weights $$ [(2/3)0.176091,(1/3)0.176091,0]\approx[0.117394,0.058697,0] $$ . #### Step 4. D2 weights Length 2; TF $$ [0,1/2,1/2] $$ . Vector $$ \approx[0,0.088046,0.088046] $$ . #### Step 5. D3 weights Length 3; TF $$ [1/3,0,2/3] $$ . Vector $$ \approx[0.058697,0,0.117394] $$ . **Answer.** D1≈[0.117394,0.058697,0]; D2≈[0,0.088046,0.088046]; D3≈[0.058697,0,0.117394]. TF: iss document mein kitna. IDF: poore corpus mein kitna rare. Dono multiply. ### TEE 05 — TF-IDF and cosine: from the corpus to the final similarity id : tee-05 title : TEE 05 — TF-IDF and cosine: from the corpus to the final similarity title\_confirmed : false topic\_title : TF-IDF and cosine: from the corpus to the final similarity mode : tee kind : focused status : queued duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Combined slide numerical: TF–IDF followed by cosine Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/u2-representation--u2-slide-combined/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/u2-representation/u2-slide-combined TEE · Unit 2 PDF pages 51–92 **Question. For D1=“Continuous learning is learning lifetime”, D2=“Continuous learning means lifetime learning”, D3=“Learning is a good practice”, find the TF–IDF cosine similarity of D1 and D2. Lowercase and remove {is,a,means}. Use normalized TF and log10 IDF.** #### Given Vocabulary [continuous,good,learning,lifetime,practice]. D1 and D2 each have 4 retained tokens; D3 has 3. #### To find TF–IDF vectors and cosine(D1,D2). #### Formula and why it applies $$ TF=C/N_d $$ , $$ IDF=\log_{10}(D/df) $$ ; $$ \cos(x,y)=(x\cdot y)/(\|x\|\|y\|) $$ . First build the vectors; cosine cannot be applied directly to words. #### Step 1. Count documents containing each term | Term | df | IDF | | --- | --- | --- | | continuous | 2 | log10(3/2)≈0.176091 | | good | 1 | log10(3)≈0.477121 | | learning | 3 | log10(1)=0 | | lifetime | 2 | ≈0.176091 | | practice | 1 | ≈0.477121 | #### Step 2. Term frequencies D1=D2=[1/4,0,2/4,1/4,0]. D3=[0,1/3,1/3,0,1/3]. #### Step 3. Multiply TF by IDF Let $$ a=\tfrac14\log_{10}(3/2)\approx0.0440228 $$ . Then D1=D2=[a,0,0,a,0]. D3≈[0,0.159040,0,0,0.159040]. #### Step 4. Dot product $$ x\cdot y=a^2+0+0+a^2+0=2a^2 $$ . #### Step 5. Vector lengths $$ \|x\|=\|y\|=\sqrt{2a^2} $$ . Their product is $$ 2a^2 $$ . #### Step 6. Divide $$ \cos(x,y)=2a^2/(2a^2)=1 $$ . **Answer.** Cosine(D1,D2)=1. Their BoW Euclidean distance is also 0 because their counts are identical. The learning coordinate becomes zero because learning occurs in every document. D1–D3 cosine is 0 under this exact TF–IDF convention: their only originally shared word was learning, whose IDF is zero. This is a consequence of the model, not proof that the sentences have unrelated meanings. Combined sum: pehle cleaned words → counts → TF-IDF → similarity. Beech ka stage skip mat kar. ### TEE 05 — TF-IDF and cosine: from the corpus to the final similarity id : tee-05 title : TEE 05 — TF-IDF and cosine: from the corpus to the final similarity title\_confirmed : false topic\_title : TF-IDF and cosine: from the corpus to the final similarity mode : tee kind : focused status : queued duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Cosine: products, lengths and division Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/u2-similarity--u2-cosine/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/u2-similarity/u2-cosine Unit 2 · PDF pages 93–103 **Cosine similarity compares the direction of two nonzero vectors.** Multiply matching coordinates and add to obtain the dot product. To obtain each vector's norm, square its coordinates, add them and take a square root. $$ \cos(x,y)=\frac{\sum_i x_i y_i}{\sqrt{\sum_i x_i^2}\sqrt{\sum_i y_i^2}}. $$ The vectors must use the same coordinate/vocabulary order. For nonnegative BoW or TF–IDF vectors, the result lies from 0 to 1; for general real-valued vectors, it can range from −1 to 1. A zero vector makes this formula undefined. **Question. Find cosine similarity for x=[1,1] and y=[1,0].** #### Given Nonzero vectors, matching coordinate order. #### To find Find cosine similarity for x=[1,1] and y=[1,0]. #### Formula and why it applies Use $$ \cos(x,y)=(x\cdot y)/(\|x\|\|y\|) $$ because direction similarity is required. #### Step 1. Multiply matching coordinates 1 × 1 = 1; 1 × 0 = 0. #### Step 2. Add products 1 + 0 = 1. #### Step 3. Length of x $$ \sqrt{(1)^2+(1)^2}=\sqrt{2}. $$ #### Step 4. Length of y $$ \sqrt{(1)^2+(0)^2}=\sqrt{1}. $$ #### Step 5. Divide $$ \frac{1}{\sqrt{2}\sqrt{1}}\approx 0.707107. $$ **Answer.** Cosine≈0.707107. **Question. Find cosine similarity for x=[1,2] and y=[2,4].** #### Given Nonzero vectors, matching coordinate order. #### To find Find cosine similarity for x=[1,2] and y=[2,4]. #### Formula and why it applies Use $$ \cos(x,y)=(x\cdot y)/(\|x\|\|y\|) $$ because direction similarity is required. #### Step 1. Multiply matching coordinates 1 × 2 = 2; 2 × 4 = 8. #### Step 2. Add products 2 + 8 = 10. #### Step 3. Length of x $$ \sqrt{(1)^2+(2)^2}=\sqrt{5}. $$ #### Step 4. Length of y $$ \sqrt{(2)^2+(4)^2}=\sqrt{20}. $$ #### Step 5. Divide $$ \frac{10}{\sqrt{5}\sqrt{20}}\approx 1.000000. $$ **Answer.** Cosine≈1.000000. **Question. Find cosine similarity for x=[1,2,0] and y=[0,1,2].** #### Given Nonzero vectors, matching coordinate order. #### To find Find cosine similarity for x=[1,2,0] and y=[0,1,2]. #### Formula and why it applies Use $$ \cos(x,y)=(x\cdot y)/(\|x\|\|y\|) $$ because direction similarity is required. #### Step 1. Multiply matching coordinates 1 × 0 = 0; 2 × 1 = 2; 0 × 2 = 0. #### Step 2. Add products 0 + 2 + 0 = 2. #### Step 3. Length of x $$ \sqrt{(1)^2+(2)^2+(0)^2}=\sqrt{5}. $$ #### Step 4. Length of y $$ \sqrt{(0)^2+(1)^2+(2)^2}=\sqrt{5}. $$ #### Step 5. Divide $$ \frac{2}{\sqrt{5}\sqrt{5}}\approx 0.400000. $$ **Answer.** Cosine≈0.400000. Proportional vectors [1,2] and [2,4] have cosine 1 but nonzero Euclidean distance. Direction similarity does not require identical magnitudes. Cosine: matching products add upar; dono lengths multiply neeche. ### TEE 05 — TF-IDF and cosine: from the corpus to the final similarity id : tee-05 title : TEE 05 — TF-IDF and cosine: from the corpus to the final similarity title\_confirmed : false topic\_title : TF-IDF and cosine: from the corpus to the final similarity mode : tee kind : focused status : queued duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Jaccard: intersection, union and distance Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/u2-similarity--u2-jaccard/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/u2-similarity/u2-jaccard Unit 2 · PDF pages 93–103 **Jaccard similarity is the fraction of shared distinct elements among all distinct elements in either set.** $$ J(A,B)=\frac{|A\cap B|}{|A\cup B|}. $$ Intersection means common elements; union means all distinct elements. Repetitions do not count again. Jaccard distance is $$ 1-J $$ , not the same quantity as similarity. Two empty sets require a declared convention because the fraction becomes 0/0. **Question. Find Jaccard similarity and distance for A={cat, dog} and B={dog, fish}.** #### Given Sets contain unique elements. #### To find Find Jaccard similarity and distance for A={cat, dog} and B={dog, fish}. #### Formula and why it applies $$ J=|A\cap B|/|A\cup B| $$ ; $$ d=1-J $$ . #### Step 1. Find intersection {dog}; count 1. #### Step 2. Find union {cat, dog, fish}; count 3. #### Step 3. Divide J=1/3≈0.333333. #### Step 4. Subtract from one Distance=1−0.333333≈0.666667. **Answer.** Similarity≈0.333333; distance≈0.666667. **Question. Find Jaccard similarity and distance for A={red, blue} and B={red, blue}.** #### Given Sets contain unique elements. #### To find Find Jaccard similarity and distance for A={red, blue} and B={red, blue}. #### Formula and why it applies $$ J=|A\cap B|/|A\cup B| $$ ; $$ d=1-J $$ . #### Step 1. Find intersection {red, blue}; count 2. #### Step 2. Find union {red, blue}; count 2. #### Step 3. Divide J=2/2≈1.000000. #### Step 4. Subtract from one Distance=1−1.000000≈0.000000. **Answer.** Similarity≈1.000000; distance≈0.000000. **Question. Find Jaccard similarity and distance for A={apple} and B={orange, pear}.** #### Given Sets contain unique elements. #### To find Find Jaccard similarity and distance for A={apple} and B={orange, pear}. #### Formula and why it applies $$ J=|A\cap B|/|A\cup B| $$ ; $$ d=1-J $$ . #### Step 1. Find intersection {}; count 0. #### Step 2. Find union {apple, orange, pear}; count 3. #### Step 3. Divide J=0/3≈0.000000. #### Step 4. Subtract from one Distance=1−0.000000≈1.000000. **Answer.** Similarity≈0.000000; distance≈1.000000. Jaccard = common unique ÷ total unique. Distance = 1 minus similarity. ### TEE 06 — Jaccard and distances: intersection, union, Manhattan, Euclidean, Minkowski id : tee-06 title : TEE 06 — Jaccard and distances: intersection, union, Manhattan, Euclidean, Minkowski title\_confirmed : false topic\_title : Jaccard and distances: intersection, union, Manhattan, Euclidean, Minkowski mode : tee kind : focused status : queued duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Manhattan, Euclidean and Minkowski: three complete comparisons Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/u2-similarity--u2-distances/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/u2-similarity/u2-distances Unit 2 · PDF pages 93–103 **Distance measures separation; smaller means closer.** Absolute values prevent positive and negative differences from cancelling. $$ d_1=\sum_i|x_i-y_i|\quad\text{Manhattan} $$ $$ d_2=\sqrt{\sum_i(x_i-y_i)^2}\quad\text{Euclidean} $$ $$ d_p=\left(\sum_i|x_i-y_i|^p\right)^{1/p},\quad p\ge1\quad\text{Minkowski}. $$ Manhattan is p=1; Euclidean is p=2. For p=3, cube absolute differences, add, then take a cube root. **Question. Find Manhattan, Euclidean and Minkowski p=3 distances for x=[1,2], y=[4,6].** #### Given Vectors share coordinate order; negative inputs are allowed. #### To find Find Manhattan, Euclidean and Minkowski p=3 distances for x=[1,2], y=[4,6]. #### Formula and why it applies Use $$ d_1=\sum|\Delta_i| $$ , $$ d_2=\sqrt{\sum|\Delta_i|^2} $$ , $$ d_3=\sqrt[3]{\sum|\Delta_i|^3} $$ . #### Step 1. Absolute differences |1 − (4)| = 3; |2 − (6)| = 4. #### Step 2. Manhattan 3 + 4 = 7. #### Step 3. Square and add 3²=9; 4²=16. Sum=25. #### Step 4. Euclidean root $$ \sqrt{25}\approx 5.000000. $$ #### Step 5. Cube and add 3³=27; 4³=64. Sum=91. #### Step 6. Minkowski root $$ \sqrt[3]{91}\approx 4.497941. $$ **Answer.** Manhattan=7; Euclidean≈5.000000; p=3≈4.497941. **Question. Find Manhattan, Euclidean and Minkowski p=3 distances for x=[0,1,2], y=[2,1,0].** #### Given Vectors share coordinate order; negative inputs are allowed. #### To find Find Manhattan, Euclidean and Minkowski p=3 distances for x=[0,1,2], y=[2,1,0]. #### Formula and why it applies Use $$ d_1=\sum|\Delta_i| $$ , $$ d_2=\sqrt{\sum|\Delta_i|^2} $$ , $$ d_3=\sqrt[3]{\sum|\Delta_i|^3} $$ . #### Step 1. Absolute differences |0 − (2)| = 2; |1 − (1)| = 0; |2 − (0)| = 2. #### Step 2. Manhattan 2 + 0 + 2 = 4. #### Step 3. Square and add 2²=4; 0²=0; 2²=4. Sum=8. #### Step 4. Euclidean root $$ \sqrt{8}\approx 2.828427. $$ #### Step 5. Cube and add 2³=8; 0³=0; 2³=8. Sum=16. #### Step 6. Minkowski root $$ \sqrt[3]{16}\approx 2.519842. $$ **Answer.** Manhattan=4; Euclidean≈2.828427; p=3≈2.519842. **Question. Find Manhattan, Euclidean and Minkowski p=3 distances for x=[-1,3], y=[2,-1].** #### Given Vectors share coordinate order; negative inputs are allowed. #### To find Find Manhattan, Euclidean and Minkowski p=3 distances for x=[-1,3], y=[2,-1]. #### Formula and why it applies Use $$ d_1=\sum|\Delta_i| $$ , $$ d_2=\sqrt{\sum|\Delta_i|^2} $$ , $$ d_3=\sqrt[3]{\sum|\Delta_i|^3} $$ . #### Step 1. Absolute differences |-1 − (2)| = 3; |3 − (-1)| = 4. #### Step 2. Manhattan 3 + 4 = 7. #### Step 3. Square and add 3²=9; 4²=16. Sum=25. #### Step 4. Euclidean root $$ \sqrt{25}\approx 5.000000. $$ #### Step 5. Cube and add 3³=27; 4³=64. Sum=91. #### Step 6. Minkowski root $$ \sqrt[3]{91}\approx 4.497941. $$ **Answer.** Manhattan=7; Euclidean≈5.000000; p=3≈4.497941. Manhattan: absolute-add. Euclidean: square-add-root. Minkowski: p-power-add-p-root. ### TEE 06 — Jaccard and distances: intersection, union, Manhattan, Euclidean, Minkowski id : tee-06 title : TEE 06 — Jaccard and distances: intersection, union, Manhattan, Euclidean, Minkowski title\_confirmed : false topic\_title : Jaccard and distances: intersection, union, Manhattan, Euclidean, Minkowski mode : tee kind : focused status : queued duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Choose the representation before the metric Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/u2-similarity--u2-choice/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/u2-similarity/u2-choice Unit 2 · PDF pages 93–103 | Question | Measure | Closer means | | --- | --- | --- | | Compare vector direction | Cosine similarity | Higher | | Compare term sets | Jaccard similarity | Higher | | Straight-line separation | Euclidean distance | Lower | | Sum coordinate changes | Manhattan distance | Lower | | General p specified | Minkowski distance | Lower | BoW and TF–IDF may give different similarities. First show preprocessing, vocabulary and vectors; then use the metric. Arbitrary label IDs do not define meaningful word distances. Never compare vectors using different term orders. Keep intermediate precision until the final result. Representation + metric dono question ke hisaab se. Formula pe seedha jump nahi. ### TEE 06 — Jaccard and distances: intersection, union, Manhattan, Euclidean, Minkowski id : tee-06 title : TEE 06 — Jaccard and distances: intersection, union, Manhattan, Euclidean, Minkowski title\_confirmed : false topic\_title : Jaccard and distances: intersection, union, Manhattan, Euclidean, Minkowski mode : tee kind : focused status : queued duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Corpus structure and annotations Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/u3-corpora--u3-corpus/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/u3-corpora/u3-corpus Unit 3 · PDF pages 1–61; pronouncing dictionary from current syllabus **A corpus is a collection of language data assembled for study or computational use.** Its plural is corpora. A corpus can contain books, news, reviews or conversations. It need not be labelled. **Annotation** adds linguistic information such as POS tags, entities, parse structures or sense labels. | Structure | Meaning | Use | | --- | --- | --- | | Plain | Collection without category labels | Word counts, language modelling | | Categorized | Documents grouped by genre/topic | Classification | | Overlapping categories | One document has several categories | Multi-topic news | | Temporal | Documents indexed by time | Language change | | Annotated | Linguistic labels attached | Tagging, parsing, WSD | | Parallel | Aligned texts in different languages | Translation | A **tagset** is the allowed label inventory. A **treebank** is a corpus annotated with syntax trees. **Metadata** records source, date, genre, language, annotation method and limitations. A common benchmark supports comparison under shared data and evaluation conditions. **Raw example:** “Riya works in Pune.” **POS annotation:** Riya/PROPN works/VERB in/ADP Pune/PROPN. **NER annotation:** Riya/PERSON, Pune/LOCATION. Each annotation answers a different task. A good corpus is relevant to its domain, sufficiently representative, consistently annotated and appropriately split. More data alone do not repair duplicated examples, incorrect labels or a mismatch between training and real use. Corpus=text collection. Annotation=text ke saath labels. Metadata=collection ki information. ### Corpora and lexical resources: where NLP evidence comes from id : tee-07 title : Corpora and lexical resources: where NLP evidence comes from title\_confirmed : false topic\_title : Corpora and lexical resources: where NLP evidence comes from mode : tee kind : focused status : not\_generated duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Gutenberg, web/chat, Brown, Reuters and Inaugural Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/u3-corpora--u3-types/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/u3-corpora/u3-types Unit 3 · PDF pages 1–61; pronouncing dictionary from current syllabus | Resource | Contents | Typical use | | --- | --- | --- | | Gutenberg selection | Literary/book texts | Long-form language and vocabulary | | Web text | Informal domain-specific text | Study usage beyond edited books | | Chat corpus | Conversational posts | Dialogue, informal forms | | Brown | English texts across genres | Genre comparison | | Reuters | News with topic labels | Text classification; overlapping topics | | Inaugural | Speeches indexed by time | Vocabulary change across years | | Wordlist | Words/names/stopwords | Lexical filtering | | Pronouncing dictionary | Words and phonetic forms | Speech, stress, rhyme | **Gutenberg versus web/chat:** book language does not represent every spelling or conversational pattern in messages. **Brown versus Reuters:** Brown emphasizes genres; Reuters news can have overlapping topic labels. **Inaugural addresses:** dates provide a time dimension. Corpus sizes depend on version, subset and tokenization. Explain their purpose and structure rather than assuming a historical count is a permanent property. Gutenberg=books; web/chat=informal; Brown=genres; Reuters=news topics; Inaugural=time. ### Corpora and lexical resources: where NLP evidence comes from id : tee-07 title : Corpora and lexical resources: where NLP evidence comes from title\_confirmed : false topic\_title : Corpora and lexical resources: where NLP evidence comes from mode : tee kind : focused status : not\_generated duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Raw text, tokens and sentences are different views Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/u3-corpora--u3-access/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/u3-corpora/u3-access Unit 3 · PDF pages 1–61; pronouncing dictionary from current syllabus Corpus readers expose several views. **raw** returns a character string, **words** returns token occurrences, and **sents** returns a list of sentences where each sentence contains tokens. | Conceptual method | Returned view | Its length counts | | --- | --- | --- | | fileids() | Document IDs | Files | | raw(fileid) | Character string | Characters | | words(fileid) | Token list | Token occurrences | | sents(fileid) | Sentence lists | Sentences | | categories() | Genre/topic labels | Categories | | tagged\_words(fileid) | Word–tag pairs | Annotated tokens | **Question. For sentence lists [the,cat,sleeps,.] and [the,dog,runs,.], count all tokens, sentences and lowercase alphabetic word types.** #### Given Punctuation is included in token count but excluded from word types. #### To find For sentence lists [the,cat,sleeps,.] and [the,dog,runs,.], count all tokens, sentences and lowercase alphabetic word types. #### Formula and why it applies Count every token occurrence, then distinct alphabetic forms. Different counting rules give different totals. #### Step 1. Token count 4+4=8. #### Step 2. Sentence count Two outer lists → 2. #### Step 3. Types {the,cat,sleeps,dog,runs} → 5. **Answer.** 8 tokens, 2 sentences, 5 word types. For “Cats run. Dogs sleep.”, a reader may return six tokens including two periods, but only four alphabetic word tokens. State the convention before calculating TTR or probabilities. raw=characters, words=tokens, sents=sentences. Teen alag counts. ### Corpora and lexical resources: where NLP evidence comes from id : tee-07 title : Corpora and lexical resources: where NLP evidence comes from title\_confirmed : false topic\_title : Corpora and lexical resources: where NLP evidence comes from mode : tee kind : focused status : not\_generated duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Wordlists, pronunciation and semantic resources Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/u3-corpora--u3-resources/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/u3-corpora/u3-resources Unit 3 · PDF pages 1–61; pronouncing dictionary from current syllabus **A wordlist** stores lexical entries without necessarily giving usage context. A stopword list gives task-specific filtering terms. Names lists can support lexical analysis but cannot reliably infer a real person's identity or gender. **A pronouncing dictionary** maps a written word to one or more phonetic symbol sequences. CMU-style dictionaries use ARPAbet; vowel stress suffixes distinguish unstressed, primary-stressed and secondary-stressed vowels. A spelling can have more than one pronunciation, as “read” does. | Question | Resource | Reason | | --- | --- | --- | | Which words rhyme with night? | Pronouncing dictionary | Compare sounds, not just spelling | | Which terms should be filtered? | Specified stopword list | Apply task's filtering convention | | What broader concept includes car? | WordNet | Follow hypernym relation | | How frequently does car occur? | Text corpus | Count actual usage occurrences | **WordNet** groups word senses in synsets, supplies glosses and connects meanings by semantic relations. Its full treatment is in Unit 5. A corpus supplies usage evidence; a lexical resource supplies organized knowledge. A system can combine both. Sound→pronouncing dictionary. Meaning→WordNet. Occurrence→corpus. ### Corpora and lexical resources: where NLP evidence comes from id : tee-07 title : Corpora and lexical resources: where NLP evidence comes from title\_confirmed : false topic\_title : Corpora and lexical resources: where NLP evidence comes from mode : tee kind : focused status : not\_generated duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Acquiring data and preserving labels during augmentation Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/u3-corpora--u3-acquisition/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/u3-corpora/u3-acquisition Unit 3 · PDF pages 1–61; pronouncing dictionary from current syllabus **Data acquisition** obtains language examples relevant to the task from existing data, public resources or appropriate new collection. Product instrumentation can gather useful feedback. Supervised learning additionally needs an annotation process when labels are absent. **Data augmentation** creates training variants intended to preserve the correct label. It is not simply changing words arbitrarily. | Method | Procedure | Caution | | --- | --- | --- | | Synonym replacement | Use suitable synonyms for selected words | Choose the right sense | | Back translation | Translate to another language and back | May change meanings/entities | | TF–IDF-informed replacement | Protect informative words while varying others | Do not remove label-bearing clues | | Bigram flipping | Swap adjacent words | Can break grammar or meaning | | Entity replacement | Replace by same-type entity | Must preserve the task label | | Noise injection | Simulate typos/keyboard errors | Match expected noise | | Active learning | Select informative cases for human labels | This acquires labels; it is not synthetic text | ### Three label decisions “The film is good”→“The film is excellent” may preserve a positive sentiment label. “The film is not good”→“The film is good” reverses the label and is not valid label-preserving augmentation. “Flights to Pune”→“Flights to Mumbai” may preserve a travel-topic label but changes the destination label. Split train/test data before creating related augmented variants. Otherwise near-duplicates can leak between sets and produce misleading evaluation. Keep a representative evaluation set of genuine examples. Sentence badal sakta hai; required correct label nahi. Label badla toh augmentation invalid. ### Corpora and lexical resources: where NLP evidence comes from id : tee-07 title : Corpora and lexical resources: where NLP evidence comes from title\_confirmed : false topic\_title : Corpora and lexical resources: where NLP evidence comes from mode : tee kind : focused status : not\_generated duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · How corpus evidence powers later units Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/u3-corpora--u3-links/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/u3-corpora/u3-links Unit 3 · PDF pages 1–61; pronouncing dictionary from current syllabus Word-sequence counts support n-gram language models and smoothing. POS-annotated sentences support supervised taggers. Sense-labelled contexts support Naive Bayes WSD. Word-context counts support distributional semantics and GloVe. Parallel aligned sentences support translation. For a question asking the importance of corpora, give three points: empirical language evidence, examples for training/evaluation, and reproducible comparison. For an annotated-corpus question, define the added labels and give a concrete example. For resource comparisons, distinguish running text, lexical lists, pronunciation entries and semantic relations. Same text se task ke hisaab se counts, tags, senses ya context patterns nikaalte hain. ### Corpora and lexical resources: where NLP evidence comes from id : tee-07 title : Corpora and lexical resources: where NLP evidence comes from title\_confirmed : false topic\_title : Corpora and lexical resources: where NLP evidence comes from mode : tee kind : focused status : not\_generated duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Characters, classes, anchors and repetition Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/u3-regex--regex-language/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/u3-regex/regex-language TEE · current Unit 3 syllabus and supplied Regular Expressions handout; outside confirmed M2 page cutoff **A regular expression (regex) is a pattern language for specifying classes of text strings.** A regex engine tests or finds strings matching a pattern. It can extract numbers, locate words, clean text and implement rule-based NLP features. A match is a string satisfying the pattern; it is not an interpretation of the string's meaning. Start with a literal: `cat` matches those three consecutive letters. Without boundary constraints it also matches the cat part inside “catalog” or “wildcat”. Add restrictions only when the question asks for them. | Pattern | Meaning | Example | | --- | --- | --- | | `[aeiou]` | Exactly one listed lowercase vowel | a, e; not b | | `[A-Z]` | One uppercase ASCII letter | A through Z | | `[^0-9]` | One character not in the digit range | a or a space; not 5 | | `\d` | One digit under the regex engine's character rules | Use [0-9] if only ASCII digits are intended | | `\w` | Word character, usually letters/digits/underscore | Unicode details depend on engine | | `\s` | Whitespace character | Space, tab, newline | | `\D, \W, \S` | Complement classes | Non-digit, non-word, non-whitespace | | `.` | One arbitrary character except newline by default | DOTALL flag can include newlines | | `\.` | Literal full stop | Escaping removes the wildcard meaning | | `^` | Start of string/line depending on mode | ^The | | `$` | End anchor under the engine's rules | Spain$ | | `\b` | Boundary between word and non-word positions | Whole-word matching | | `*` | Zero or more repetitions of previous item | ab\* matches a, ab, abb | | `+` | One or more | ab+ requires at least one b | | `?` | Zero or one | colou?r allows color and colour | | `{m}, {m,n}, {m,}` | Exact, bounded or minimum repetitions | [0-9]{3} | | `(...)` | Group a sequence | (ab)+ repeats the pair | | `|` | Choose an alternative | cat|dog | **Critical distinction:** a character class chooses one character, whereas alternatives can choose whole strings. `[cat]` matches one c, a or t; `(cat|dog)` matches either complete word. A quantifier acts on the immediately preceding atom/group. `abc*` means ab followed by any number of c; `(abc)*` repeats the entire abc group. Regex left-to-right padho: kaunsa character, kitni baar, aur start/end/word boundary kya hai? ### Regular expressions: build and explain exact text patterns id : tee-08 title : Regular expressions: build and explain exact text patterns title\_confirmed : false topic\_title : Regular expressions: build and explain exact text patterns mode : tee kind : focused status : not\_generated duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Search, match, fullmatch, findall, substitution and flags Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/u3-regex--regex-functions/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/u3-regex/regex-functions TEE · current Unit 3 syllabus and supplied Regular Expressions handout; outside confirmed M2 page cutoff | Python operation | What it does | Typical use | | --- | --- | --- | | re.search(pattern,text) | Finds the first match anywhere | Locate a clue inside a sentence | | re.match(pattern,text) | Tests only from the start | Check a prefix | | re.fullmatch(pattern,text) | Requires the complete string to match | Validate an exact format | | re.findall(pattern,text) | Returns non-overlapping matches | Extract all numbers | | re.finditer(pattern,text) | Returns match objects with spans | Locate match positions | | re.sub(pattern,replacement,text) | Replaces matches | Normalize repeated whitespace | | re.split(pattern,text) | Splits at matches | Custom token boundaries | Python raw strings such as `r"\bcat\b"` keep the backslashes for the regex engine. In an ordinary Python string, `\b` is a backspace escape, so using a raw string prevents a common mistake. Capturing groups can change what `findall` returns; use a noncapturing group `(?:...)` when grouping is needed but capturing is not. ### Flags and engine differences **IGNORECASE / i** ignores case. **MULTILINE / m** lets ^ and $ refer to line boundaries. **DOTALL / s** lets dot match newline. JavaScript's **g** flag requests repeated/global matching; Python has no separate global flag for `findall`, which already finds successive non-overlapping matches. Do not mix JavaScript slash delimiters with Python's string syntax. ### Greedy versus lazy quantifiers A greedy quantifier tries to consume as much as possible while permitting the whole match; a lazy quantifier tries as little as possible. On `onetwo`, `<.*>` can match the entire string from the first < to last >. The lazy `<.*?>` returns shorter tag-shaped matches during repeated matching. Regex is useful for restricted patterns; robust arbitrary HTML processing should use an HTML parser. search=kahin bhi; match=start se; fullmatch=poora text; findall=sab matches. Greedy zyada, lazy kam. ### Regular expressions: build and explain exact text patterns id : tee-08 title : Regular expressions: build and explain exact text patterns title\_confirmed : false topic\_title : Regular expressions: build and explain exact text patterns mode : tee kind : focused status : not\_generated duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Six full regex questions and solutions Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/u3-regex--regex-worked/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/u3-regex/regex-worked TEE · current Unit 3 syllabus and supplied Regular Expressions handout; outside confirmed M2 page cutoff **Question. From “cat catalog wildcat Cat”, extract only the lowercase whole word cat using Python regex.** #### Given Case-sensitive matching; whole words only. #### To find A pattern and its matches. #### Rule and why it applies Use `\b` boundaries around the literal word. This prevents substring matches inside longer word tokens. #### Step 1. Write the literal `cat` alone would also match catalog and wildcat. #### Step 2. Add both word boundaries `r"\bcat\b"`. #### Step 3. Check candidates cat matches. catalog fails at the ending boundary. wildcat fails at the starting boundary. Cat fails because matching is case sensitive. #### Step 4. Extract `re.findall(r"\bcat\b", text)` returns `["cat"]`. **Answer.** Pattern `\bcat\b`; one match: cat. With IGNORECASE, Cat would also match. **Question. Write a regex that validates exactly ten ASCII digits, then test 9876543210, 98765 and A9876543210.** #### Given No country-prefix restriction; digits only; entire input must match. #### To find A validation pattern and each result. #### Rule and why it applies `[0-9]` permits one ASCII digit; `{10}` repeats it exactly ten times; fullmatch prevents extra characters. #### Step 1. Choose the character class `[0-9]`. #### Step 2. Specify exact length `[0-9]{10}`. #### Step 3. Require the whole input `re.fullmatch(r"[0-9]{10}", text)`. In a simple single-line exam pattern, `^[0-9]{10}$` expresses the intended format. #### Step 4. Test all examples 9876543210: 10 digits → valid. 98765: only 5 → invalid. A9876543210: contains an extra letter → invalid. **Answer.** Use fullmatch with `[0-9]{10}`. Only 9876543210 passes. This checks format, not whether a phone number exists. **Question. Extract date-shaped strings in DD/MM/YYYY format from “On 05/10/2026 and 7/10/2026 we met.” Require two digits for day and month and four for year.** #### Given Format recognition only; no calendar-validity test. #### To find Pattern and extracted strings. #### Rule and why it applies Use fixed repetition counts separated by literal slashes, and boundaries around the full pattern. #### Step 1. Day part `[0-9]{2}`. #### Step 2. Month and year `[0-9]{2}/[0-9]{2}/[0-9]{4}`. #### Step 3. Add boundaries `r"\b[0-9]{2}/[0-9]{2}/[0-9]{4}\b"`. #### Step 4. Evaluate the two candidates 05/10/2026 has 2/2/4 digits and matches. 7/10/2026 has a one-digit day and fails. **Answer.** Only 05/10/2026 is extracted. A date such as 99/99/2026 has the right shape but needs a calendar parser to reject it semantically. **Question. Use a regex to check a simple email shape: one or more username characters from letters, digits, dot, underscore, percent, plus or hyphen; @; a domain of letters, digits, dots or hyphens; a literal dot; a letter-only suffix of length at least two.** #### Given This is the simplified handout format, not full standards-compliant validation. #### To find The pattern and an explanation of every part. #### Rule and why it applies Build the requested components and use fullmatch so no extra prefix or suffix is accepted. #### Step 1. Username `[A-Za-z0-9._%+-]+`: one or more allowed username characters. The hyphen is last, so it is literal. #### Step 2. Separator and domain `@[A-Za-z0-9.-]+`: literal @ followed by domain characters. #### Step 3. Suffix `\.[A-Za-z]{2,}`: escaped dot, then at least two letters. #### Step 4. Combine and test `re.fullmatch(r"[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}", text)`. `riya+nlp@example.com` matches; `riya.example.com` lacks @ and fails. **Answer.** The combined pattern meets the stated classroom shape. It neither proves address existence nor enforces every real email/domain constraint. **Question. For text “The rain in Spain”, write patterns that test starts with The, ends with Spain, and satisfies both.** #### Given Single-line, case-sensitive text. #### To find Three patterns and outcomes. #### Rule and why it applies Use ^ for beginning, $ for ending, and .\* for the intervening sequence in this simple one-line task. #### Step 1. Starts with `^The` succeeds. #### Step 2. Ends with `Spain$` succeeds. #### Step 3. Both `^The.*Spain$` succeeds because the text begins with The and ends with Spain. #### Step 4. Why dot-star? `.` consumes one intervening character; `*` allows zero or more, including spaces here. **Answer.** All three checks succeed. Without anchors a substring match alone would not enforce the required position. **Question. Construct a case-sensitive pattern for full filenames ending in .jpg or .png and test photo.jpg, chart.png and photo.jpg.exe.** #### Given Filename must have at least one character before the dot. Treat the filename as a single line. #### To find Pattern and three results. #### Rule and why it applies Use a noncapturing alternative for the two complete suffixes and fullmatch to disallow trailing text. #### Step 1. Filename stem `.+` requires one or more characters. #### Step 2. Literal dot `\.`. #### Step 3. Choose suffix `(?:jpg|png)` chooses jpg or png. #### Step 4. Full match and test `re.fullmatch(r".+\.(?:jpg|png)", filename)`. photo.jpg passes; chart.png passes; photo.jpg.exe fails. **Answer.** Full pattern `.+\.(?:jpg|png)`; use IGNORECASE only if uppercase extensions should also be accepted. Pattern likhke ek valid, ek short/wrong, ek extra-character case zaroor mentally run karo. ### Regular expressions: build and explain exact text patterns id : tee-08 title : Regular expressions: build and explain exact text patterns title\_confirmed : false topic\_title : Regular expressions: build and explain exact text patterns mode : tee kind : focused status : not\_generated duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Pattern reference for the handout's remaining tasks Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/u3-regex--regex-reference/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/u3-regex/regex-reference TEE · current Unit 3 syllabus and supplied Regular Expressions handout; outside confirmed M2 page cutoff | Task | Pattern / operation | Read it correctly | | --- | --- | --- | | Extract hashtags | `#[A-Za-z0-9_]+` | # followed by one or more permitted tag characters | | Extract integer sequences | `[0-9]+` | Does not include decimal dots or signs | | Extract words ending ing | `\b[A-Za-z]*ing\b` | Zero or more prefix letters, then ing | | Currency amounts, simple integers | `[$₹][0-9]+` | One currency symbol followed by digits; decimals need extension | | Words containing exact uppercase AI | `\b[A-Za-z]*AI[A-Za-z]*\b` | Case-sensitive; letters before/after may be absent | | Capital-initial alphabetic words | `\b[A-Z][A-Za-z]*\b` | One capital then zero or more letters | | Extract mentions | `@[A-Za-z0-9_]+` | Simple username rule, not any platform's complete policy | | Exactly five-letter words | `\b[A-Za-z]{5}\b` | Word boundaries plus exact length | | Starts N and ends P | `\bN[A-Za-z]*P\b` | NP itself is allowed; require + to demand middle letters | | Alphabetic words only | `\b[A-Za-z]+\b` | ASCII letter-only token matches | | Basic http/https URL extraction | `https?://[^\s<>]+` | Matches a URL-shaped span; trailing punctuation may need removal | | Collapse whitespace | `re.sub(r"\s+", " ", text)` | Replace each whitespace run with one space | ### IPv4 addresses: shape versus numeric validity `(?:[0-9]{1,3}\.){3}[0-9]{1,3}` recognizes four dotted digit groups, but permits 999.999.999.999. For numeric octets 0–255 without leading zeros, define one octet as `(?:25[0-5]|2[0-4][0-9]|1[0-9]{2}|[1-9]?[0-9])`, join four with escaped dots, and require fullmatch. An IP-address parser is clearer for production validation. **Limits:** regex recognizes surface patterns; it cannot infer the intended sense of bank, establish a person's identity from capitalization, or decide arbitrary grammatical meaning. A date/email/URL-shaped match is not automatically a valid or real-world existing item. Regex shape verify karta hai; real meaning/existence ke liye extra logic lagta hai. ### Regular expressions: build and explain exact text patterns id : tee-08 title : Regular expressions: build and explain exact text patterns title\_confirmed : false topic\_title : Regular expressions: build and explain exact text patterns mode : tee kind : focused status : not\_generated duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Words, counts and probabilities: the notation you need Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/m2-language-models--words-counts-probability/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/m2-language-models/words-counts-probability Unit 3, pages 62–108; essential probability revision **A token** is one occurrence of a word. A **type** is one distinct word. A **corpus** is a collection of texts used for analysis or training. In “cats chase cats”, there are three word tokens but only two word types: cats and chase. A language model learns regularities from such a corpus to assign probabilities to word sequences. It does not simply count whether a sentence is grammatically correct. | Symbol | Read it as | Concrete meaning | | --- | --- | --- | | C(cat) | count of cat | Number of times cat occurs. | | C(the, cat) | count of the followed by cat | Count adjacent ordered pairs; cat the is different. | | P(cat) | probability of cat | A number from 0 to 1. | | P(cat | the) | probability of cat given the | Among contexts ending in the, how often cat follows. | | V | vocabulary size | Number of distinct outcomes the model may predict. | | N | number of tokens or predictions | Its exact meaning must be stated for the calculation. | The vertical bar **| means “given”**. The condition is on its right. In P(cat | the), the model has already seen “the” and is predicting “cat”. This direction matters: P(cat | the) and P(the | cat) usually have different denominators. **Question. A corpus contains 20 word tokens. The word cat occurs 5 times. Find the maximum-likelihood unigram probability of cat.** #### Given Total tokens N = 20; count C(cat) = 5. #### To find P(cat). #### Formula and why it applies $$ P(w)=\frac{C(w)}{N} $$ Relative frequency estimates the probability of drawing this word from the corpus. The denominator includes every token, not just distinct words. #### Solution ##### Step 1. Select the matching count The required outcome is cat, which occurs 5 times. ##### Step 2. Divide by the total number of trials $$ P(\text{cat})=\frac{5}{20}=0.25 $$ ##### Step 3. Interpret the result Under this unigram model, cat receives probability 0.25, or 25%. **Final answer.** P(cat) = 0.25. **Yaad rakho:** Jo chahiye uska count upar; total possible trials neeche. **Question. The word the has 10 outgoing occurrences in a corpus. It is followed by cat on 4 occasions and dog on 6. Find P(cat | the) and P(dog | the).** #### Given C(the, cat) = 4; C(the, dog) = 6; C(the) = 10 outgoing contexts. #### To find Both next-word probabilities. #### Formula and why it applies $$ P(w\mid h)=\frac{C(h,w)}{C(h)} $$ The condition h = the restricts attention to its 10 continuations. We do not divide by the total corpus size. #### Solution ##### Step 1. Use the cat continuation $$ P(\text{cat}\mid\text{the})=\frac{4}{10}=0.4 $$ ##### Step 2. Use the dog continuation $$ P(\text{dog}\mid\text{the})=\frac{6}{10}=0.6 $$ ##### Step 3. Check the complete row $$ 0.4+0.6=1 $$ **Final answer.** P(cat | the) = 0.4 and P(dog | the) = 0.6. **Yaad rakho:** Given waala word denominator choose karata hai. **Question. A collection has 8 sentences; 6 start with I and 2 start with we. Estimate each starting-word probability.** #### Given Each sentence receives a start marker . C() = 8; C(, I) = 6; C(, we) = 2. #### To find P(I | ) and P(we | ). #### Formula and why it applies $$ P(w_1\mid\langle s\rangle)=\frac{C(\langle s\rangle,w_1)}{C(\langle s\rangle)} $$ The start marker supplies the context for the first word; one start context is available per sentence. #### Solution ##### Step 1. Calculate the first option $$ P(I\mid\langle s\rangle)=6/8=0.75 $$ ##### Step 2. Calculate the second option $$ P(we\mid\langle s\rangle)=2/8=0.25 $$ **Final answer.** The estimated probabilities are 0.75 and 0.25. ### M2 01 — NLP basics, counts and conditional probability id : nlp-m2-01-foundations-counts-probability title : M2 01 — NLP basics, counts and conditional probability title\_confirmed : false topic\_title : NLP basics, counts and conditional probability mode : m2 kind : focused status : queued duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — Probability and Bayes | Reviewed step-by-step numericals id : overview-probability-bayes title : NLP M2 — Probability and Bayes | Reviewed step-by-step numericals title\_confirmed : true topic\_title : NLP M2 — Probability and Bayes | Reviewed step-by-step numericals mode : m2 kind : supplemental status : available duration : 8:53 reviewed : true notebook\_url : youtube\_url corrected\_visuals : true upload\_pending\_correction : true correction\_note : Complete HMM and Naive Bayes calculations; clarified joint scores versus model posteriors. Full transcript and detected slides checked; short voice corrections included. The corrected YouTube upload is still processing. related\_topic\_ids : - nlp-m2-01-foundations-counts-probability - nlp-m2-02-bayes-generative-discriminative These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Unigrams, bigrams and trigrams: construct them correctly Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/m2-language-models--n-grams-construction/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/m2-language-models/n-grams-construction Unit 3, pages 62–108; essential probability revision **Definition.** An n-gram is a contiguous sequence of n items. For word n-grams, the items are consecutive word tokens. A unigram contains one word; a bigram contains two; a trigram contains three. Character n-grams use characters instead, so always identify the unit being counted. Tokens: we | study | natural | language → Move a window one token at a time → Bigrams: we study / study natural / natural language For m tokens without padding, the number of n-gram *positions* is max(0, m−n+1). Repeated positions may contain the same n-gram type. A bag-of-n-grams representation counts each n-gram type and loses the overall sentence order beyond the local order inside each n-gram. $$ \text{Number of unpadded n-gram positions}=\max(0,m-n+1) $$ **Question. List all word unigrams, bigrams and trigrams for “the prime minister of our country”. Do not add boundary tokens.** #### Given Six tokens: the, prime, minister, of, our, country. n = 1, 2 and 3. #### To find All three lists and their counts. #### Formula and why it applies $$ \#\text{positions}=m-n+1 $$ Each window must contain consecutive words and move one position. No wraparound or invented final word is allowed. #### Solution ##### Step 1. Take one word at a time Unigrams: (the), (prime), (minister), (of), (our), (country). Count = 6−1+1 = 6. ##### Step 2. Take two consecutive words Bigrams: (the, prime), (prime, minister), (minister, of), (of, our), (our, country). Count = 6−2+1 = 5. ##### Step 3. Take three consecutive words Trigrams: (the, prime, minister), (prime, minister, of), (minister, of, our), (of, our, country). Count = 6−3+1 = 4. **Final answer.** 6 unigrams, 5 bigrams and 4 trigrams. **Question. Construct bigrams for “I like tea” using one start and one end marker.** #### Given Original tokens: I, like, tea. Add and once each. #### To find The boundary-aware bigram sequence. #### Formula and why it applies $$ \#\text{bigrams}=(m+2)-1=m+1 $$ The start context predicts the first word, and the last word predicts that the sentence ends. #### Solution ##### Step 1. Write the padded sequence I like tea ##### Step 2. Move a two-token window (, I), (I, like), (like, tea), (tea, ). ##### Step 3. Count the windows $$ 3+1=4 $$ **Final answer.** There are four boundary-aware bigrams. **Question. Construct trigrams for “I like tea” using two start markers and one end marker.** #### Given Original sentence has three words. For a trigram model each predicted token needs two prior context tokens. #### To find All trigram prediction events. #### Formula and why it applies $$ P(w_i\mid w_{i-2},w_{i-1}) $$ Two start markers provide a complete context even for the first word. #### Solution ##### Step 1. Pad with two starts I like tea ##### Step 2. Slide a three-token window (, , I), (, I, like), (I, like, tea), (like, tea, ). ##### Step 3. Count predicted events The model predicts I, like, tea and the end marker: four predictions. **Final answer.** Four trigrams. Start markers are context; they are not ordinary predicted words. **Question. Give the bag-of-bigrams counts for “a b a b” without padding.** #### Given Four tokens in their given order. #### To find Bigram positions, distinct bigram types and their counts. #### Formula and why it applies $$ \#\text{positions}=4-2+1=3 $$ A bag stores frequencies; repeated bigrams must not be discarded. #### Solution ##### Step 1. Enumerate all positions (a,b), (b,a), (a,b). ##### Step 2. Group identical pairs | Bigram type | Count | | --- | --- | | a b | 2 | | b a | 1 | **Final answer.** Three bigram positions but two distinct bigram types. **Yaad rakho:** n-gram mein n words; context mein n−1 words. ### NLP M2 03: N-grams & Unigrams id : nlp-m2-03-ngrams-language-models title : NLP M2 03: N-grams & Unigrams title\_confirmed : true topic\_title : N-grams, chain rule and unigram models mode : m2 kind : focused status : available duration : 9:52 reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — N-grams and Laplace | Reviewed worked numericals id : overview-ngrams-laplace title : NLP M2 — N-grams and Laplace | Reviewed worked numericals title\_confirmed : true topic\_title : NLP M2: N-Grams & Laplace mode : m2 kind : supplemental status : available duration : 7:41 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Complete counts and four Laplace factors; corrected formulas, conditional-factor wording and rounding. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids : - nlp-m2-03-ngrams-language-models - nlp-m2-04-bigram-trigram-tables - nlp-m2-05-laplace-smoothing-discount These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Collocations and the purpose of a language model Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/m2-language-models--collocations-and-lm/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/m2-language-models/collocations-and-lm Unit 3, pages 62–108; essential probability revision **A collocation** is a conventional word combination whose words occur together more strongly than ordinary chance or independent frequencies would suggest. Examples include “strong tea”, “heavy rain” and “red wine”. A bigram is merely two consecutive items; it need not be a collocation. “of the” can be frequent because both words are frequent, while a collocation expresses a particularly strong association. | Term | What it represents | Example | | --- | --- | --- | | Bigram | Any adjacent two-token sequence | the book | | Collocation | A characteristic strongly associated expression | heavy rain | | Language model | A distribution used to score or predict sequences | P(rain | heavy) | **Definition.** A statistical language model assigns a probability to a sequence of words, or equivalently predicts the next word from preceding context. It is useful for autocomplete, speech recognition, machine translation, spelling correction, assistive communication and ranking candidate text. High probability reflects patterns learned from its training distribution; it is not a guarantee of truth or grammatical correctness. Previous words: please turn your homework → Candidate: in / over / elephant → Language model gives each candidate a probability → Application selects or ranks candidates **Noisy-channel connection.** Suppose x is the observed noisy text or speech and y is a candidate intended message. The noisy-channel approach chooses the y that maximizes P(x | y)P(y). The channel model P(x | y) asks how y could produce x; the language model P(y) asks whether y is a plausible message. P(x) is the same for all candidate y and therefore can be omitted when ranking. $$ \hat y=\arg\max_y P(y\mid x)=\arg\max_y P(x\mid y)P(y) $$ **Question. A spelling-correction system considers “flowers” and “flours”. For the observed input, the channel likelihoods are 0.2 and 0.4, and the respective language-model probabilities in context are 0.08 and 0.01. Which candidate does the noisy-channel rule choose?** #### Given Candidate flowers: P(x|flowers)=0.2, P(flowers)=0.08. Candidate flours: P(x|flours)=0.4, P(flours)=0.01. #### To find The larger unnormalized candidate score. #### Formula and why it applies $$ \operatorname{score}(y)=P(x\mid y)P(y) $$ Both the error mechanism and the language context matter. The same denominator would normalize both candidates. #### Solution ##### Step 1. Score flowers $$ 0.2\times0.08=0.016 $$ ##### Step 2. Score flours $$ 0.4\times0.01=0.004 $$ ##### Step 3. Compare $$ 0.016>0.004 $$ **Final answer.** Choose flowers. The scores are not yet posterior probabilities. **Yaad rakho:** Har word-pair bigram hai; har frequent pair meaningful collocation nahi. ### NLP M2 03: N-grams & Unigrams id : nlp-m2-03-ngrams-language-models title : NLP M2 03: N-grams & Unigrams title\_confirmed : true topic\_title : N-grams, chain rule and unigram models mode : m2 kind : focused status : available duration : 9:52 reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — N-grams and Laplace | Reviewed worked numericals id : overview-ngrams-laplace title : NLP M2 — N-grams and Laplace | Reviewed worked numericals title\_confirmed : true topic\_title : NLP M2: N-Grams & Laplace mode : m2 kind : supplemental status : available duration : 7:41 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Complete counts and four Laplace factors; corrected formulas, conditional-factor wording and rounding. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids : - nlp-m2-03-ngrams-language-models - nlp-m2-04-bigram-trigram-tables - nlp-m2-05-laplace-smoothing-discount These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Probability chain rule and the Markov assumption Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/m2-language-models--chain-rule-markov/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/m2-language-models/chain-rule-markov Unit 3, pages 62–108; essential probability revision **The probability chain rule** decomposes the probability of a complete sequence into successive conditional probabilities. It is different from the derivative chain rule used in deep learning. Here we multiply probabilities, not derivatives. $$ P(w_1,w_2,w_3)=P(w_1)P(w_2\mid w_1)P(w_3\mid w_1,w_2) $$ Read left to right: probability of the first word; then probability of the second after the first; then probability of the third after both previous words. For a longer sentence, continue the same pattern. The product symbol ∏ simply means “multiply all of these terms”. $$ P(w_1,\ldots,w_m)=\prod_{i=1}^{m}P(w_i\mid w_1,\ldots,w_{i-1}) $$ **The Markov assumption** approximates long history by a limited recent context. A unigram model ignores preceding words. A bigram model keeps the previous word. A trigram model keeps the previous two words. This reduces the number of counts needed, but loses longer-range dependencies. | Model | Prediction rule | For the next word country after the prime minister of our | | --- | --- | --- | | Unigram | P(wᵢ) | P(country) | | Bigram | P(wᵢ | wᵢ₋₁) | P(country | our) | | Trigram | P(wᵢ | wᵢ₋₂,wᵢ₋₁) | P(country | of, our) | | Full chain rule | P(wᵢ | all previous words) | P(country | the, prime, minister, of, our) | **Question. For the three-word sentence “I like tea”, P(I)=0.2, P(like|I)=0.5 and P(tea|I,like)=0.4. Find its probability using the full chain rule.** #### Given Three conditional factors are given, including the full two-word history for tea. No end probability is requested. #### To find P(I, like, tea). #### Formula and why it applies $$ P(I,like,tea)=P(I)P(like\mid I)P(tea\mid I,like) $$ The chain rule supplies one factor per word in the specified sequence. #### Solution ##### Step 1. Insert the three supplied factors $$ P=0.2\times0.5\times0.4 $$ ##### Step 2. Multiply the first two $$ 0.2\times0.5=0.1 $$ ##### Step 3. Multiply by the third $$ 0.1\times0.4=0.04 $$ **Final answer.** The sentence probability is 0.04. **Question. A bigram model gives P(I|)=0.5, P(like|I)=0.6, P(tea|like)=0.25 and P(|tea)=0.8. Find the full sentence probability.** #### Given All four conditional probabilities, including the end event, are supplied. #### To find P(I like tea | ). #### Formula and why it applies $$ P=P(I\mid\langle s\rangle)P(like\mid I)P(tea\mid like)P(\langle/s\rangle\mid tea) $$ Under the bigram assumption, every factor uses one preceding token. #### Solution ##### Step 1. Insert the factors in sentence order $$ P=0.5\times0.6\times0.25\times0.8 $$ ##### Step 2. Start and second word $$ 0.5\times0.6=0.3 $$ ##### Step 3. Third word $$ 0.3\times0.25=0.075 $$ ##### Step 4. End the sentence $$ 0.075\times0.8=0.06 $$ **Final answer.** The complete sentence probability is 0.06. **Yaad rakho:** Sequence probability = saare required transitions ka product. **Typical 2-mark answer.** An n-gram language model predicts a word from its previous n−1 words using a Markov approximation. For example, a bigram model uses P(wᵢ | wᵢ₋₁). It estimates these probabilities from corpus counts and multiplies the appropriate factors to score a sentence. ### NLP M2 03: N-grams & Unigrams id : nlp-m2-03-ngrams-language-models title : NLP M2 03: N-grams & Unigrams title\_confirmed : true topic\_title : N-grams, chain rule and unigram models mode : m2 kind : focused status : available duration : 9:52 reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — N-grams and Laplace | Reviewed worked numericals id : overview-ngrams-laplace title : NLP M2 — N-grams and Laplace | Reviewed worked numericals title\_confirmed : true topic\_title : NLP M2: N-Grams & Laplace mode : m2 kind : supplemental status : available duration : 7:41 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Complete counts and four Laplace factors; corrected formulas, conditional-factor wording and rounding. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids : - nlp-m2-03-ngrams-language-models - nlp-m2-04-bigram-trigram-tables - nlp-m2-05-laplace-smoothing-discount These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Unigram estimation and complete worked examples Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/m2-count-to-probability--unigram-estimation/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/m2-count-to-probability/unigram-estimation Unit 3, pages 97–120 and appendix **Maximum likelihood estimation (MLE)** chooses probabilities equal to observed relative frequencies. In a unigram model, every word is predicted independently of its surrounding words. If a word occurs c times in N training tokens, its estimate is c/N. To score m specified words, multiply their m unigram probabilities. $$ P_{\mathrm{MLE}}(w)=\frac{C(w)}{N}\qquad P(w_1,\ldots,w_m)\approx\prod_{i=1}^{m}P(w_i) $$ Choose a tokenization convention before counting. Lowercasing, punctuation handling and boundary markers can change the vocabulary and counts. Never count words with one convention and form the denominator with another. A unigram model gives the same product to rearrangements of the same word multiset; that is one of its limitations. **Question. Train an unpadded, case-insensitive unigram model on “I like tea” and “I like coffee”. Find the probability of “I like tea”.** #### Given Training sentences: I like tea; I like coffee. Use lowercase words; exclude punctuation and sentence boundaries. #### To find The vocabulary, word probabilities and test-sentence probability. #### Formula and why it applies $$ P(w)=C(w)/N,\qquad P(I\ like\ tea)=P(I)P(like)P(tea) $$ The unigram approximation ignores order and uses total token count for every denominator. #### Solution ##### Step 1. Count every training token Each sentence contains three words, so N = 3+3 = 6. ##### Step 2. Count types and frequencies | Word | Count | MLE probability | | --- | --- | --- | | i | 2 | 2/6 = 1/3 | | like | 2 | 2/6 = 1/3 | | tea | 1 | 1/6 | | coffee | 1 | 1/6 | Vocabulary size V = 4. ##### Step 3. Write one factor per test word $$ P=\frac{2}{6}\times\frac{2}{6}\times\frac{1}{6} $$ ##### Step 4. Multiply the fractions $$ P=\frac{4}{216}=\frac{1}{54}\approx0.018519 $$ **Final answer.** The unigram sentence probability is approximately 0.018519. **Question. A corpus contains counts a: 4, b: 3 and c: 1. Find the unigram probability of “a b a”.** #### Given C(a)=4, C(b)=3, C(c)=1. No boundary markers. #### To find P(a b a). #### Formula and why it applies $$ P(a\,b\,a)=P(a)P(b)P(a) $$ A repeated test word contributes a probability factor at every occurrence; do not include it just once. #### Solution ##### Step 1. Find the total $$ N=4+3+1=8 $$ ##### Step 2. Estimate the required probabilities $$ P(a)=4/8=1/2,\qquad P(b)=3/8 $$ ##### Step 3. Use both occurrences of a $$ P=\frac12\times\frac38\times\frac12 $$ ##### Step 4. Simplify $$ P=\frac3{32}=0.09375 $$ **Final answer.** P(a b a) = 0.09375. **Yaad rakho:** Repeated word ka probability bhi repeat multiply hoga. **Question. An unsmoothed unigram model is trained on “red blue red”. Find the probability of “red green”.** #### Given N=3. Counts: red=2, blue=1, green=0. The test contains red and green. #### To find The unsmoothed test probability and the reason for its value. #### Formula and why it applies $$ P(red\ green)=P(red)P(green) $$ MLE assigns zero to an unseen outcome. Smoothing is needed if an unseen in-vocabulary item must receive probability. #### Solution ##### Step 1. Estimate red $$ P(red)=2/3 $$ ##### Step 2. Estimate the unseen word $$ P(green)=0/3=0 $$ ##### Step 3. Multiply $$ (2/3)\times0=0 $$ **Final answer.** Unsmoothed probability = 0. This reflects missing training evidence, not proof that the phrase is impossible. ### NLP M2 03: N-grams & Unigrams id : nlp-m2-03-ngrams-language-models title : NLP M2 03: N-grams & Unigrams title\_confirmed : true topic\_title : N-grams, chain rule and unigram models mode : m2 kind : focused status : available duration : 9:52 reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — N-grams and Laplace | Reviewed worked numericals id : overview-ngrams-laplace title : NLP M2 — N-grams and Laplace | Reviewed worked numericals title\_confirmed : true topic\_title : NLP M2: N-Grams & Laplace mode : m2 kind : supplemental status : available duration : 7:41 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Complete counts and four Laplace factors; corrected formulas, conditional-factor wording and rounding. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids : - nlp-m2-03-ngrams-language-models - nlp-m2-04-bigram-trigram-tables - nlp-m2-05-laplace-smoothing-discount These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Bigram counts: build the table before multiplying Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/m2-count-to-probability--bigram-count-table/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/m2-count-to-probability/bigram-count-table Unit 3, pages 97–120 and appendix A **bigram probability table** uses the previous word as its row and the next word as its column. Divide each cell count by its row's total outgoing count. With one end marker on every sentence, each ordinary training word has a continuation, so its outgoing count equals its token frequency. Do not create a bigram that crosses from one sentence into the next. $$ P(w\mid h)=\frac{C(h,w)}{\sum_{v\in V}C(h,v)}=\frac{C(h,w)}{C(h)} $$ Add boundaries to each sentence separately → List adjacent pairs → Count repeated pairs → Divide each row by its total → Follow the test sentence through the table **Question. Train a bigram model on “I like tea”, “I like coffee” and “you like tea”. Use one start and one end marker. Find the probability of “I like tea”.** #### Given Corpus: I like tea ; I like coffee ; you like tea . Preserve these tokens. #### To find All nonzero bigram estimates and the test probability. #### Formula and why it applies $$ P(w\mid h)=C(h,w)/C(h) $$ Every sentence starts and ends independently. Four probability factors are required for the three test words plus end. #### Solution ##### Step 1. List the training pairs Sentence 1: (,I), (I,like), (like,tea), (tea,). Sentence 2: (,I), (I,like), (like,coffee), (coffee,). Sentence 3: (,you), (you,like), (like,tea), (tea,). ##### Step 2. Create the nonzero count table | History → next | Pair count | History count | Probability | | --- | --- | --- | --- | | → I | 2 | 3 | 2/3 | | → you | 1 | 3 | 1/3 | | I → like | 2 | 2 | 1 | | you → like | 1 | 1 | 1 | | like → tea | 2 | 3 | 2/3 | | like → coffee | 1 | 3 | 1/3 | | tea → | 2 | 2 | 1 | | coffee → | 1 | 1 | 1 | ##### Step 3. Write the test path $$ P=P(I\mid\langle s\rangle)P(like\mid I)P(tea\mid like)P(\langle/s\rangle\mid tea) $$ ##### Step 4. Substitute each matching row $$ P=\frac23\times1\times\frac23\times1 $$ ##### Step 5. Multiply $$ P=\frac49\approx0.444444 $$ **Final answer.** P(I like tea) = 4/9 ≈ 0.444444. **Question. Using the same three-sentence corpus given here—“I like tea”, “I like coffee”, “you like tea”—find the bigram probabilities of “you like coffee” and “tea like I”. Include boundaries.** #### Given Corpus with one start/end marker per sentence. Counts: start→you=1/3; you→like=1; like→coffee=1/3; coffee→end=1. No training sentence starts with tea. #### To find The two test probabilities. #### Formula and why it applies $$ P(\text{sentence})=\prod P(\text{next token}\mid\text{previous token}) $$ A model can assign a nonzero probability to a sentence it never saw if all of its local bigrams were seen. #### Solution ##### Step 1. Score you like coffee $$ P=\frac13\times1\times\frac13\times1=\frac19\approx0.111111 $$ ##### Step 2. Inspect tea like I $$ P(tea\mid\langle s\rangle)=0/3=0 $$ ##### Step 3. Apply the zero factor The full product is zero, regardless of the remaining factors. **Final answer.** P(you like coffee) = 1/9; P(tea like I) = 0. A new sentence is not necessarily a zero-probability sentence. **Question. Obtain the unsmoothed bigram probability of “students are from Vellore” from this corpus: “I am from Vellore”; “I am a teacher”; “students are good and are from various cities”; “students from Vellore do engineering”. Use sentence boundaries.** #### Given Four training sentences, each padded with one and . Test sequence: students are from Vellore . #### To find The five required conditional probabilities and their product. #### Formula and why it applies $$ P(w\mid h)=C(h,w)/C(h) $$ Count all occurrences in the entire corpus. In particular, are occurs twice inside the third sentence, and Vellore is followed by end once and do once. #### Solution ##### Step 1. Count the histories | History | Occurrences with continuation | | --- | --- | | | 4 | | students | 2 | | are | 2 | | from | 3 | | Vellore | 2 | ##### Step 2. Count the required pairs | Required pair | Count | Probability | | --- | --- | --- | | students | 2 | 2/4 | | students are | 1 | 1/2 | | are from | 1 | 1/2 | | from Vellore | 2 | 2/3 | | Vellore | 1 | 1/2 | ##### Step 3. Write the complete product $$ P=\frac24\times\frac12\times\frac12\times\frac23\times\frac12 $$ ##### Step 4. Multiply progressively $$ \frac24\times\frac12=\frac14,\quad \frac14\times\frac12=\frac18 $$ $$ \frac18\times\frac23=\frac1{12},\quad \frac1{12}\times\frac12=\frac1{24} $$ **Final answer.** P = 1/24 ≈ 0.0416667. **Yaad rakho:** Same sentence mein word do baar aaye toh count bhi do. ### Bigram & Trigram Probs id : nlp-m2-04-bigram-trigram-tables title : Bigram & Trigram Probs title\_confirmed : true topic\_title : Build bigram and trigram probabilities from a corpus mode : m2 kind : focused status : available duration : 10:05 reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — N-grams and Laplace | Reviewed worked numericals id : overview-ngrams-laplace title : NLP M2 — N-grams and Laplace | Reviewed worked numericals title\_confirmed : true topic\_title : NLP M2: N-Grams & Laplace mode : m2 kind : supplemental status : available duration : 7:41 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Complete counts and four Laplace factors; corrected formulas, conditional-factor wording and rounding. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids : - nlp-m2-03-ngrams-language-models - nlp-m2-04-bigram-trigram-tables - nlp-m2-05-laplace-smoothing-discount These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Trigram estimates: two-word history, not two denominators Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/m2-count-to-probability--trigram-estimation/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/m2-count-to-probability/trigram-estimation Unit 3, pages 97–120 and appendix A trigram model predicts the next word from the two preceding words. The numerator is the count of a triple. The denominator is the count of its two-word history with an available next token. Do not multiply two independent bigram probabilities and call that a trigram probability. $$ P(w_i\mid w_{i-2},w_{i-1})=\frac{C(w_{i-2},w_{i-1},w_i)}{C(w_{i-2},w_{i-1})} $$ **Question. A corpus contains 12 occurrences of “prime minister”. Eight are followed by “of”, three by “said”, and one by “visited”. Estimate the three trigram probabilities.** #### Given History h = (prime, minister). Count of this history = 12. Triple counts for of, said, visited are 8, 3, 1. #### To find The conditional distribution of the next word. #### Formula and why it applies $$ P(w\mid prime,minister)=C(prime,minister,w)/12 $$ All three candidates share one two-word history, so they share denominator 12. #### Solution ##### Step 1. Calculate of $$ P(of\mid prime,minister)=8/12=2/3 $$ ##### Step 2. Calculate said $$ P(said\mid prime,minister)=3/12=1/4 $$ ##### Step 3. Calculate visited $$ P(visited\mid prime,minister)=1/12 $$ ##### Step 4. Confirm normalization $$ 8/12+3/12+1/12=12/12=1 $$ **Final answer.** The probabilities are 2/3, 1/4 and 1/12. **Question. Train a trigram model on “I like tea”, “I like coffee” and “you like tea”. Use two start markers and one end marker. Score “I like tea”.** #### Given Three sentences. Padded first sentence is I like tea ; pad the other two in the same way. #### To find Four trigram prediction probabilities and the sentence probability. #### Formula and why it applies $$ P=\prod_i P(w_i\mid w_{i-2},w_{i-1}) $$ Each prediction uses a two-token context; the two start markers supply the first history. #### Solution ##### Step 1. Count the first-word event The history (,) occurs three times, and two are followed by I. $$ P(I\mid\langle s\rangle,\langle s\rangle)=2/3 $$ ##### Step 2. Count the second-word event (,I) occurs twice; both are followed by like. $$ P(like\mid\langle s\rangle,I)=2/2=1 $$ ##### Step 3. Count the third-word event (I,like) occurs twice; one is followed by tea. $$ P(tea\mid I,like)=1/2 $$ ##### Step 4. Count the end event (like,tea) occurs twice; both are followed by end. $$ P(\langle/s\rangle\mid like,tea)=2/2=1 $$ ##### Step 5. Multiply all four $$ P=\frac23\times1\times\frac12\times1=\frac13 $$ **Final answer.** Trigram P(I like tea) = 1/3. It differs from the bigram estimate because the context and denominator changed. **Question. A trigram model supplies P(we|,)=0.4, P(study|,we)=0.5, P(NLP|we,study)=0.8 and P(|study,NLP)=0.25. Score “we study NLP”.** #### Given Four trigram conditional factors. The end marker is included. #### To find The complete sentence probability. #### Formula and why it applies $$ P=P(we\mid s,s)P(study\mid s,we)P(NLP\mid we,study)P(end\mid study,NLP) $$ Use each supplied two-word history exactly; do not swap its word order. #### Solution ##### Step 1. Insert the values $$ P=0.4\times0.5\times0.8\times0.25 $$ ##### Step 2. Multiply the first pair $$ 0.4\times0.5=0.2 $$ ##### Step 3. Include NLP $$ 0.2\times0.8=0.16 $$ ##### Step 4. Include the end $$ 0.16\times0.25=0.04 $$ **Final answer.** P = 0.04. **Yaad rakho:** Trigram: teen-word count upar, pehle do-word count neeche. **Why larger n is not automatically better.** More context can capture local order more precisely, but each longer history occurs less often. Parameter count, storage and sparsity grow. A model trained on Shakespeare may assign poor probabilities to financial news because training and test distributions differ. Keep a held-out test set; do not evaluate only on memorized training sentences. ### Bigram & Trigram Probs id : nlp-m2-04-bigram-trigram-tables title : Bigram & Trigram Probs title\_confirmed : true topic\_title : Build bigram and trigram probabilities from a corpus mode : m2 kind : focused status : available duration : 10:05 reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — N-grams and Laplace | Reviewed worked numericals id : overview-ngrams-laplace title : NLP M2 — N-grams and Laplace | Reviewed worked numericals title\_confirmed : true topic\_title : NLP M2: N-Grams & Laplace mode : m2 kind : supplemental status : available duration : 7:41 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Complete counts and four Laplace factors; corrected formulas, conditional-factor wording and rounding. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids : - nlp-m2-03-ngrams-language-models - nlp-m2-04-bigram-trigram-tables - nlp-m2-05-laplace-smoothing-discount These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Class-style count-table questions and partial-table traps Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/m2-count-to-probability--class-count-table-questions/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/m2-count-to-probability/class-count-table-questions Unit 3, pages 97–120 and appendix **Question. Train a bigram model on “I am Sam”, “Sam I am”, and “I do not like green eggs and ham”. Include sentence boundaries. Find P(I|), P(Sam|), P(am|I), P(do|I), P(Sam|am) and P(|Sam).** #### Given Three sentences; one and per sentence; preserve capital I and Sam. #### To find Six bigram estimates. #### Formula and why it applies $$ P(w\mid h)=C(h,w)/C(h) $$ Count all matching histories, including repeated words in different sentence positions. #### Solution ##### Step 1. Count the histories START occurs 3 times; I occurs 3 times; am occurs 2 times; Sam occurs 2 times. ##### Step 2. Count the requested pairs | Pair | Pair count | History count | | --- | --- | --- | | START I | 2 | 3 | | START Sam | 1 | 3 | | I am | 2 | 3 | | I do | 1 | 3 | | am Sam | 1 | 2 | | Sam END | 1 | 2 | ##### Step 3. Divide pair by history | Requested probability | Result | | --- | --- | | P(I|START) | 2/3 | | P(Sam|START) | 1/3 | | P(am|I) | 2/3 | | P(do|I) | 1/3 | | P(Sam|am) | 1/2 | | P(END|Sam) | 1/2 | **Final answer.** The six estimates are 2/3,1/3,2/3,1/3,1/2,1/2 in the requested order. **Question. A bigram table gives P(i|)=0.25, P(want|i)=0.33, P(english|want)=0.0011, P(food|english)=0.5 and P(|food)=0.68. Score “i want english food” using these supplied rounded values.** #### Given All five transition probabilities are explicitly given. Include the end marker. #### To find The sentence probability from the supplied table. #### Formula and why it applies $$ P=P(i\mid START)P(want\mid i)P(english\mid want)P(food\mid english)P(END\mid food) $$ A table-based question uses the provided values; slight differences can arise if exact corpus fractions are used instead. #### Solution ##### Step 1. First two events $$ 0.25(0.33)=0.0825 $$ ##### Step 2. Add english $$ 0.0825(0.0011)=0.00009075 $$ ##### Step 3. Add food $$ 0.00009075(0.5)=0.000045375 $$ ##### Step 4. End the sentence $$ 0.000045375(0.68)=0.000030855 $$ **Final answer.** P≈0.000030855 using the supplied rounded probabilities. **Question. A corpus has vocabulary size 1446, C(i)=2533 and C(i,want)=827. Compute the unsmoothed and add-one-smoothed P(want|i). The displayed bigram table contains only eight vocabulary columns.** #### Given Full history count C(i)=2533; pair count 827; complete vocabulary size V=1446. The displayed table is only a subset. #### To find Both conditional probabilities. #### Formula and why it applies $$ P(want\mid i)=827/2533,\qquad P_L(want\mid i)=\frac{827+1}{2533+1446} $$ Use the supplied full history total. Summing only the eight displayed columns would wrongly ignore all other possible continuations. #### Solution ##### Step 1. Calculate the unsmoothed ratio $$ P=827/2533\approx0.326490 $$ ##### Step 2. Add one to the pair count $$ 827+1=828 $$ ##### Step 3. Add V to the history count $$ 2533+1446=3979 $$ ##### Step 4. Divide $$ P_L=828/3979\approx0.208092 $$ **Final answer.** Unsmoothed≈0.326490; smoothed≈0.208092. **Yaad rakho:** Partial table ka visible row-sum, full history total nahi hota. ### Bigram & Trigram Probs id : nlp-m2-04-bigram-trigram-tables title : Bigram & Trigram Probs title\_confirmed : true topic\_title : Build bigram and trigram probabilities from a corpus mode : m2 kind : focused status : available duration : 10:05 reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — N-grams and Laplace | Reviewed worked numericals id : overview-ngrams-laplace title : NLP M2 — N-grams and Laplace | Reviewed worked numericals title\_confirmed : true topic\_title : NLP M2: N-Grams & Laplace mode : m2 kind : supplemental status : available duration : 7:41 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Complete counts and four Laplace factors; corrected formulas, conditional-factor wording and rounding. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids : - nlp-m2-03-ngrams-language-models - nlp-m2-04-bigram-trigram-tables - nlp-m2-05-laplace-smoothing-discount These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Perplexity: meaning, formula and four complete examples Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/m2-evaluation-smoothing--evaluation-perplexity/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/m2-evaluation-smoothing/evaluation-perplexity Unit 3, pages 121–156 **Intrinsic evaluation** evaluates a model directly on a held-out test corpus, for example using perplexity. **Extrinsic evaluation** places it inside a downstream application, such as a speech recognizer or translation system, and measures the application's performance. Extrinsic testing is closer to actual use but often slower and more expensive. A lower perplexity is not a guarantee of improvement on every downstream task. **Perplexity** is the inverse probability of a test sequence, normalized by the number of prediction events. Intuitively, it is the model's effective average number of equally plausible choices. A model that always gives the correct next word probability 1/4 has perplexity 4. Lower is better when the models are tested on the same data, vocabulary, tokenization and boundary convention. $$ \operatorname{PP}(W)=P(W)^{-1/N}=\sqrt[N]{\frac1{P(W)}} $$ | Symbol | Meaning | | --- | --- | | W | The complete test sequence. | | P(W) | Product of the probabilities assigned to the actual next tokens. | | N | Number of scored prediction events under the stated convention. | | Exponent −1/N | Take the reciprocal, then the Nth root. | For calculator entry, use parentheses: P raised to (−1 ÷ N), or (1 ÷ P) raised to (1 ÷ N). Retain several digits in intermediate products. The ordinary start marker is usually given context, not a predicted event; the end marker counts if its probability is included. **Question. A model assigns probability 0.0625 to a test sequence containing four scored tokens. Compute its perplexity.** #### Given P(W)=0.0625=1/16; N=4. #### To find PP(W). #### Formula and why it applies $$ \operatorname{PP}(W)=\left(\frac1{P(W)}\right)^{1/N} $$ The root removes the effect of sequence length; the reciprocal makes lower surprise correspond to lower perplexity. #### Solution ##### Step 1. Take the reciprocal $$ 1/P(W)=1/0.0625=16 $$ ##### Step 2. Use the fourth root $$ \operatorname{PP}=16^{1/4} $$ ##### Step 3. Evaluate $$ 2^4=16\quad\Rightarrow\quad\operatorname{PP}=2 $$ **Final answer.** Perplexity = 2. **Question. Two language models score the same three-token test sequence. Model A gives next-token probabilities 0.5, 0.5, 0.5. Model B gives 0.5, 0.25, 0.25. Compute their perplexities and select the better intrinsic result.** #### Given Same sequence and N=3 for both models. A factors: 0.5,0.5,0.5. B factors: 0.5,0.25,0.25. #### To find Both sequence probabilities and perplexities. #### Formula and why it applies $$ P(W)=\prod_i p_i,\qquad\operatorname{PP}=P(W)^{-1/3} $$ A fair comparison uses identical test tokens and scoring conventions. #### Solution ##### Step 1. Calculate A's probability $$ P_A=0.5\times0.5\times0.5=0.125=1/8 $$ ##### Step 2. Calculate A's perplexity $$ \operatorname{PP}_A=8^{1/3}=2 $$ ##### Step 3. Calculate B's probability $$ P_B=0.5\times0.25\times0.25=0.03125=1/32 $$ ##### Step 4. Calculate B's perplexity $$ \operatorname{PP}_B=32^{1/3}\approx3.174802 $$ ##### Step 5. Compare 2 is lower than 3.174802; A places more probability on the observed sequence. **Final answer.** Model A is better by perplexity on this test: 2 versus about 3.174802. **Question. The bigram training corpus is “This is a dog”, “This is a cat”, “I love my cat”, “This is my name”. Score “This is my cat”, including an end marker, and compute perplexity using five predicted events.** #### Given Each training sentence has one start and end marker. Test predictions: This, is, my, cat, . Thus N=5. #### To find Bigram sentence probability and perplexity. #### Formula and why it applies $$ P(w\mid h)=C(h,w)/C(h),\qquad\operatorname{PP}=P(W)^{-1/5} $$ Start is given context. Four words plus the end marker produce five probability factors. #### Solution ##### Step 1. Count the required transitions | Transition | Pair count / history count | | --- | --- | | →This | 3/4 | | This→is | 3/3 | | is→my | 1/3 | | my→cat | 1/2 | | cat→ | 2/2 | ##### Step 2. Form the product $$ P=\frac34\times1\times\frac13\times\frac12\times1=\frac18 $$ ##### Step 3. Take the reciprocal $$ 1/P=8 $$ ##### Step 4. Take the fifth root $$ \operatorname{PP}=8^{1/5}\approx1.515717 $$ **Final answer.** P(W)=1/8 and perplexity≈1.515717 using five predicted events. **If the question explicitly states N = 6:** use that supplied convention and report PP = 81/6 ≈ 1.414214. The class slide counts both boundary markers in N. Write the chosen N beside the formula; do not silently switch between 5 and 6. Also, This occurs three times and is follows it three times, so P(is|This)=3/3, not 3/4. **Question. A uniform model assigns probability 1/10 to each actual next word in a five-token test. Find its perplexity.** #### Given Five scored tokens, each with assigned probability 0.1. N=5. #### To find The sentence probability and perplexity. #### Formula and why it applies $$ P(W)=(1/10)^5,\qquad \operatorname{PP}=P(W)^{-1/5} $$ Uniform equal probabilities make the effective number of choices especially clear. #### Solution ##### Step 1. Multiply the five factors $$ P(W)=0.1^5=0.00001=10^{-5} $$ ##### Step 2. Invert $$ 1/P(W)=10^5 $$ ##### Step 3. Take the fifth root $$ \operatorname{PP}=(10^5)^{1/5}=10 $$ **Final answer.** Perplexity = 10. **Yaad rakho:** Probability high → surprise low → perplexity low. ### M2 06 — Perplexity, logarithms and calculator working id : nlp-m2-06-perplexity-logs-calculator title : M2 06 — Perplexity, logarithms and calculator working title\_confirmed : false topic\_title : Perplexity, logarithms and calculator working mode : m2 kind : focused status : queued duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — Perplexity | Reviewed formulas and solved numericals id : overview-perplexity-logs title : NLP M2 — Perplexity | Reviewed formulas and solved numericals title\_confirmed : true topic\_title : NLP M2 — Perplexity | Reviewed formulas and solved numericals mode : m2 kind : supplemental status : available duration : 9:35 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Readable formulas and all worked examples; corrected comparison conditions and context claims. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids : - nlp-m2-06-perplexity-logs-calculator These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Log probabilities: replace long products with sums Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/m2-evaluation-smoothing--log-probabilities/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/m2-evaluation-smoothing/log-probabilities Unit 3, pages 121–156 Probabilities are at most 1, so multiplying many of them produces very small numbers. Computers can round extremely small positive products to zero: **numerical underflow**. Log probabilities convert multiplication into addition and are more stable for long sequences. Natural logs use base e; log₂ uses base 2. Choose one base consistently. $$ \ln P(W)=\sum_{i=1}^{N}\ln p_i $$ $$ P(W)=e^{\ln P(W)},\qquad \operatorname{PP}(W)=\exp\left(-\frac1N\sum_i\ln p_i\right) $$ The log of a probability below 1 is negative. A value closer to zero means a larger probability: ln(0.5) ≈ −0.693 is larger than ln(0.1) ≈ −2.303. A zero probability has log −∞; taking logs does not solve the unseen-event problem. Smoothing handles that separate problem. **Question. The next-word probabilities are 0.5, 0.25 and 0.5. Calculate the natural log probability, ordinary probability and perplexity.** #### Given p₁=0.5; p₂=0.25; p₃=0.5; N=3. #### To find ln P(W), P(W), and PP. #### Formula and why it applies $$ \ln P=\sum_i\ln p_i,\quad P=e^{\ln P},\quad\operatorname{PP}=e^{-\ln P/N} $$ Logarithms preserve products as sums and the exponent converts the result back. #### Solution ##### Step 1. Compute individual logs $$ \ln0.5\approx-0.693147,\quad\ln0.25\approx-1.386294 $$ ##### Step 2. Add all three terms $$ \ln P=-0.693147-1.386294-0.693147\approx-2.772589 $$ ##### Step 3. Recover the probability $$ P=e^{-2.772589}\approx0.0625 $$ ##### Step 4. Find average negative log probability $$ -\ln P/3\approx2.772589/3=0.924196 $$ ##### Step 5. Exponentiate $$ \operatorname{PP}=e^{0.924196}\approx2.519842 $$ **Final answer.** ln P≈−2.772589, P=0.0625, PP≈2.519842. **Question. A model has a total natural log probability −12 over six scored tokens. Calculate its perplexity.** #### Given ln P(W)=−12; N=6. #### To find PP without forming the tiny ordinary probability. #### Formula and why it applies $$ \operatorname{PP}=\exp(-\ln P(W)/N) $$ The logarithmic formula directly uses the supplied log score. #### Solution ##### Step 1. Negate the log score $$ -\ln P=12 $$ ##### Step 2. Divide by token count $$ 12/6=2 $$ ##### Step 3. Apply exponential $$ \operatorname{PP}=e^2\approx7.389056 $$ **Final answer.** Perplexity≈7.389056. **Question. A language model's average negative log₂ probability is 3 bits per token. Find its perplexity.** #### Given Cross-entropy H=3 bits/token. Log base is 2. #### To find PP. #### Formula and why it applies $$ \operatorname{PP}=2^H $$ The inverse operation of log₂ is raising 2 to that power; use e only for natural logs. #### Solution ##### Step 1. Identify the log base Bits mean base 2. ##### Step 2. Raise the base to H $$ \operatorname{PP}=2^3=8 $$ **Final answer.** Perplexity = 8. **Yaad rakho:** ln use kiya toh e se wapas; log₂ use kiya toh 2 se wapas. ### M2 06 — Perplexity, logarithms and calculator working id : nlp-m2-06-perplexity-logs-calculator title : M2 06 — Perplexity, logarithms and calculator working title\_confirmed : false topic\_title : Perplexity, logarithms and calculator working mode : m2 kind : focused status : queued duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — Perplexity | Reviewed formulas and solved numericals id : overview-perplexity-logs title : NLP M2 — Perplexity | Reviewed formulas and solved numericals title\_confirmed : true topic\_title : NLP M2 — Perplexity | Reviewed formulas and solved numericals mode : m2 kind : supplemental status : available duration : 9:35 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Readable formulas and all worked examples; corrected comparison conditions and context claims. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids : - nlp-m2-06-perplexity-logs-calculator These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Laplace smoothing: why both numerator and denominator change Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/m2-evaluation-smoothing--laplace-basics/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/m2-evaluation-smoothing/laplace-basics Unit 3, pages 121–156 **Smoothing** reallocates some probability mass to events that received zero or unreliable estimates from finite training data. **Laplace, or add-one, smoothing** adds one pseudo-count to every possible next outcome. If there are V possible outcomes, the total added count is V, so adding 1 only to the numerator would produce probabilities that do not sum to 1. $$ P_{\mathrm{Laplace}}(w)=\frac{C(w)+1}{N+V} $$ $$ P_{\mathrm{Laplace}}(w\mid h)=\frac{C(h,w)+1}{C(h)+V} $$ For unigram smoothing, N is the total training token count. For bigram smoothing, C(h) is the count of the particular history. In both cases V is the number of outcomes being predicted, not the number of training sentences and not the number of distinct bigrams. When end is a possible prediction, include it in V; the start marker is usually context only. Observed counts: 3, 1, 0 → Add one to every outcome: 4, 2, 1 → New total: 7 → Smoothed probabilities: 4/7, 2/7, 1/7 **Question. A fixed unigram vocabulary is {a,b,c}. The training counts are 3,1,0. Compute unsmoothed and Laplace-smoothed probabilities.** #### Given C(a)=3, C(b)=1, C(c)=0; N=4; V=3. #### To find The complete distributions. #### Formula and why it applies $$ P(w)=C(w)/N,\qquad P_L(w)=(C(w)+1)/(N+V) $$ The zero-count c is still in the fixed vocabulary, so it must receive one pseudo-count and contribute to V. #### Solution ##### Step 1. Compute unsmoothed values | Word | Unsmoothed | | --- | --- | | a | 3/4 = 0.75 | | b | 1/4 = 0.25 | | c | 0/4 = 0 | ##### Step 2. Calculate the new denominator $$ N+V=4+3=7 $$ ##### Step 3. Add one to each numerator $$ C(a)+1=4,\quad C(b)+1=2,\quad C(c)+1=1 $$ ##### Step 4. Divide | Word | Smoothed | | --- | --- | | a | 4/7 ≈ 0.571429 | | b | 2/7 ≈ 0.285714 | | c | 1/7 ≈ 0.142857 | ##### Step 5. Confirm the total $$ 4/7+2/7+1/7=1 $$ **Final answer.** Smoothed probabilities: a=4/7, b=2/7, c=1/7. Common a loses mass; unseen c gains mass. **Question. After history I, a corpus records like three times, study once and sleep zero times. The allowed next vocabulary is {like,study,sleep}. Compute all Laplace-smoothed bigram probabilities.** #### Given C(I,like)=3; C(I,study)=1; C(I,sleep)=0. History count C(I)=4. V=3. #### To find P\_L(like|I), P\_L(study|I), P\_L(sleep|I). #### Formula and why it applies $$ P_L(w\mid I)=\frac{C(I,w)+1}{C(I)+V} $$ This is smoothing within the I row; use its total 4, not the global corpus token count. #### Solution ##### Step 1. Form the common denominator $$ C(I)+V=4+3=7 $$ ##### Step 2. Calculate like $$ P_L(like\mid I)=(3+1)/7=4/7 $$ ##### Step 3. Calculate study $$ P_L(study\mid I)=(1+1)/7=2/7 $$ ##### Step 4. Calculate sleep $$ P_L(sleep\mid I)=(0+1)/7=1/7 $$ **Final answer.** The row is (4/7, 2/7, 1/7). An unseen bigram is no longer forced to zero. **Question. A fixed vocabulary has five possible next tokens. A history h is unseen in training. What is its add-one probability for each next token?** #### Given C(h)=0; C(h,w)=0 for each outcome; V=5. #### To find P\_L(w|h) for each token. #### Formula and why it applies $$ P_L(w\mid h)=\frac{0+1}{0+V} $$ All outcomes receive equal pseudo-counts because the training data gives no preference for this history. #### Solution ##### Step 1. Add the pseudo-count $$ 0+1=1 $$ ##### Step 2. Form the denominator $$ 0+5=5 $$ ##### Step 3. Divide and sum $$ P_L(w\mid h)=1/5=0.2,\qquad5(0.2)=1 $$ **Final answer.** Every outcome gets 0.2 under this fixed-vocabulary model. **Vocabulary warning.** Add-one does not automatically define probabilities for every word in the world. Unknown words require a declared vocabulary/UNK convention. If a question supplies V, use that stated V rather than inventing additional tokens. **Yaad rakho:** Add-one: upar +1, neeche +V; bigram mein neeche history ka count. ### M2 05: Laplace Smoothing id : nlp-m2-05-laplace-smoothing-discount title : M2 05: Laplace Smoothing title\_confirmed : true topic\_title : Laplace smoothing, unseen sequences and adjusted counts mode : m2 kind : focused status : available duration : 11:11 reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — N-grams and Laplace | Reviewed worked numericals id : overview-ngrams-laplace title : NLP M2 — N-grams and Laplace | Reviewed worked numericals title\_confirmed : true topic\_title : NLP M2: N-Grams & Laplace mode : m2 kind : supplemental status : available duration : 7:41 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Complete counts and four Laplace factors; corrected formulas, conditional-factor wording and rounding. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids : - nlp-m2-03-ngrams-language-models - nlp-m2-04-bigram-trigram-tables - nlp-m2-05-laplace-smoothing-discount These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · A complete unseen-bigram question from corpus to final answer Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/m2-evaluation-smoothing--smoothed-sentence/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/m2-evaluation-smoothing/smoothed-sentence Unit 3, pages 121–156 **Question. Consider the corpus “I love to code in Python. I love to learn new things. I code every day.” Obtain the bigram probabilities needed for “I love Python”. Explain the zero-probability issue, then use Laplace smoothing to score it. Treat each sentence separately, preserve case, remove punctuation, and include an end token among predicted outcomes.** #### Given Three sentences padded with one start/end marker. Distinct words: I, love, to, code, in, Python, learn, new, things, every, day. With , V=12. Start is context only. #### To find Unsmoothed and smoothed P(I love Python | ). #### Formula and why it applies $$ P(w\mid h)=\frac{C(h,w)}{C(h)},\qquad P_L(w\mid h)=\frac{C(h,w)+1}{C(h)+12} $$ The test contains an unseen transition love→Python. Smoothing assigns probability to that transition and renormalizes every required history row. #### Solution ##### Step 1. Count the contexts | History | Count | | --- | --- | | | 3 | | I | 3 | | love | 2 | | Python | 1 | ##### Step 2. Count each required pair | Pair | Count | | --- | --- | | I | 3 | | I love | 2 | | love Python | 0 | | Python | 1 | ##### Step 3. Compute the unsmoothed product $$ P=\frac33\times\frac23\times\frac02\times\frac11=0 $$ Both words love and Python were seen, but their adjacent pair was not. This is an unseen bigram, not an unknown-word case. ##### Step 4. Smooth the first event $$ P_L(I\mid\langle s\rangle)=\frac{3+1}{3+12}=\frac4{15} $$ ##### Step 5. Smooth the second event $$ P_L(love\mid I)=\frac{2+1}{3+12}=\frac3{15} $$ ##### Step 6. Smooth the unseen event $$ P_L(Python\mid love)=\frac{0+1}{2+12}=\frac1{14} $$ ##### Step 7. Smooth the end event $$ P_L(\langle/s\rangle\mid Python)=\frac{1+1}{1+12}=\frac2{13} $$ ##### Step 8. Multiply the smoothed factors $$ P_L=\frac4{15}\times\frac3{15}\times\frac1{14}\times\frac2{13} $$ $$ P_L=\frac{24}{40950}=\frac4{6825}\approx0.000586081 $$ **Final answer.** Unsmoothed P=0. With the stated V=12 and boundaries, Laplace-smoothed P≈0.000586081. If an exam states a different convention—for example excluding boundary tokens from the entire calculation—recompute both V and the set of probability factors consistently. Do not combine a boundary-aware numerator with a boundary-free denominator. ### M2 05: Laplace Smoothing id : nlp-m2-05-laplace-smoothing-discount title : M2 05: Laplace Smoothing title\_confirmed : true topic\_title : Laplace smoothing, unseen sequences and adjusted counts mode : m2 kind : focused status : available duration : 11:11 reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — N-grams and Laplace | Reviewed worked numericals id : overview-ngrams-laplace title : NLP M2 — N-grams and Laplace | Reviewed worked numericals title\_confirmed : true topic\_title : NLP M2: N-Grams & Laplace mode : m2 kind : supplemental status : available duration : 7:41 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Complete counts and four Laplace factors; corrected formulas, conditional-factor wording and rounding. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids : - nlp-m2-03-ngrams-language-models - nlp-m2-04-bigram-trigram-tables - nlp-m2-05-laplace-smoothing-discount These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Adjusted counts and relative discounting Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/m2-evaluation-smoothing--adjusted-count-discount/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/m2-evaluation-smoothing/adjusted-count-discount Unit 3, pages 121–156 The slides also express smoothing through an **adjusted count** c\*. Instead of writing the smoothed probability with denominator N+V, they rescale its numerator so the denominator can remain N. Algebraically these are equivalent: $$ c^*=(c+1)\frac{N}{N+V},\qquad P_L(w)=\frac{c^*}{N} $$ $$ d=\frac{c^*}{c}\quad(c>0) $$ The relative discount d compares adjusted and original counts. It is undefined when c=0 because division by zero is undefined. Add-one smoothing moves the distribution toward uniformity: frequent events are reduced, but some rare seen events may increase. It is not correct to claim that every nonzero count must decrease under Laplace smoothing. **Question. For N=20 and V=5, a word has original count c=10. Find its adjusted count, smoothed probability and relative discount.** #### Given N=20; V=5; c=10. #### To find c\*, P\_L and d. #### Formula and why it applies $$ c^*=(c+1)N/(N+V),\quad P_L=c^*/N,\quad d=c^*/c $$ These expressions rewrite add-one smoothing without changing its probability. #### Solution ##### Step 1. Form the new total $$ N+V=20+5=25 $$ ##### Step 2. Find the adjusted count $$ c^*=(10+1)\frac{20}{25}=11(0.8)=8.8 $$ ##### Step 3. Find the probability $$ P_L=8.8/20=0.44 $$ ##### Step 4. Find the relative discount $$ d=8.8/10=0.88 $$ **Final answer.** c\*=8.8, P\_L=0.44, d=0.88. The original probability 0.5 is reduced. **Question. Under N=20, V=5, a rare word occurs once. Compute its adjusted count and compare the original and smoothed probabilities.** #### Given N=20; V=5; c=1. #### To find c\*, original P and smoothed P\_L. #### Formula and why it applies $$ c^*=(c+1)N/(N+V) $$ The same smoothing rule may increase a rare event's probability. #### Solution ##### Step 1. Original probability $$ P=1/20=0.05 $$ ##### Step 2. Adjusted count $$ c^*=(1+1)(20/25)=1.6 $$ ##### Step 3. Smoothed probability $$ P_L=1.6/20=0.08 $$ **Final answer.** c\*=1.6 and P\_L=0.08, which exceeds the original 0.05. **Question. For N=20 and V=5, an in-vocabulary word has count zero. Find its adjusted count and smoothed probability. Can a relative discount be calculated?** #### Given c=0; N=20; V=5. #### To find c\*, P\_L and whether d exists. #### Formula and why it applies $$ c^*=(c+1)N/(N+V),\qquad P_L=c^*/N $$ The pseudo-count creates positive mass for the unseen vocabulary item. #### Solution ##### Step 1. Adjusted count $$ c^*=(0+1)(20/25)=0.8 $$ ##### Step 2. Smoothed probability $$ P_L=0.8/20=0.04 $$ ##### Step 3. Relative discount d would be 0.8/0. That is undefined, so do not report a finite discount. **Final answer.** c\*=0.8 and P\_L=0.04; relative discount is undefined for c=0. ### M2 05: Laplace Smoothing id : nlp-m2-05-laplace-smoothing-discount title : M2 05: Laplace Smoothing title\_confirmed : true topic\_title : Laplace smoothing, unseen sequences and adjusted counts mode : m2 kind : focused status : available duration : 11:11 reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — N-grams and Laplace | Reviewed worked numericals id : overview-ngrams-laplace title : NLP M2 — N-grams and Laplace | Reviewed worked numericals title\_confirmed : true topic\_title : NLP M2: N-Grams & Laplace mode : m2 kind : supplemental status : available duration : 7:41 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Complete counts and four Laplace factors; corrected formulas, conditional-factor wording and rounding. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids : - nlp-m2-03-ngrams-language-models - nlp-m2-04-bigram-trigram-tables - nlp-m2-05-laplace-smoothing-discount These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Character-model perplexity: keep exact probabilities Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/m2-evaluation-smoothing--class-perplexity-characters/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/m2-evaluation-smoothing/class-perplexity-characters Unit 3, pages 121–156 **Question. A unigram character model gives P(A)=0.25, P(B)=0.50, P(C)=0.25. Find the perplexity of AAA and ABC. Then find both perplexities when the three characters are equally likely.** #### Given Each test string contains 3 scored characters, so N=3. Initial distribution 0.25,0.50,0.25; comparison distribution 1/3,1/3,1/3. #### To find Both sequence probabilities and perplexities under each distribution. #### Formula and why it applies $$ P(c_1c_2c_3)=P(c_1)P(c_2)P(c_3),\qquad PP=P(W)^{-1/3} $$ Characters, not words, are the model's prediction units in this question. Use exact 1/3 for uniform probabilities, rather than rounded 0.33. #### Solution ##### Step 1. Score AAA $$ P(AAA)=0.25^3=\frac1{64}=0.015625 $$ ##### Step 2. Find AAA perplexity $$ PP(AAA)=64^{1/3}=4 $$ ##### Step 3. Score ABC $$ P(ABC)=0.25(0.5)(0.25)=\frac1{32}=0.03125 $$ ##### Step 4. Find ABC perplexity $$ PP(ABC)=32^{1/3}\approx3.174802 $$ ##### Step 5. Change to an exactly uniform model $$ P(AAA)=P(ABC)=(1/3)^3=1/27 $$ ##### Step 6. Find both uniform-model perplexities $$ PP=27^{1/3}=3 $$ **Final answer.** Original model: AAA→4, ABC→3.174802. Uniform model: both→3. These values follow the stated formulas without intermediate rounding. ### M2 06 — Perplexity, logarithms and calculator working id : nlp-m2-06-perplexity-logs-calculator title : M2 06 — Perplexity, logarithms and calculator working title\_confirmed : false topic\_title : Perplexity, logarithms and calculator working mode : m2 kind : focused status : queued duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — Perplexity | Reviewed formulas and solved numericals id : overview-perplexity-logs title : NLP M2 — Perplexity | Reviewed formulas and solved numericals title\_confirmed : true topic\_title : NLP M2 — Perplexity | Reviewed formulas and solved numericals mode : m2 kind : supplemental status : available duration : 9:35 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Readable formulas and all worked examples; corrected comparison conditions and context claims. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids : - nlp-m2-06-perplexity-logs-calculator These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · N-gram interpolation: weighted evidence from several histories Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/u3-smoothing-extensions--interpolation/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/u3-smoothing-extensions/interpolation Unit 3 PDF p147 names these methods. Neither supplied TEE paper asks their numerical calculation; the worked values below are authored examples. **Faculty topic; not an observed numerical PYQ.** Unit 3 PDF p147 names these methods. Neither supplied TEE paper asks their numerical calculation; the worked values below are authored examples. **Interpolation combines several context lengths for every prediction.** A trigram estimate is specific but may have little evidence. Bigram and unigram estimates use less context and can support unseen longer sequences. A weighted mixture combines their information. $$ P_{\mathrm{mix}}(w\mid u,v)=\lambda_3P(w\mid u,v)+\lambda_2P(w\mid v)+\lambda_1P(w) $$ $$ \lambda_1,\lambda_2,\lambda_3\ge0,\qquad\lambda_1+\lambda_2+\lambda_3=1. $$ Each component must be a normalized distribution over the same vocabulary. Then summing the mixture over all possible words gives λ₁+λ₂+λ₃=1. A zero higher-order count can be rescued by positive lower-order evidence. Unlike simple backoff, interpolation can combine all levels even when the trigram was seen. **Question. For predicting tea after I like, suppose trigram, bigram and unigram probabilities are 0, 0.2 and 0.1. Use weights λ₃=0.5, λ₂=0.3 and λ₁=0.2 to calculate the interpolated probability. Repeat for coffee with probabilities 0.4, 0.1 and 0.05. [Authored worked example]** ### Answer #### Given The three distributions use one vocabulary. Nonnegative weights sum to 1. These are supplied probabilities; no count estimation is needed. #### To find Pmix(tea | I,like) and Pmix(coffee | I,like). #### Formula and why it applies $$ P_{mix}=\lambda_3P_3+\lambda_2P_2+\lambda_1P_1 $$ We add weighted alternatives from models, rather than multiply sequential word events. #### Solution ##### Step 1. Check the weights $$ 0.5+0.3+0.2=1. $$ ##### Step 2. Calculate the three tea contributions $$ 0.5(0)=0,\qquad0.3(0.2)=0.06,\qquad0.2(0.1)=0.02. $$ ##### Step 3. Add the tea contributions $$ P_{mix}(tea)=0+0.06+0.02=0.08. $$ ##### Step 4. Calculate the coffee contributions $$ 0.5(0.4)=0.20,\qquad0.3(0.1)=0.03,\qquad0.2(0.05)=0.01. $$ ##### Step 5. Add and compare $$ P_{mix}(coffee)=0.20+0.03+0.01=0.24. $$ Coffee gets three times tea’s probability in this model. Other vocabulary outcomes carry the remaining mass; these two displayed entries are not the whole distribution. **Final answer.** Tea=0.08 and coffee=0.24. A zero trigram probability does not force a zero mixture. **Yaad rakho:** Interpolation = sab context-lengths ka weighted mix. Word-sequence product aur model-mixture sum alag operations hain. This worked lesson was added after the video plan. No matching video has been generated or reviewed. --- ## TEE · Backoff and the important score-versus-probability distinction Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/u3-smoothing-extensions--stupid-backoff/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/u3-smoothing-extensions/stupid-backoff Unit 3 PDF p147 names these methods. Neither supplied TEE paper asks their numerical calculation; the worked values below are authored examples. **Faculty topic; not an observed numerical PYQ.** Unit 3 PDF p147 names these methods. Neither supplied TEE paper asks their numerical calculation; the worked values below are authored examples. **Backoff uses a shorter history when the longer-history estimate lacks evidence.** A properly normalized backoff model discounts some seen-event probability and allocates the remaining mass across unseen continuations using lower-order probabilities and normalization factors. **Stupid Backoff** is a specifically named efficient scoring method in the slides’ list. It uses an observed n-gram relative frequency when available; otherwise it multiplies the shorter-history score by a fixed penalty, commonly 0.4. Its output is generally an **unnormalized score**, not a conditional probability distribution. Do not insert raw Stupid Backoff scores into standard probability-based perplexity and call the result a normalized-model perplexity. $$ S(w\mid h)=\begin{cases}C(h,w)/C(h),&C(h,w)>0\\ \alpha S(w\mid h'),&C(h,w)=0.\end{cases} $$ Here h′ drops the oldest history word; α is the backoff penalty. At the unigram base, use C(w)/N. The letter α here is a penalty, not the forward-HMM variable introduced later. **Question. A bigram scorer has complete vocabulary {a,b}. After h, C(h,a)=2 and C(h,b)=0. The unigram probabilities are P(a)=P(b)=0.5. Apply Stupid Backoff with penalty 0.4 and show whether the scores form a normalized row. [Authored worked example]** ### Answer #### Given C(h)=2. The a bigram is observed; the b bigram is unseen. Unigram probabilities are supplied. #### To find S(a|h), S(b|h), their sum, and whether they are probabilities. #### Formula and why it applies Use relative frequency for an observed pair and α times the unigram base for an unseen pair. The method does not discount a’s observed score in this example. #### Solution ##### Step 1. Score the observed pair $$ S(a\mid h)=2/2=1. $$ ##### Step 2. Back off for the unseen pair $$ S(b\mid h)=0.4(0.5)=0.2. $$ ##### Step 3. Add the complete row $$ 1+0.2=1.2\ne1. $$ ##### Step 4. Interpret the failure of normalization The scores can rank a above b, but the raw row cannot be called a probability distribution. Merely labelling the method “smoothing” does not change this. **Final answer.** Scores are 1 and 0.2; their sum is 1.2. They are useful ranking scores under this rule, not normalized probabilities. | Method | Operation | Normalization | | --- | --- | --- | | Add-one/add-k | Adjust all allowed outcome counts and the total | Normalized with a fixed vocabulary | | Interpolation | Mix normalized distributions using nonnegative weights summing to 1 | Normalized | | Normalized backoff | Discount seen events and distribute remaining mass over unseen events | Normalized when its factors are correctly computed | | Stupid Backoff | Keep seen relative frequencies; penalize shorter-history scores for unseen n-grams | Generally not normalized | Scope: Unit 3 slide 147 lists Stupid Backoff and n-gram interpolation; the class calculations emphasize Laplace. These short worked extensions explain the listed methods without claiming extra official exam weight. This worked lesson was added after the video plan. No matching video has been generated or reviewed. --- ## TEE · Parts of speech and contextual ambiguity Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/m2-pos-tagging--word-classes-pos/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/m2-pos-tagging/word-classes-pos Unit 4, pages 3–46 **Part-of-speech (POS) tagging** assigns a grammatical category to every word token in context, such as noun, verb, adjective or determiner. It is a sequence-labeling task: the input is a sequence of words, and the output is a tag for each word. A tag describes the word's role in this sentence, not its meaning in every possible sentence. | Tag | Role | Example in context | | --- | --- | --- | | NN / NNS | Singular / plural common noun | a book / two books | | NNP | Proper noun | Mary, Mumbai | | VB / VBD / VBZ | Base / past / third-person singular verb | read / read / reads | | JJ | Adjective | the red book | | RB | Adverb | runs quickly | | DT | Determiner | the, a, this | | IN | Preposition or subordinating conjunction | on the table | | PRP | Personal pronoun | she, they | | MD | Modal auxiliary | can, will, should | **Open classes** readily accept new content words: nouns, verbs, adjectives and adverbs. Examples include chatbot and livestream. **Closed classes** contain comparatively stable grammatical/function words: determiners, pronouns, prepositions and conjunctions. These are tendencies, not a claim that language never changes. Input: Book that flight → Context: an imperative request → Output: Book/VB that/DT flight/NN Compare “hand me that **book**”: book is a noun here. Similarly, “I **can** swim” has can/MD, while “the **can** is empty” has can/NN. The spelling stays the same; neighboring words and sentence structure change the correct tag. This is **POS ambiguity**. | Sentence | Tagged interpretation | Reason | | --- | --- | --- | | They watch birds. | They/PRP watch/VBP birds/NNS | watch is the action. | | The watch stopped. | The/DT watch/NN stopped/VBD | watch is the object that stopped. | | She has a light bag. | She/PRP has/VBZ a/DT light/JJ bag/NN | light describes bag. | | Light the lamp. | Light/VB the/DT lamp/NN | Light is an imperative verb. | POS tags support parsing, named entity recognition, question answering, sentiment analysis and word-sense disambiguation. They do not by themselves identify every entity or fully resolve meaning. **Yaad rakho:** Word ka naam same ho sakta hai; sentence mein uska kaam tag decide karta hai. ### NLP M2 — POS tagging and Brill | Reviewed explanation and worked rule id : nlp-m2-07-pos-tags-tagger-methods title : NLP M2 — POS tagging and Brill | Reviewed explanation and worked rule title\_confirmed : true topic\_title : POS tags, ambiguity and tagging methods mode : m2 kind : focused status : available duration : 9:06 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Tagger methods, Brill net gain and the full conditional rule; clarified tagging and imperative explanations. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Lexical, rule-based, stochastic, Brill and neural taggers Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/m2-pos-tagging--tagger-methods/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/m2-pos-tagging/tagger-methods Unit 4, pages 3–46 **A lexical baseline** gives each known word its most frequent training tag. It is simple and often useful, but ignores context. A **rule-based tagger** uses a lexicon of possible tags plus manually designed contextual or morphological rules to resolve ambiguity. For example, after a determiner, a noun reading of “can” may be preferred in “the can”. Rules are usually heuristics with exceptions. “A red car” shows that the word immediately after an article is not always a noun. The suffix -ed does not make “red” a past-tense verb. A robust rule system uses multiple linguistic constraints instead of treating such shortcuts as universal laws. | Method | How it chooses tags | Strength | Limitation | | --- | --- | --- | --- | | Most frequent / lexical | Most common training tag for the word | Simple baseline | Misses contextual changes and unknown words | | Rule-based | Lexicon plus handwritten rules | Readable, controllable linguistic decisions | Expert effort; rules may conflict or need updates | | Stochastic | Scores tag sequences using estimated probabilities | Uses corpus evidence and context | Sparse/unseen events; depends on training domain | | Transformation-based / Brill | Start with baseline, then apply learned correction rules | Readable learned corrections | Needs tagged training data and suitable rule templates | | Neural | Learns contextual representations and sequence predictions | Can exploit richer context | Needs training resources and data; less directly interpretable | **Transformation-based error-driven learning (Brill tagging)** combines a simple starting assignment with learned correction rules. First tag the training corpus with a baseline. Compare against the gold tags. Generate candidate correction rules from templates, such as “change MD to NN after DT”. Apply the rule that fixes the most errors net of newly introduced errors; repeat until further improvement is too small. At test time apply the learned ordered rule list. Baseline: The/DT can/MD was/VBD crushed/VBN → Context rule: change can from MD to NN after DT → Result: The/DT can/NN was/VBD crushed/VBN **Rule-based versus Brill:** a purely rule-based system depends on linguists specifying its rules; a Brill tagger learns which transformations improve a labeled corpus. A **stochastic tagger** such as an HMM finds a high-probability tag sequence using transition and emission probabilities. CRFs and neural sequence taggers are other learning approaches; they do not have to make the same HMM independence assumptions. **Exam-ready answer.** POS tagging assigns a grammatical tag to each word in context. A rule-based tagger uses lexicons and linguistic rules, whereas a stochastic tagger chooses tags through probabilities estimated from a corpus. For example, can is MD in “I can swim” but NN in “the can”; context resolves the ambiguity. ### NLP M2 — POS tagging and Brill | Reviewed explanation and worked rule id : nlp-m2-07-pos-tags-tagger-methods title : NLP M2 — POS tagging and Brill | Reviewed explanation and worked rule title\_confirmed : true topic\_title : POS tags, ambiguity and tagging methods mode : m2 kind : focused status : available duration : 9:06 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Tagger methods, Brill net gain and the full conditional rule; clarified tagging and imperative explanations. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Generative versus discriminative models and Bayes' rule Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/m2-pos-tagging--generative-discriminative/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/m2-pos-tagging/generative-discriminative Unit 4, pages 3–46 Let x be the observed words and y be the desired tag sequence. A **generative model** describes their joint distribution P(x,y), often as P(y)P(x|y). A **discriminative model** learns P(y|x) directly or learns a decision boundary that separates possible labels. Both kinds can be trained with labeled data; “generative” does not mean “always unsupervised”. $$ P(y\mid x)=\frac{P(x\mid y)P(y)}{P(x)} $$ | Quantity | Meaning | POS interpretation | | --- | --- | --- | | P(y) | Prior | How plausible is this tag or tag sequence before seeing the words? | | P(x|y) | Likelihood | How plausible are these words under that tag sequence? | | P(x) | Evidence | Total probability of the observed words across possible labels. | | P(y|x) | Posterior | Probability of the tags after seeing the words. | | Aspect | Generative | Discriminative | | --- | --- | --- | | Models | Joint P(x,y), or prior × likelihood | Conditional P(y|x) or decision boundary | | Classification examples | Naive Bayes | Logistic regression, SVM | | Sequence-labeling examples | HMM | Linear-chain CRF | | Intuition | Model how a label could generate the observation | Model which label fits the observation | **Question. A token has two possible tags, noun N and verb V. P(N)=0.6, P(V)=0.4, P(book|N)=0.2 and P(book|V)=0.5. Calculate P(N|book) and P(V|book).** #### Given Two tags are exhaustive. Priors: 0.6,0.4. Word likelihoods: 0.2,0.5. #### To find Normalized posterior probabilities. #### Formula and why it applies $$ P(t\mid w)=\frac{P(w\mid t)P(t)}{\sum_{u}P(w\mid u)P(u)} $$ Bayes' rule combines prior and likelihood. Normalization is needed to convert scores into posterior probabilities. #### Solution ##### Step 1. Calculate the noun score $$ 0.2\times0.6=0.12 $$ ##### Step 2. Calculate the verb score $$ 0.5\times0.4=0.20 $$ ##### Step 3. Find the evidence $$ P(book)=0.12+0.20=0.32 $$ ##### Step 4. Normalize the noun score $$ P(N\mid book)=0.12/0.32=0.375 $$ ##### Step 5. Normalize the verb score $$ P(V\mid book)=0.20/0.32=0.625 $$ **Final answer.** Verb is more probable in this simplified model: 0.625 versus 0.375. **Question. Two candidate tags have prior-likelihood products 0.06 and 0.14. Find their normalized probabilities.** #### Given Unnormalized scores s₁=0.06 and s₂=0.14; these are the only candidates. #### To find The two posterior probabilities. #### Formula and why it applies $$ P_i=s_i/(s_1+s_2) $$ A common denominator converts the candidate scores to a distribution summing to one. #### Solution ##### Step 1. Add scores $$ 0.06+0.14=0.20 $$ ##### Step 2. Normalize first $$ 0.06/0.20=0.3 $$ ##### Step 3. Normalize second $$ 0.14/0.20=0.7 $$ **Final answer.** Posterior probabilities are 0.3 and 0.7. **Yaad rakho:** Generative: tag se word ka model. Discriminative: word se tag ka decision. ### Bayes Rule & Models id : nlp-m2-02-bayes-generative-discriminative title : Bayes Rule & Models title\_confirmed : true topic\_title : Bayes rule and generative versus discriminative models mode : m2 kind : focused status : available duration : 10:51 reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — Probability and Bayes | Reviewed step-by-step numericals id : overview-probability-bayes title : NLP M2 — Probability and Bayes | Reviewed step-by-step numericals title\_confirmed : true topic\_title : NLP M2 — Probability and Bayes | Reviewed step-by-step numericals mode : m2 kind : supplemental status : available duration : 8:53 reviewed : true notebook\_url : youtube\_url corrected\_visuals : true upload\_pending\_correction : true correction\_note : Complete HMM and Naive Bayes calculations; clarified joint scores versus model posteriors. Full transcript and detected slides checked; short voice corrections included. The corrected YouTube upload is still processing. related\_topic\_ids : - nlp-m2-01-foundations-counts-probability - nlp-m2-02-bayes-generative-discriminative These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Markov chains: read a transition table and multiply a path Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/m2-hmm--markov-chain/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/m2-hmm/markov-chain Unit 4, pages 47–73 **A Markov chain** is a probabilistic model of a sequence of states. Under the first-order Markov assumption, the next state's probability depends on the current state rather than the complete previous history. The **initial probability** tells us how likely each first state is. A **transition probability** tells us how likely a move from one state to another is. $$ P(s_t\mid s_1,\ldots,s_{t-1})=P(s_t\mid s_{t-1}) $$ $$ P(s_1,\ldots,s_T)=P(s_1)\prod_{t=2}^{T}P(s_t\mid s_{t-1}) $$ For T states in a path, use one initial factor and T−1 transition factors. A row-oriented transition matrix must have each complete row sum to 1. Always read the row/column labels; some exam diagrams use a transposed arrangement. | Current → next | Rain | Dry | | --- | --- | --- | | Rain | 0.3 | 0.7 | | Dry | 0.2 | 0.8 | Initial probabilities: P(Rain)=0.4, P(Dry)=0.6. Rows describe the current state; columns describe the next state. **Question. For the Rain/Dry Markov chain with initial P(Rain)=0.4 and P(Dry)=0.6, and transitions Rain→Rain=0.3, Rain→Dry=0.7, Dry→Rain=0.2, Dry→Dry=0.8, find the probability of Dry, Dry, Rain, Rain.** #### Given | Current → next | Rain | Dry | | --- | --- | --- | | Rain | 0.3 | 0.7 | | Dry | 0.2 | 0.8 | Initial probabilities: P(Rain)=0.4, P(Dry)=0.6. Rows describe the current state; columns describe the next state. #### To find P(Dry,Dry,Rain,Rain). #### Formula and why it applies $$ P(D,D,R,R)=P(D)P(D\mid D)P(R\mid D)P(R\mid R) $$ Each step conditions on the immediately preceding state. The sequence contains four states, so it needs one initial plus three transition factors. #### Solution ##### Step 1. Start in Dry $$ P(D)=0.6 $$ ##### Step 2. Move Dry to Dry $$ 0.6\times0.8=0.48 $$ ##### Step 3. Move Dry to Rain $$ 0.48\times0.2=0.096 $$ ##### Step 4. Move Rain to Rain $$ 0.096\times0.3=0.0288 $$ **Final answer.** Path probability = 0.0288. **Question. For initial probabilities Rain=0.4, Dry=0.6 and transition rows Rain=(0.3,0.7), Dry=(0.2,0.8) in column order Rain,Dry, find the probability of Rain, Dry, Rain.** #### Given | Current → next | Rain | Dry | | --- | --- | --- | | Rain | 0.3 | 0.7 | | Dry | 0.2 | 0.8 | Initial probabilities: P(Rain)=0.4, P(Dry)=0.6. Rows describe the current state; columns describe the next state. #### To find P(Rain,Dry,Rain). #### Formula and why it applies $$ P(R,D,R)=P(R)P(D\mid R)P(R\mid D) $$ Do not replace Dry→Rain with Rain→Dry: the two directional probabilities differ. #### Solution ##### Step 1. Initial Rain $$ P(R)=0.4 $$ ##### Step 2. First transition $$ P(D\mid R)=0.7,\quad0.4(0.7)=0.28 $$ ##### Step 3. Second transition $$ P(R\mid D)=0.2,\quad0.28(0.2)=0.056 $$ **Final answer.** Path probability = 0.056. **Question. For the same fully specified chain—initial Rain=0.4, Dry=0.6; Rain→Rain=0.3, Rain→Dry=0.7, Dry→Rain=0.2, Dry→Dry=0.8—find P(Dry,Dry). Also find the conditional probability of next Dry given current Dry.** #### Given | Current → next | Rain | Dry | | --- | --- | --- | | Rain | 0.3 | 0.7 | | Dry | 0.2 | 0.8 | Initial probabilities: P(Rain)=0.4, P(Dry)=0.6. Rows describe the current state; columns describe the next state. #### To find The two-state joint probability and the one-step conditional probability. #### Formula and why it applies $$ P(D,D)=P(D)P(D\mid D) $$ The joint path includes an uncertain initial state. The conditional transition assumes the current state is already Dry. #### Solution ##### Step 1. Read the conditional transition $$ P(D\mid D)=0.8 $$ ##### Step 2. Include the initial state for the joint path $$ P(D,D)=0.6\times0.8=0.48 $$ **Final answer.** P(Dry,Dry)=0.48; P(next Dry|current Dry)=0.8. These answer different questions. **Yaad rakho:** Path: pehle start probability, phir har arrow ki probability. ### M2 08: Markov Paths & HMMs id : nlp-m2-08-markov-hmm-components title : M2 08: Markov Paths & HMMs title\_confirmed : true topic\_title : Markov paths, HMM components and training counts mode : m2 kind : focused status : available duration : 9:45 reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — HMM and Viterbi | Reviewed worked numericals id : overview-hmm-viterbi title : NLP M2 — HMM and Viterbi | Reviewed worked numericals title\_confirmed : true topic\_title : HMM Prob & Viterbi mode : m2 kind : supplemental status : available duration : 8:30 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Complete HMM factors, all Viterbi candidate scores and backpointers; clarified zero emissions. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids : - nlp-m2-08-markov-hmm-components - nlp-m2-09-hmm-joint-paths - nlp-m2-10-viterbi-decoding These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Hidden states, observations, transitions and emissions Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/m2-hmm--hmm-components/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/m2-hmm/hmm-components Unit 4, pages 47–73 **A Hidden Markov Model (HMM)** is a probabilistic sequence model in which the underlying state sequence is unobserved, while each state emits an observed item. For POS tagging, words are observed and their grammatical tags are hidden. The task is to infer a suitable tag sequence from the words. Hidden: Noun → Hidden: Verb → Hidden: Noun Observed: birds → Observed: eat → Observed: seeds Horizontal state-to-state links are **transitions**. The relation from each hidden state to its observed word is an **emission**. A transition P(V|N) asks how often a verb tag follows a noun tag. An emission P(eat|V) asks how often the observed word eat appears among verb-tagged tokens. | Component | Notation | Meaning | | --- | --- | --- | | State set | S={N,V,…} | Possible hidden tags. | | Observation vocabulary | O={birds,eat,seeds,…} | Visible word tokens. | | Initial distribution | πⱼ=P(t₁=j) | Probability of starting in tag j. | | Transition matrix | aᵢⱼ=P(tₖ=j|tₖ₋₁=i) | Probability of next tag j after i. | | Emission matrix | bⱼ(w)=P(w|tag j) | Probability that state j emits word w. | Two basic assumptions are used: the current hidden state depends only on the previous hidden state, and the current observation depends only on the current hidden state. A standard first-order HMM therefore does not explicitly condition an emitted word on the whole sentence. $$ P(t_{1:T},w_{1:T})=\pi_{t_1}b_{t_1}(w_1)\prod_{k=2}^{T}a_{t_{k-1},t_k}b_{t_k}(w_k) $$ Read the formula as **start × first emission × transition × emission × transition × emission…**. There are T emission factors and T−1 transitions, plus one initial factor. If the question supplies an explicit end-state probability, multiply that too. Do not invent an end factor when the model has none. **Three different questions.** (1) Scoring a specified tag path uses the joint product above. (2) Finding the probability of the observed words sums scores over all possible tag paths. (3) Finding the best tag path maximizes the path score, typically using Viterbi. A joint path score is not a normalized posterior P(tags|words). **Yaad rakho:** Transition = tag se agla tag. Emission = tag se visible word. ### M2 08: Markov Paths & HMMs id : nlp-m2-08-markov-hmm-components title : M2 08: Markov Paths & HMMs title\_confirmed : true topic\_title : Markov paths, HMM components and training counts mode : m2 kind : focused status : available duration : 9:45 reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — HMM and Viterbi | Reviewed worked numericals id : overview-hmm-viterbi title : NLP M2 — HMM and Viterbi | Reviewed worked numericals title\_confirmed : true topic\_title : HMM Prob & Viterbi mode : m2 kind : supplemental status : available duration : 8:30 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Complete HMM factors, all Viterbi candidate scores and backpointers; clarified zero emissions. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids : - nlp-m2-08-markov-hmm-components - nlp-m2-09-hmm-joint-paths - nlp-m2-10-viterbi-decoding These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Estimate transition and emission probabilities from tagged data Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/m2-hmm--transition-emission-estimation/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/m2-hmm/transition-emission-estimation Unit 4, pages 47–73 $$ P(t'\mid t)=\frac{C(t,t')}{C(t)\text{ with a following tag/end event}} $$ $$ P(w\mid t)=\frac{C(w,t)}{C(t)} $$ In a transition denominator, count the previous tag's outgoing events. In an emission denominator, count all tokens bearing that tag. With explicit end transitions the totals align conveniently. Keep the two numerator types separate: tag→tag counts are not word→tag counts. **Question. A tagged corpus contains 116454 determiner tags DT, each counted with an outgoing event. DT is followed by NN 56509 times. Estimate P(NN|DT).** #### Given C(DT)=116454; C(DT,NN)=56509. #### To find The DT→NN transition probability. #### Formula and why it applies $$ P(NN\mid DT)=C(DT,NN)/C(DT) $$ The denominator is the number of previous DT contexts because DT is the condition. #### Solution ##### Step 1. Identify the transition count DT followed by NN occurs 56509 times. ##### Step 2. Identify the conditional total There are 116454 DT contexts. ##### Step 3. Divide $$ P(NN\mid DT)=\frac{56509}{116454}\approx0.485247 $$ **Final answer.** P(NN|DT)≈0.485247. **Question. The tag VBZ occurs 21627 times, and 10073 of those tokens are the word “is”. Estimate P(is|VBZ).** #### Given C(VBZ)=21627; C(is,VBZ)=10073. #### To find The emission probability of is from VBZ. #### Formula and why it applies $$ P(is\mid VBZ)=C(is,VBZ)/C(VBZ) $$ The condition is the tag VBZ; we ask which word it emits. #### Solution ##### Step 1. Choose the word-tag count The required word is is with tag VBZ: count 10073. ##### Step 2. Choose the tag total All VBZ-tagged word tokens: count 21627. ##### Step 3. Divide $$ P(is\mid VBZ)=\frac{10073}{21627}\approx0.465760 $$ **Final answer.** P(is|VBZ)≈0.465760. This is different from P(VBZ|is). **Question. Use the tagged sentences “dogs/N chase/V cats/N” and “cats/N sleep/V”. Estimate the initial tag probabilities, all nonzero transitions including END, and emissions.** #### Given Corpus: START dogs/N chase/V cats/N END; START cats/N sleep/V END. #### To find Initial distribution, transition rows and emission rows. #### Formula and why it applies $$ \pi_t=C(START,t)/2,\quad a_{tu}=C(t,u)/C(t),\quad b_t(w)=C(w,t)/C(t) $$ The corpus explicitly tells us the tags, so we can count training frequencies directly. #### Solution ##### Step 1. Count tag tokens N occurs 3 times: dogs, cats, cats. V occurs 2 times: chase, sleep. ##### Step 2. Count sentence starts Both sentences start with N. $$ \pi_N=2/2=1,\quad \pi_V=0/2=0 $$ ##### Step 3. Count transitions | Pair | Count | | --- | --- | | N→V | 2 | | N→END | 1 | | V→N | 1 | | V→END | 1 | ##### Step 4. Normalize transition rows | Current | To N | To V | To END | | --- | --- | --- | --- | | N | 0 | 2/3 | 1/3 | | V | 1/2 | 0 | 1/2 | ##### Step 5. Count and normalize emissions | Tag | dogs | cats | chase | sleep | | --- | --- | --- | --- | --- | | N | 1/3 | 2/3 | 0 | 0 | | V | 0 | 0 | 1/2 | 1/2 | **Final answer.** π=(1,0); transitions and emissions are given in the completed tables. Each complete probability row sums to 1. **Yaad rakho:** Tag→tag table aur tag→word table ko mix mat karna. ### M2 08: Markov Paths & HMMs id : nlp-m2-08-markov-hmm-components title : M2 08: Markov Paths & HMMs title\_confirmed : true topic\_title : Markov paths, HMM components and training counts mode : m2 kind : focused status : available duration : 9:45 reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — HMM and Viterbi | Reviewed worked numericals id : overview-hmm-viterbi title : NLP M2 — HMM and Viterbi | Reviewed worked numericals title\_confirmed : true topic\_title : HMM Prob & Viterbi mode : m2 kind : supplemental status : available duration : 8:30 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Complete HMM factors, all Viterbi candidate scores and backpointers; clarified zero emissions. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids : - nlp-m2-08-markov-hmm-components - nlp-m2-09-hmm-joint-paths - nlp-m2-10-viterbi-decoding These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · HMM path probabilities: three complete numerical examples Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/m2-hmm--hot-cold-paths/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/m2-hmm/hot-cold-paths Unit 4, pages 47–73 The following model uses hidden weather states Hot (H) and Cold (C). Observations are numbers 1, 2 and 3. These numbers are observation labels, not quantities to multiply directly. Their emission probabilities come from the table. | Initial state | Probability | | --- | --- | | H | 0.8 | | C | 0.2 | | From → to | H | C | | --- | --- | --- | | H | 0.7 | 0.3 | | C | 0.4 | 0.6 | | Emission state | Observation 1 | Observation 2 | Observation 3 | | --- | --- | --- | --- | | H | 0.2 | 0.4 | 0.4 | | C | 0.5 | 0.4 | 0.1 | **Question. An HMM has π(H)=0.8, π(C)=0.2; H→H=0.7, H→C=0.3, C→H=0.4, C→C=0.6. Emissions for observations (1,2,3) are H:(0.2,0.4,0.4) and C:(0.5,0.4,0.1). Find the joint probability of hidden path H,C,H and observations 1,3,1.** #### Given Hidden path H→C→H. Observations 1→3→1. Model probabilities are specified in the question. There is no END factor. #### To find P(H,C,H; 1,3,1). #### Formula and why it applies $$ P=\pi_H b_H(1)a_{HC}b_C(3)a_{CH}b_H(1) $$ There are three hidden states and observations: three emissions, two transitions, and one initial probability. #### Solution ##### Step 1. Start Hot and emit 1 $$ 0.8\times0.2=0.16 $$ ##### Step 2. Transition Hot to Cold $$ 0.16\times0.3=0.048 $$ ##### Step 3. Cold emits 3 $$ 0.048\times0.1=0.0048 $$ ##### Step 4. Transition Cold to Hot $$ 0.0048\times0.4=0.00192 $$ ##### Step 5. Hot emits the last 1 $$ 0.00192\times0.2=0.000384 $$ **Final answer.** Joint probability = 0.000384 = 3.84×10⁻⁴. **Question. For π(H)=0.8, π(C)=0.2; H→H=0.7, H→C=0.3, C→H=0.4, C→C=0.6; emissions H:(0.2,0.4,0.4), C:(0.5,0.4,0.1) for labels (1,2,3), find the joint probability of path H,H,H and observations 1,3,1.** #### Given State path H→H→H, observations 1→3→1; no END factor. #### To find P(H,H,H;1,3,1). #### Formula and why it applies $$ P=\pi_H b_H(1)a_{HH}b_H(3)a_{HH}b_H(1) $$ The state remains Hot; each transition still contributes its probability 0.7. #### Solution ##### Step 1. Initial state and first emission $$ 0.8(0.2)=0.16 $$ ##### Step 2. First self-transition $$ 0.16(0.7)=0.112 $$ ##### Step 3. Emit observation 3 $$ 0.112(0.4)=0.0448 $$ ##### Step 4. Second self-transition $$ 0.0448(0.7)=0.03136 $$ ##### Step 5. Emit final observation 1 $$ 0.03136(0.2)=0.006272 $$ **Final answer.** Joint probability = 0.006272. A self-transition is not automatically probability 1. **Question. For π(H)=0.8, π(C)=0.2; transitions H:(0.7,0.3), C:(0.4,0.6) in next-state order H,C; emissions H:(0.2,0.4,0.4), C:(0.5,0.4,0.1) for observations 1,2,3, find the score of path C,C,H with observations 1,3,1.** #### Given Hidden path C→C→H. Observation sequence 1→3→1. No end-state factor. #### To find The joint path-and-observation probability. #### Formula and why it applies $$ P=\pi_C b_C(1)a_{CC}b_C(3)a_{CH}b_H(1) $$ The initial state is Cold, so use π(C), not π(H). Each emission is selected from the state at the same position. #### Solution ##### Step 1. Start Cold and emit 1 $$ 0.2(0.5)=0.1 $$ ##### Step 2. Stay Cold $$ 0.1(0.6)=0.06 $$ ##### Step 3. Emit 3 from Cold $$ 0.06(0.1)=0.006 $$ ##### Step 4. Move Cold to Hot $$ 0.006(0.4)=0.0024 $$ ##### Step 5. Emit 1 from Hot $$ 0.0024(0.2)=0.00048 $$ **Final answer.** Joint probability = 0.00048. **Yaad rakho:** Har column par: state ka emission. Columns ke beech: transition. ### HMM Path Probabilities id : nlp-m2-09-hmm-joint-paths title : HMM Path Probabilities title\_confirmed : true topic\_title : Complete HMM path scores and observation probabilities mode : m2 kind : focused status : available duration : 8:34 reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — HMM and Viterbi | Reviewed worked numericals id : overview-hmm-viterbi title : NLP M2 — HMM and Viterbi | Reviewed worked numericals title\_confirmed : true topic\_title : HMM Prob & Viterbi mode : m2 kind : supplemental status : available duration : 8:30 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Complete HMM factors, all Viterbi candidate scores and backpointers; clarified zero emissions. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids : - nlp-m2-08-markov-hmm-components - nlp-m2-09-hmm-joint-paths - nlp-m2-10-viterbi-decoding These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · POS HMM worked corpus: Mary, Jane, can, will and spot Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/m2-hmm--tagging-corpus-case/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/m2-hmm/tagging-corpus-case Unit 4, pages 47–73 The class example contains words that can be either names or ordinary verbs/modal auxiliaries. The term is **modal** (M), not “model”. To avoid guessing tags from spelling, the training corpus below supplies them explicitly and lowercases words for the emission vocabulary. | Training sentence | Gold tags | | --- | --- | | mary jane can see will | N N M V N | | spot will see mary | N M V N | | will jane spot mary | M N V N | | mary will pat spot | N M V N | There are 9 noun tokens, 4 modal tokens and 4 verb tokens. Each sentence includes a start and end transition. For nouns, the next event can be another noun, modal, verb or END. | Current → next | N | M | V | END | | --- | --- | --- | --- | --- | | START | 3/4 | 1/4 | 0 | 0 | | N | 1/9 | 3/9 | 1/9 | 4/9 | | M | 1/4 | 0 | 3/4 | 0 | | V | 4/4 | 0 | 0 | 0 | | Tag → word | mary | jane | can | see | will | spot | pat | | --- | --- | --- | --- | --- | --- | --- | --- | | N | 4/9 | 2/9 | 0 | 0 | 1/9 | 2/9 | 0 | | M | 0 | 0 | 1/4 | 0 | 3/4 | 0 | 0 | | V | 0 | 0 | 0 | 2/4 | 0 | 1/4 | 1/4 | **Question. Using the fully tagged four-sentence corpus above, calculate the joint score of “will jane spot mary” assigned tags M,N,V,N, including END.** #### Given Required factors from the tables: START→M=1/4; emit will|M=3/4; M→N=1/4; jane|N=2/9; N→V=1/9; spot|V=1/4; V→N=1; mary|N=4/9; N→END=4/9. #### To find The specified tag-path score, including words and sentence end. #### Formula and why it applies $$ P=\pi_M b_M(will)a_{MN}b_N(jane)a_{NV}b_V(spot)a_{VN}b_N(mary)a_{N,END} $$ Both transitions and emissions are needed. A tag transition alone cannot tell us whether a particular word is plausible. #### Solution ##### Step 1. Start and emit will $$ \frac14\times\frac34=\frac3{16} $$ ##### Step 2. Move to N and emit jane $$ \frac3{16}\times\frac14\times\frac29=\frac1{96} $$ ##### Step 3. Move to V and emit spot $$ \frac1{96}\times\frac19\times\frac14=\frac1{3456} $$ ##### Step 4. Move to N and emit mary $$ \frac1{3456}\times1\times\frac49=\frac1{7776} $$ ##### Step 5. End the sentence $$ \frac1{7776}\times\frac49=\frac1{17496}\approx0.0000571559 $$ **Final answer.** The joint score is 1/17496 ≈ 0.0000571559. **Question. For the same tagged corpus and tables, explain the unsmoothed score of “will can spot mary” assigned tags M,V,N,N.** #### Given The proposed second word is can with hidden tag V. In the training corpus, can occurs only with M, so b\_V(can)=0. #### To find Whether the complete path score is zero and what this means. #### Formula and why it applies $$ P(\text{path,words})=(\text{other required factors})\times b_V(can) $$ A product containing one zero factor is zero even if every other factor is positive. #### Solution ##### Step 1. Find the critical emission $$ b_V(can)=C(can,V)/C(V)=0/4=0 $$ ##### Step 2. Use the full-product consequence $$ P(\text{path,words})=0 $$ ##### Step 3. Interpret correctly This unsmoothed corpus model has no evidence for can as a verb. It does not prove that a verb usage of can is impossible in English. **Final answer.** The unsmoothed path score is zero. ### HMM Path Probabilities id : nlp-m2-09-hmm-joint-paths title : HMM Path Probabilities title\_confirmed : true topic\_title : Complete HMM path scores and observation probabilities mode : m2 kind : focused status : available duration : 8:34 reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — HMM and Viterbi | Reviewed worked numericals id : overview-hmm-viterbi title : NLP M2 — HMM and Viterbi | Reviewed worked numericals title\_confirmed : true topic\_title : HMM Prob & Viterbi mode : m2 kind : supplemental status : available duration : 8:30 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Complete HMM factors, all Viterbi candidate scores and backpointers; clarified zero emissions. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids : - nlp-m2-08-markov-hmm-components - nlp-m2-09-hmm-joint-paths - nlp-m2-10-viterbi-decoding These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Why exhaustive search grows exponentially Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/m2-hmm-decoding--sequence-count/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/m2-hmm-decoding/sequence-count Unit 4 page 73: Viterbi motivation, explained with small worked extensions If each of T observed words can receive any of N tags, there are N choices at each position. Thus the total number of complete tag sequences is N multiplied by itself T times. This counts possible assignments before grammar rules or zero probabilities eliminate any. $$ \text{Number of tag sequences}=N^T $$ **Question. A four-word sentence is tagged using three possible POS tags per word. How many complete tag sequences are possible?** #### Given N=3 possible tags per position; T=4 word positions. #### To find The number of tag sequences. #### Formula and why it applies $$ M=N^T $$ Every independent position contributes a factor of three choices. #### Solution ##### Step 1. Write one factor per position $$ M=3\times3\times3\times3 $$ ##### Step 2. Multiply $$ 3\times3=9,\quad9\times3=27,\quad27\times3=81 $$ **Final answer.** 81 possible tag sequences. **Question. A five-word sentence has two candidate tags per word. Count its tag sequences.** #### Given N=2; T=5. #### To find M. #### Formula and why it applies $$ M=N^T $$ The length is the exponent, not a multiplier. #### Solution ##### Step 1. Substitute $$ M=2^5 $$ ##### Step 2. Evaluate $$ 2\times2\times2\times2\times2=32 $$ **Final answer.** 32 possible sequences. **Question. Using 45 possible tags, compare the number of tag sequences for sentences of length 2 and 3.** #### Given N=45. First T=2, then T=3. #### To find Both counts and the growth factor. #### Formula and why it applies $$ M_2=45^2,\quad M_3=45^3 $$ Adding one position multiplies the possibilities by the number of tag choices. #### Solution ##### Step 1. Two words $$ 45^2=2025 $$ ##### Step 2. Three words $$ 45^3=2025\times45=91125 $$ ##### Step 3. Compare $$ 91125/2025=45 $$ **Final answer.** 2 words: 2025 paths. 3 words: 91125 paths. One additional word multiplies the count by 45. The slides introduce Viterbi to avoid enumerating every path. It reuses the best prefix score for each ending state. For a dense first-order HMM, it needs O(TN²) recurrence work instead of enumerating NT complete paths. **Yaad rakho:** Choices per word ki power = sentence length. ### M2 10 Viterbi Decoding id : nlp-m2-10-viterbi-decoding title : M2 10 Viterbi Decoding title\_confirmed : true topic\_title : Viterbi decoding with backpointers mode : m2 kind : focused status : available duration : 9:34 reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — HMM and Viterbi | Reviewed worked numericals id : overview-hmm-viterbi title : NLP M2 — HMM and Viterbi | Reviewed worked numericals title\_confirmed : true topic\_title : HMM Prob & Viterbi mode : m2 kind : supplemental status : available duration : 8:30 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Complete HMM factors, all Viterbi candidate scores and backpointers; clarified zero emissions. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids : - nlp-m2-08-markov-hmm-components - nlp-m2-09-hmm-joint-paths - nlp-m2-10-viterbi-decoding These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Viterbi decoding: compare incoming paths, keep the best, trace back Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/m2-hmm-decoding--viterbi-explained/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/m2-hmm-decoding/viterbi-explained Unit 4 page 73: Viterbi motivation, explained with small worked extensions **The Viterbi algorithm** finds the most likely hidden-state sequence for a fixed observation sequence. At each position, it keeps the best path ending in each state. This is safe because, under the HMM assumptions, future extensions depend on that ending state, not the discarded details of the prefix. $$ \delta_1(j)=\pi_jb_j(o_1) $$ $$ \delta_t(j)=b_j(o_t)\max_i[\delta_{t-1}(i)a_{ij}] $$ $$ \psi_t(j)=\arg\max_i[\delta_{t-1}(i)a_{ij}] $$ | Symbol | Meaning in ordinary words | | --- | --- | | δₜ(j) | Score of the best prefix that ends in state j at position t. | | ψₜ(j) | Previous state that supplied that best prefix; keep it as a backpointer. | | max | Choose the larger candidate score; do not add. | | argmax | Remember which previous state won, rather than only its value. | Initialize: start × first emission → For each next state: compare incoming scores → Multiply winner by this state's emission → Keep score + winning predecessor → Choose best final state and trace arrows backward Do not choose the best state independently at each word. The best last state can come from a different best predecessor than a local guess. Backpointers recover one consistent complete path. If scores tie, either tied path is optimal under the model; state the tie. **Question. Use an HMM with π(H)=0.8, π(C)=0.2, transitions H:(0.7,0.3), C:(0.4,0.6) in H,C order, and emissions H:(0.2,0.4,0.4), C:(0.5,0.4,0.1) for observations 1,2,3. Find the most likely hidden path for observations 1,3,1 using Viterbi.** #### Given Initial probabilities: π(H)=0.8, π(C)=0.2. Transition matrix: H→H=0.7, H→C=0.3, C→H=0.4, C→C=0.6. Emissions for labels (1,2,3): H=(0.2,0.4,0.4), C=(0.5,0.4,0.1). There is no END-state factor. #### To find The best hidden-state sequence and its joint score. #### Formula and why it applies $$ \delta_1(j)=\pi_jb_j(o_1),\quad\delta_t(j)=b_j(o_t)\max_i[\delta_{t-1}(i)a_{ij}] $$ Viterbi keeps two best prefix scores per column, one ending H and one ending C. #### Solution ##### Step 1. Initialize at observation 1 $$ \delta_1(H)=0.8(0.2)=0.16 $$ $$ \delta_1(C)=0.2(0.5)=0.10 $$ ##### Step 2. At observation 3, calculate the H cell $$ H\to H:\ 0.16(0.7)=0.112 $$ $$ C\to H:\ 0.10(0.4)=0.040 $$ $$ \delta_2(H)=\max(0.112,0.040)(0.4)=0.0448 $$ Backpointer for this H cell: H. ##### Step 3. At observation 3, calculate the C cell $$ H\to C:\ 0.16(0.3)=0.048 $$ $$ C\to C:\ 0.10(0.6)=0.060 $$ $$ \delta_2(C)=\max(0.048,0.060)(0.1)=0.006 $$ Backpointer for this C cell: C. ##### Step 4. At final observation 1, calculate H $$ H\to H:\ 0.0448(0.7)=0.03136 $$ $$ C\to H:\ 0.006(0.4)=0.0024 $$ $$ \delta_3(H)=0.03136(0.2)=0.006272 $$ Backpointer: H. ##### Step 5. At final observation 1, calculate C $$ H\to C:\ 0.0448(0.3)=0.01344 $$ $$ C\to C:\ 0.006(0.6)=0.0036 $$ $$ \delta_3(C)=0.01344(0.5)=0.00672 $$ Backpointer: H. ##### Step 6. Select the final state 0.00672 > 0.006272, so the best final state is C. ##### Step 7. Trace back Final C came from H at position2. That H came from H at position1. Reading forward gives H,H,C. ##### Step 8. Verify the recovered complete path $$ 0.8(0.2)(0.7)(0.4)(0.3)(0.5)=0.00672 $$ **Final answer.** Best path H→H→C with joint score 0.00672. **Question. For π(H)=0.8, π(C)=0.2; transitions H:(0.7,0.3), C:(0.4,0.6); emissions H:(0.2,0.4,0.4), C:(0.5,0.4,0.1) for labels1,2,3, decode observations 3,1.** #### Given Initial probabilities: π(H)=0.8, π(C)=0.2. Transition matrix: H→H=0.7, H→C=0.3, C→H=0.4, C→C=0.6. Emissions for labels (1,2,3): H=(0.2,0.4,0.4), C=(0.5,0.4,0.1). There is no END-state factor. #### To find Best two-state hidden path and score. #### Formula and why it applies $$ \delta_1(j)=\pi_jb_j(3),\quad\delta_2(j)=b_j(1)\max_i[\delta_1(i)a_{ij}] $$ The same recurrence applies even though the observation order changed. #### Solution ##### Step 1. Initialize observation 3 $$ \delta_1(H)=0.8(0.4)=0.32,\quad\delta_1(C)=0.2(0.1)=0.02 $$ ##### Step 2. Calculate the final H $$ \max(0.32(0.7),0.02(0.4))(0.2) $$ $$ =\max(0.224,0.008)(0.2)=0.0448 $$ Winning predecessor H. ##### Step 3. Calculate the final C $$ \max(0.32(0.3),0.02(0.6))(0.5) $$ $$ =\max(0.096,0.012)(0.5)=0.048 $$ Winning predecessor H. ##### Step 4. Select and trace Final C wins with 0.048 and came from H. The path is H,C. **Final answer.** Best path H→C, score 0.048. **Question. For π(H)=0.8, π(C)=0.2; transitions H:(0.7,0.3), C:(0.4,0.6); emissions H:(0.2,0.4,0.4), C:(0.5,0.4,0.1) for labels1,2,3, decode observations 2,2.** #### Given Initial probabilities: π(H)=0.8, π(C)=0.2. Transition matrix: H→H=0.7, H→C=0.3, C→H=0.4, C→C=0.6. Emissions for labels (1,2,3): H=(0.2,0.4,0.4), C=(0.5,0.4,0.1). There is no END-state factor. #### To find Best path and score. #### Formula and why it applies $$ \delta_1(j)=\pi_jb_j(2),\quad\delta_2(j)=b_j(2)\max_i[\delta_1(i)a_{ij}] $$ Both states emit2 with probability 0.4, but their initial and transition probabilities still differ. #### Solution ##### Step 1. Initialize $$ \delta_1(H)=0.8(0.4)=0.32,\quad\delta_1(C)=0.2(0.4)=0.08 $$ ##### Step 2. Final H $$ \max(0.32(0.7),0.08(0.4))(0.4) $$ $$ =\max(0.224,0.032)(0.4)=0.0896 $$ Predecessor H. ##### Step 3. Final C $$ \max(0.32(0.3),0.08(0.6))(0.4) $$ $$ =\max(0.096,0.048)(0.4)=0.0384 $$ Predecessor H. ##### Step 4. Select and trace The final H score 0.0896 is larger. Its predecessor is H. **Final answer.** Best path H→H with score 0.0896. **Yaad rakho:** Har destination ke liye incoming paths compare; winner ka arrow yaad. ### M2 10 Viterbi Decoding id : nlp-m2-10-viterbi-decoding title : M2 10 Viterbi Decoding title\_confirmed : true topic\_title : Viterbi decoding with backpointers mode : m2 kind : focused status : available duration : 9:34 reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — HMM and Viterbi | Reviewed worked numericals id : overview-hmm-viterbi title : NLP M2 — HMM and Viterbi | Reviewed worked numericals title\_confirmed : true topic\_title : HMM Prob & Viterbi mode : m2 kind : supplemental status : available duration : 8:30 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Complete HMM factors, all Viterbi candidate scores and backpointers; clarified zero emissions. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids : - nlp-m2-08-markov-hmm-components - nlp-m2-09-hmm-joint-paths - nlp-m2-10-viterbi-decoding These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · A path score is not the probability of the observed sentence Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/m2-hmm-decoding--joint-vs-posterior/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/m2-hmm-decoding/joint-vs-posterior Unit 4 page 73: Viterbi motivation, explained with small worked extensions **Question. An HMM has π(H)=0.8, π(C)=0.2; transitions H:(0.7,0.3), C:(0.4,0.6); emissions H:(0.2,0.4,0.4), C:(0.5,0.4,0.1) for labels1,2,3. For observations 1,3, calculate the probability of the observations by summing all four two-state paths.** #### Given Initial probabilities: π(H)=0.8, π(C)=0.2. Transition matrix: H→H=0.7, H→C=0.3, C→H=0.4, C→C=0.6. Emissions for labels (1,2,3): H=(0.2,0.4,0.4), C=(0.5,0.4,0.1). There is no END-state factor. #### To find P(observations 1,3), not just the best-path score. #### Formula and why it applies $$ P(o_1,o_2)=\sum_{s_1,s_2}\pi_{s_1}b_{s_1}(o_1)a_{s_1s_2}b_{s_2}(o_2) $$ The hidden path is unspecified, so its mutually exclusive alternatives must be summed. #### Solution ##### Step 1. Score H,H $$ 0.8(0.2)(0.7)(0.4)=0.0448 $$ ##### Step 2. Score H,C $$ 0.8(0.2)(0.3)(0.1)=0.0048 $$ ##### Step 3. Score C,H $$ 0.2(0.5)(0.4)(0.4)=0.016 $$ ##### Step 4. Score C,C $$ 0.2(0.5)(0.6)(0.1)=0.006 $$ ##### Step 5. Add all paths $$ P(1,3)=0.0448+0.0048+0.016+0.006=0.0716 $$ **Final answer.** Observation probability =0.0716. The best single-path score 0.0448 is a different quantity. If asked for the posterior of path H,H given observations 1,3, normalize: 0.0448/0.0716 ≈0.625698. Viterbi chooses the best path using joint scores because the observation denominator is common to every candidate. It does not compute that denominator by taking a maximum. ### HMM Path Probabilities id : nlp-m2-09-hmm-joint-paths title : HMM Path Probabilities title\_confirmed : true topic\_title : Complete HMM path scores and observation probabilities mode : m2 kind : focused status : available duration : 8:34 reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — HMM and Viterbi | Reviewed worked numericals id : overview-hmm-viterbi title : NLP M2 — HMM and Viterbi | Reviewed worked numericals title\_confirmed : true topic\_title : HMM Prob & Viterbi mode : m2 kind : supplemental status : available duration : 8:30 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Complete HMM factors, all Viterbi candidate scores and backpointers; clarified zero emissions. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids : - nlp-m2-08-markov-hmm-components - nlp-m2-09-hmm-joint-paths - nlp-m2-10-viterbi-decoding These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Chunking groups words; chinking removes words from a candidate chunk Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/m2-chunks-grammar--chunking-chinking/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/m2-chunks-grammar/chunking-chinking Unit 4, pages 14–19, 74–97 **Chunking**, also called shallow parsing, groups adjacent tagged words into meaningful non-overlapping units, commonly noun phrases. It usually identifies local phrase boundaries without producing the complete hierarchical structure of a sentence. **Chinking** removes specified tokens from an existing candidate chunk, splitting or trimming it. The/DT small/JJ cat/NN sat/VBD on/IN the/DT mat/NN → NP: [The small cat] → Outside: sat on → NP: [the mat] A POS tagger labels individual words. A chunker groups words into phrases. A full parser shows how phrases relate recursively to form the entire sentence. These are related tasks with different outputs. | Operation | Input | Output | | --- | --- | --- | | POS tagging | The small cat | The/DT small/JJ cat/NN | | NP chunking | Tagged tokens | [NP The small cat] | | Chinking | A broad candidate chunk | Remove verbs/prepositions, keep noun-phrase segments | | Full parsing | Sentence + grammar | A nested tree with sentence, noun phrase and verb phrase nodes | **Rule-based chunking** uses tag patterns. Statistical or learned chunkers infer boundaries from labeled examples. A useful NP rule is an optional determiner, followed by zero or more adjectives, followed by a noun. In NLTK-style notation: `NP: {
?*}`. Here ? means zero or one occurrence; \* means zero or more; + means one or more when used. The braces mean “form a chunk around the matched sequence”. **Question. Apply the NP rule optional DT, zero or more JJ, then one NN to “The/DT quick/JJ brown/JJ fox/NN jumps/VBZ over/IN the/DT lazy/JJ dog/NN”.** #### Given Tagged input is supplied. Rule: DT? JJ\* NN. #### To find All noun-phrase chunks. #### Formula and why it applies `NP → DT? JJ* NN` The rule groups a noun with its preceding determiner and adjectives; a verb or preposition does not match this NP pattern. #### Solution ##### Step 1. Start at The The matches optional DT; quick and brown both match JJ\*; fox matches NN. ##### Step 2. Form the first NP **[NP The quick brown fox]** ##### Step 3. Skip nonmatching tokens jumps/VBZ and over/IN do not satisfy the NP pattern. ##### Step 4. Match the second NP the is DT, lazy is JJ, dog is NN: **[NP the lazy dog]**. **Final answer.** [NP The quick brown fox] jumps over [NP the lazy dog]. **Question. Using NP → DT? JJ\* NN, identify the chunks in “bright/JJ stars/NN shine/VBP and/CC a/DT moon/NN glows/VBZ”. Use the supplied tags exactly.** #### Given Input tags label stars as NN for this exercise. Rule requires optional DT, zero or more JJ, then NN. #### To find The matched NP spans. #### Formula and why it applies `NP → DT? JJ* NN` The optional determiner may be absent and the adjective repetition may be empty. #### Solution ##### Step 1. Match bright stars No determiner is present, which DT? permits. bright matches JJ; stars matches NN. ##### Step 2. Inspect shine and and VBP and CC fall outside the NP rule. ##### Step 3. Match a moon a matches DT; no adjective is present; moon matches NN. **Final answer.** Chunks are [NP bright stars] and [NP a moon]. With ordinary Penn tagging, stars is usually NNS; broaden the noun pattern to include NNS when that tag is used. **Question. Begin with one candidate chunk containing “The/DT cat/NN sat/VBD on/IN the/DT mat/NN”. Chink out verbs and prepositions. Show the remaining chunks.** #### Given Initial candidate: [The cat sat on the mat]. Remove tags VBD and IN. #### To find The chunk segments after exclusion. #### Formula and why it applies Chinking removes matched tokens from the existing chunk. Removing the middle verb/preposition region divides the original candidate into two remaining contiguous spans. #### Solution ##### Step 1. Locate the tokens to remove sat is VBD and on is IN. ##### Step 2. Remove those tokens from chunk membership The remaining token groups are “The cat” and “the mat”. sat and on still exist in the sentence; they are simply outside these chunks. ##### Step 3. Write the final chunked sentence **[The cat]** sat on **[the mat]**. **Final answer.** Two chunks remain: The cat; the mat. **Yaad rakho:** Chunking = andar group karo; chinking = unwanted tokens ko group se bahar karo. ### NLP M2 — Chunking and Chinking | Reviewed rules and worked examples id : nlp-m2-11-chunking-chinking title : NLP M2 — Chunking and Chinking | Reviewed rules and worked examples title\_confirmed : true topic\_title : Chunking and chinking from POS-tagged words mode : m2 kind : focused status : available duration : 8:22 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Correct POS labels including JJ = adjective, full NP rule application and chinking boundaries. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · CFG: terminals, nonterminals, production rules and parse trees Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/m2-chunks-grammar--cfg-foundation/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/m2-chunks-grammar/cfg-foundation Unit 4, pages 14–19, 74–97 **A context-free grammar (CFG)** specifies a language using production rules whose left side is a single nonterminal. Formally, G=(N,Σ,R,S): N is the set of nonterminals, Σ the terminal vocabulary, R the production rules, and S the start symbol. A rule can expand its nonterminal regardless of the neighboring symbols; that is the meaning of “context-free”. | Concept | Plain explanation | Example | | --- | --- | --- | | Terminal | A word at a leaf; it is not expanded further | the, girl, read | | Nonterminal | A grammatical category that can be expanded | S, NP, VP, Det, N, V | | Production | A permitted replacement | S → NP VP | | Start symbol | Category for a complete sentence | S | | Lexicon | Word-to-category rules | N → girl | paper | | Derivation | Sequence of rule applications | S ⇒ NP VP ⇒ Det N VP ⇒ … | | Parse tree | Hierarchical picture of those expansions | Root S, internal phrases, word leaves | $$ S\to NP\ VP,\quad NP\to Det\ N,\quad VP\to V\ NP $$ Read S→NP VP as “a sentence can contain a noun phrase followed by a verb phrase”. It does not mean subtraction or multiplication. A vertical bar in a grammar rule means alternatives: Det→the | a allows either the or a. This is different from the probability bar meaning “given”. S: the whole sentence → NP: the girl + VP: read the paper → NP→Det N; VP→V NP → Leaves in order: the / girl / read / the / paper **A constituent** is a group of words that acts as a grammatical unit, such as “the girl”. **Syntactic parsing** assigns a grammatical structure to an input sentence using a grammar. It can support grammar checking, information extraction, translation and later semantic analysis. Failure under one restricted grammar means that grammar could not derive the sentence; it does not necessarily prove the sentence is invalid English. **A treebank** is a corpus annotated with parse trees. It supplies training/evaluation structures for parsers and lets linguists study recurring syntactic patterns. A tagged corpus gives word tags; a treebank additionally records relationships among phrases. | Representation | Example information retained | | --- | --- | | Bag of words | Words occur, but word order/grammar is lost | | POS tags | Each word's local grammatical category | | Chunks | Short phrase boundaries | | Parse tree | Nested constituent structure of the whole sentence | **Yaad rakho:** Grammar rule ek expansion instruction hai: S ko NP aur VP mein kholo. ### NLP M2 — CFG and top-down parsing | Reviewed derivations and trees id : nlp-m2-12-cfg-top-down-parsing title : NLP M2 — CFG and top-down parsing | Reviewed derivations and trees title\_confirmed : true topic\_title : CFG, ambiguity and top-down parsing mode : m2 kind : focused status : available duration : 8:54 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Complete grammars, derivations and valid parse trees; corrected the acceptance condition. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Ambiguity and grammar coverage Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/m2-chunks-grammar--structural-ambiguity/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/m2-chunks-grammar/structural-ambiguity Unit 4, pages 14–19, 74–97 **Structural ambiguity** occurs when a sentence has more than one valid parse tree. In “I saw the man with a telescope”, the prepositional phrase “with a telescope” can attach to saw (the instrument I used) or to the man (the man possessing the telescope). The words and their order stay the same, but the grouping changes. | Reading | Attachment | Meaning | | --- | --- | --- | | I [saw the man] [with a telescope] | PP modifies VP | I used a telescope to see the man. | | I saw [the man with a telescope] | PP modifies NP | The man had a telescope. | A parser may need context, probabilities or semantic constraints to choose between trees. A plain CFG describes permitted structures but does not itself assign a probability to each tree. Do not invent rule probabilities for an ordinary CFG question. Lexical ambiguity also matters: book can be a noun or verb. In “Book the flight”, the imperative analysis begins S→VP, followed by VP→V NP. A parser that initially chooses book as a noun may reach a dead end and must try another permitted analysis. **Yaad rakho:** Same words, alag grouping → alag meaning. ### NLP M2 — CFG and top-down parsing | Reviewed derivations and trees id : nlp-m2-12-cfg-top-down-parsing title : NLP M2 — CFG and top-down parsing | Reviewed derivations and trees title\_confirmed : true topic\_title : CFG, ambiguity and top-down parsing mode : m2 kind : focused status : available duration : 8:54 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Complete grammars, derivations and valid parse trees; corrected the acceptance condition. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Recursive-descent top-down parsing Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/m2-parsing--top-down-theory/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/m2-parsing/top-down-theory Unit 4, pages 98–116 **Top-down parsing** starts at the grammar's start symbol S and expands nonterminals until the leaves match the input words in order. **Recursive descent** commonly implements each nonterminal as a procedure that tries its productions. It builds from the root toward the leaves. Start with S → Choose an allowed S production → Expand its next nonterminal → Match the resulting terminal to the input → Continue, or backtrack if the choice fails **Algorithm.** Start with S as the current goal. Choose a production for the leftmost unexpanded nonterminal. Replace that nonterminal by the production's right side. If a terminal appears, compare it with the next input token. If it matches, consume the token; if it does not, reject that branch and try another production. Accept only when the complete input is consumed and all grammar goals are satisfied. A **leftmost derivation** always expands the leftmost remaining nonterminal first. Top-down parsing can make guesses that the actual words cannot support, so it may need backtracking. A naive recursive-descent parser can loop on left-recursive rules such as NP→NP PP unless the grammar or algorithm is changed. Predictive LL(1) parsing chooses productions using one lookahead token when the grammar permits; not every English CFG is LL(1). | Approach | Begins with | Direction | Common difficulty | | --- | --- | --- | --- | | Top-down | Start symbol S | Root to word leaves | Predictions that do not match input; left recursion | | Bottom-up | Actual input tokens | Leaves to root S | Reductions that cannot contribute to a complete sentence | **Yaad rakho:** Top-down: sentence ka structure pehle, words tak neeche aao. ### NLP M2 — CFG and top-down parsing | Reviewed derivations and trees id : nlp-m2-12-cfg-top-down-parsing title : NLP M2 — CFG and top-down parsing | Reviewed derivations and trees title\_confirmed : true topic\_title : CFG, ambiguity and top-down parsing mode : m2 kind : focused status : available duration : 8:54 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Complete grammars, derivations and valid parse trees; corrected the acceptance condition. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Three full top-down derivations with parse trees Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/m2-parsing--top-down-examples/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/m2-parsing/top-down-examples Unit 4, pages 98–116 **Question. Using S→NP VP, NP→Det N, VP→V NP, Det→the, N→girl|paper and V→read, give a leftmost top-down derivation and parse tree for “the girl read the paper”.** #### Given | Rule type | Rules | | --- | --- | | Structure | S → NP VP; NP → Det N; VP → V NP | | Lexicon | Det → the; N → girl | paper; V → read | #### To find A successful derivation and tree whose leaves equal the input. #### Formula and why it applies At every stage expand the leftmost nonterminal using a supplied production. A top-down derivation begins at S; the input determines which terminal alternatives can succeed. #### Solution ##### Step 1. Begin at the sentence symbol S ##### Step 2. Expand the sentence S ⇒ NP VP ##### Step 3. Expand its first noun phrase ⇒ Det N VP ##### Step 4. Match the first determiner ⇒ the N VP ##### Step 5. Match the subject noun ⇒ the girl VP ##### Step 6. Expand the verb phrase ⇒ the girl V NP ##### Step 7. Match the verb ⇒ the girl read NP ##### Step 8. Expand the object noun phrase ⇒ the girl read Det N ##### Step 9. Match its determiner ⇒ the girl read the N ##### Step 10. Match the last noun ⇒ the girl read the paper ##### Step 11. Draw the tree Parse tree: the girl read the paperSNPDettheNgirlVPVreadNPDettheNpaper Parse tree: the girl read the paper **Final answer.** The grammar derives the complete sentence. The subject NP is “the girl”; the object NP is “the paper”. **Question. Use S→NP VP, NP→Det N|N, VP→V NP, Det→A, N→restaurant|Dosas and V→serves to parse “A restaurant serves Dosas” by a leftmost top-down derivation.** #### Given | Rule type | Rules | | --- | --- | | Structure | S → NP VP; NP → Det N | N; VP → V NP | | Lexicon | Det → A; N → restaurant | Dosas; V → serves | #### To find A derivation and parse tree. #### Formula and why it applies Expand the leftmost nonterminal; choose NP→Det N for the subject and NP→N for the object. The object has no determiner, so forcing NP→Det N there would fail. #### Solution ##### Step 1. Start and expand S S ⇒ NP VP ##### Step 2. Choose the subject NP ⇒ Det N VP ##### Step 3. Match A ⇒ A N VP ##### Step 4. Match restaurant ⇒ A restaurant VP ##### Step 5. Expand the VP ⇒ A restaurant V NP ##### Step 6. Match serves ⇒ A restaurant serves NP ##### Step 7. Choose the object NP rule ⇒ A restaurant serves N ##### Step 8. Match Dosas ⇒ A restaurant serves Dosas ##### Step 9. Draw the successful tree Parse tree: A restaurant serves DosasSNPDetANrestaurantVPVservesNPNDosas Parse tree: A restaurant serves Dosas **Final answer.** The parse succeeds with subject NP→Det N and object NP→N. **Question. Using S→NP VP|VP, NP→Det N, VP→V NP, Det→that, N→book|flight and V→book, derive the imperative “book that flight” and explain the role of book.** #### Given | Rule type | Rules | | --- | --- | | Structure | S → NP VP | VP; NP → Det N; VP → V NP | | Lexicon | Det → that; N → book | flight; V → book | #### To find The successful derivation, tree and POS of book. #### Formula and why it applies Choose a production for S that can derive the actual input; an imperative may consist of VP. The grammar allows book as both noun and verb, but this imperative needs the verb reading. #### Solution ##### Step 1. Choose the imperative sentence rule S ⇒ VP ##### Step 2. Expand the verb phrase ⇒ V NP ##### Step 3. Choose the verb book ⇒ book NP ##### Step 4. Expand the object ⇒ book Det N ##### Step 5. Match that ⇒ book that N ##### Step 6. Match flight ⇒ book that flight ##### Step 7. Draw the tree Imperative parse: book that flightSVPVbookNPDetthatNflight Imperative parse: book that flight **Final answer.** book is V; that flight is NP; S→VP derives the imperative. ### NLP M2 — CFG and top-down parsing | Reviewed derivations and trees id : nlp-m2-12-cfg-top-down-parsing title : NLP M2 — CFG and top-down parsing | Reviewed derivations and trees title\_confirmed : true topic\_title : CFG, ambiguity and top-down parsing mode : m2 kind : focused status : available duration : 8:54 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Complete grammars, derivations and valid parse trees; corrected the acceptance condition. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Shift-reduce bottom-up parsing: stack, input and actions Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/m2-parsing--shift-reduce-theory/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/m2-parsing/shift-reduce-theory Unit 4, pages 98–116 **Bottom-up parsing** starts from the input tokens and combines them into larger constituents until the start symbol is reached. A **shift-reduce parser** maintains a stack of recognized material and a list of remaining input tokens. | Action | Exact operation | Example | | --- | --- | --- | | Shift | Move the next input token onto the stack | Stack: the; remaining input: dog runs | | Reduce | Replace a suffix of the stack matching a rule's RHS with its LHS | Det N becomes NP using NP→Det N | | Accept | Stack is S and input is exhausted | Stack: S; input: empty | | Fail/backtrack | Chosen reductions cannot complete a parse | A noun reading of book blocks an imperative parse | Reduction uses a production in reverse: the grammar NP→Det N expands an NP top-down, but reduces Det N into NP bottom-up. The stack's **rightmost item is its top** in the traces below. Each row shows the state **after** the listed action, avoiding an off-by-one mismatch between actions and stack contents. A lexical reduction such as N→dog still counts as a reduction. Do not skip the sequence from word to category to phrase when the question asks for a complete trace. Backtracking means returning to an earlier choice and trying another applicable rule, not changing the original input. **Yaad rakho:** Shift = input se stack. Reduce = chhote pieces se bada grammatical unit. ### M2 13 — Shift-reduce parsing and backtracking id : nlp-m2-13-bottom-up-shift-reduce title : M2 13 — Shift-reduce parsing and backtracking title\_confirmed : false topic\_title : Shift-reduce parsing and backtracking mode : m2 kind : focused status : queued duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Three full shift-reduce traces Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/m2-parsing--shift-reduce-examples/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/m2-parsing/shift-reduce-examples Unit 4, pages 98–116 **Question. Show a complete shift-reduce parse of “the big dog sees the cat” using S→NP VP, NP→Det N|Det Adj N, VP→V NP, Det→the, Adj→big, N→dog|cat and V→sees.** #### Given Grammar is as specified. Stack starts empty; input is the big dog sees the cat. Top of stack is on the right. #### To find The stack, remaining input and action at every step. #### Formula and why it applies Shift tokens; reduce matched right-hand sides to their left-hand nonterminals. The first noun phrase includes an adjective and must use NP→Det Adj N. #### Solution ##### Step 1. Trace each action | Step / action | Stack after action | Remaining input | | --- | --- | --- | | 0. Start | ∅ | the big dog sees the cat | | 1. Shift the | the | big dog sees the cat | | 2. Reduce Det→the | Det | big dog sees the cat | | 3. Shift big | Det big | dog sees the cat | | 4. Reduce Adj→big | Det Adj | dog sees the cat | | 5. Shift dog | Det Adj dog | sees the cat | | 6. Reduce N→dog | Det Adj N | sees the cat | | 7. Reduce NP→Det Adj N | NP | sees the cat | | 8. Shift sees | NP sees | the cat | | 9. Reduce V→sees | NP V | the cat | | 10. Shift the | NP V the | cat | | 11. Reduce Det→the | NP V Det | cat | | 12. Shift cat | NP V Det cat | ∅ | | 13. Reduce N→cat | NP V Det N | ∅ | | 14. Reduce NP→Det N | NP V NP | ∅ | | 15. Reduce VP→V NP | NP VP | ∅ | | 16. Reduce S→NP VP | S | ∅ | ##### Step 2. Check acceptance Input is empty and stack is exactly S; therefore accept. ##### Step 3. Draw the resulting structure Parse tree: the big dog sees the catSNPDettheAdjbigNdogVPVseesNPDettheNcat Parse tree: the big dog sees the cat **Final answer.** Accepted. The parser builds both noun phrases, then VP, then S. **Question. Use S→NP VP, NP→Det N, VP→V NP, Det→the|this, N→man|book and V→read to give a shift-reduce trace for “the man read this book”.** #### Given The input has five tokens. Stack is empty initially. The lexical verb read is supplied by V→read. #### To find A complete bottom-up trace. #### Formula and why it applies Reduce only when the top stack sequence matches a supplied production's right side. Both noun phrases follow Det N, and the verb phrase follows V NP. #### Solution ##### Step 1. Trace all operations | Step / action | Stack after action | Remaining input | | --- | --- | --- | | 0. Start | ∅ | the man read this book | | 1. Shift the | the | man read this book | | 2. Reduce Det→the | Det | man read this book | | 3. Shift man | Det man | read this book | | 4. Reduce N→man | Det N | read this book | | 5. Reduce NP→Det N | NP | read this book | | 6. Shift read | NP read | this book | | 7. Reduce V→read | NP V | this book | | 8. Shift this | NP V this | book | | 9. Reduce Det→this | NP V Det | book | | 10. Shift book | NP V Det book | ∅ | | 11. Reduce N→book | NP V Det N | ∅ | | 12. Reduce NP→Det N | NP V NP | ∅ | | 13. Reduce VP→V NP | NP VP | ∅ | | 14. Reduce S→NP VP | S | ∅ | ##### Step 2. Verify the final state The complete sentence has reduced to S with no input left. **Final answer.** Accepted. Subject: the man. Predicate: read this book. Object: this book. **Question. Parse “book the flight” bottom-up using S→NP VP|VP, NP→Det N, VP→V NP, Det→the, N→book|flight, V→book. Explain why choosing the noun reading first can fail and show the successful alternative.** #### Given book has two lexical categories N and V. The input is an imperative. Stack starts empty. #### To find The failed choice and a complete successful trace with backtracking. #### Formula and why it applies Try lexical reductions allowed by the grammar; on a dead end, return to the ambiguous choice and try another. Grammar ambiguity requires a search choice. The noun reading cannot supply the required imperative verb. #### Solution ##### Step 1. Show the noun-first dead end Shift book and reduce it to N. Continue shifting/reducing the flight to Det N. Reducing Det N→NP leaves stack N NP and empty input. No supplied rule reduces N NP into S, so this branch fails. ##### Step 2. Return to the lexical choice Restore the state just after shifting book, then choose V→book instead of N→book. ##### Step 3. Trace the successful branch | Step / action | Stack after action | Remaining input | | --- | --- | --- | | 0. Restored after shifting book | book | the flight | | 1. Reduce V→book | V | the flight | | 2. Shift the | V the | flight | | 3. Reduce Det→the | V Det | flight | | 4. Shift flight | V Det flight | ∅ | | 5. Reduce N→flight | V Det N | ∅ | | 6. Reduce NP→Det N | V NP | ∅ | | 7. Reduce VP→V NP | VP | ∅ | | 8. Reduce S→VP | S | ∅ | ##### Step 4. Accept the alternative The verb reading produces S and consumes all tokens. **Final answer.** The noun-first branch fails; backtracking to book/V yields the successful imperative parse. **Yaad rakho:** Wrong branch fail hua toh sentence wrong nahi; next legal rule try karo. ### M2 13 — Shift-reduce parsing and backtracking id : nlp-m2-13-bottom-up-shift-reduce title : M2 13 — Shift-reduce parsing and backtracking title\_confirmed : false topic\_title : Shift-reduce parsing and backtracking mode : m2 kind : focused status : queued duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Entity spans, categories and context Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/m2-ner--ner-definition/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/m2-ner/ner-definition Unit 4, pages 117–128 **Named Entity Recognition (NER)** identifies text spans referring to entities and assigns their types, such as PERSON, ORGANIZATION and LOCATION. Many practical schemas also include DATE, TIME, MONEY or domain-specific types. The exact allowed labels depend on the application's annotation scheme. NER has two parts: **boundary detection** finds where the entity starts and ends; **classification** assigns the appropriate type. A multiword name should normally be one entity span: “New York” is a location span, not two unrelated location decisions. Text: Mary joined Google in New York on Monday. → Mary → PERSON → Google → ORGANIZATION → New York → LOCATION → Monday → DATE Not every noun is a named entity. In “a developer works at Google”, developer is an ordinary job description while Google is a named organization. A specialized HR system may define a JOB\_TITLE category, but it must state that schema explicitly. NER identifies mentions; entity linking is the separate task of connecting a mention to a unique database identity. | Task | Question it answers | Example | | --- | --- | --- | | POS | What grammatical role does this token have? | Google/NNP | | Chunking | Which adjacent words form a phrase? | [NP a skilled developer] | | NER | Which span names an entity, and of what type? | [Google] ORGANIZATION | | Entity linking | Which real entity does this mention denote? | Connect Paris to the intended city or person record | **Applications.** NER structures search results and documents, extracts organizations/people from news, routes support tickets by product or organization, supports recommendation, and extracts specialized entities from healthcare or legal text. Context is essential: “Apple released a laptop” names an organization, while “she ate an apple” describes fruit. **Yaad rakho:** NER mein do kaam: naam kahan se kahan tak, aur kis category ka. ### M2 14 — Named entities, boundaries and recognition approaches id : nlp-m2-14-named-entity-recognition title : M2 14 — Named entities, boundaries and recognition approaches title\_confirmed : false topic\_title : Named entities, boundaries and recognition approaches mode : m2 kind : focused status : queued duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Dictionary, rules, machine learning and deep learning Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/m2-ner--ner-approaches/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/m2-ner/ner-approaches Unit 4, pages 117–128 | Approach | How it works | Example | Main limitation | | --- | --- | --- | --- | | Dictionary / gazetteer | Match text against lists of known names | A city list includes Mumbai and Pune | New names, spelling variants and ambiguous names are missed | | Rule-based | Use word shapes, regular expressions and context | A title such as Dr. followed by a name-like span suggests PERSON | Manual maintenance; exceptions and domain/language dependence | | Machine learning | Learn a labeling rule from annotated examples using features | Capitalization, suffixes, POS and surrounding words help a classifier/CRF | Requires labeled data and useful features | | Deep learning | Learn contextual word/character representations and predict labels | A contextual sequence encoder labels multiword entity spans | Training/domain dependence and resource demands | A **pattern-based rule** examines the form of text, such as an email pattern or a date format. A **context-based rule** examines neighboring words: “Dr.” can support a following PERSON span, while “headquartered in” can support a following LOCATION span. Capitalization alone is insufficient because sentence-initial common nouns are also capitalized. A dictionary may find “May” as a person's name, but “in May” may refer to a month. A learned model uses surrounding evidence to disambiguate. Hybrid systems can combine dictionaries, rules and learned models; these categories are practical tools rather than mutually exclusive restrictions. **Preprocessing for NER.** Tokenize while preserving token offsets and useful capitalization. Blindly removing capital letters or stopwords before recognition can damage names such as “Bank of India”. NER output should preserve the original span so the reader can locate it in the input. **Exam-ready 3-mark answer.** NER locates named-entity spans and classifies them into categories such as person, organization and location. A dictionary-based approach matches known name lists; a rule-based approach uses patterns or context; machine-learning and deep-learning approaches learn entity decisions from labeled data and representations. For example, in “Mary works at Google in Mumbai”, Mary is PERSON, Google is ORGANIZATION and Mumbai is LOCATION. ### M2 14 — Named entities, boundaries and recognition approaches id : nlp-m2-14-named-entity-recognition title : M2 14 — Named entities, boundaries and recognition approaches title\_confirmed : false topic\_title : Named entities, boundaries and recognition approaches mode : m2 kind : focused status : queued duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Three complete entity-recognition examples Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/m2-ner--ner-worked-examples/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/m2-ner/ner-worked-examples Unit 4, pages 117–128 **Question. Identify PERSON, ORGANIZATION and LOCATION entities in “Asha Rao joined Infosys in Bengaluru.”** #### Given Sentence: Asha Rao joined Infosys in Bengaluru. Allowed entity types: PERSON, ORGANIZATION, LOCATION. #### To find Complete entity spans and labels. #### Formula and why it applies Find the span first, then assign its type from context. A multiword personal name is a single entity; the verb and preposition remain outside entities. #### Solution ##### Step 1. Identify the person span Asha Rao names the person who joined, so take both words together. ##### Step 2. Identify the organization Infosys is the organization joined. ##### Step 3. Identify the location Bengaluru names the city. ##### Step 4. Write the output | Span | Label | | --- | --- | | Asha Rao | PERSON | | Infosys | ORGANIZATION | | Bengaluru | LOCATION | **Final answer.** [Asha Rao]PERSON joined [Infosys]ORGANIZATION in [Bengaluru]LOCATION. **Question. Using PERSON, ORGANIZATION, LOCATION and DATE, identify entities in “Dr. Meera Shah visited New Delhi on 5 October 2026.” Treat professional titles as outside the PERSON span for this exercise.** #### Given Allowed types and title-boundary convention are specified. Tokens include a multiword name, city and date. #### To find Entity boundaries and types. #### Formula and why it applies Boundary rules come from the annotation schema; use the stated title convention. Dr. supplies context for a person but is excluded from that span here. #### Solution ##### Step 1. Locate the person Meera Shah is the PERSON span; Dr. is outside by instruction. ##### Step 2. Locate the location New Delhi is one LOCATION span. ##### Step 3. Locate the date 5 October 2026 is one DATE span. ##### Step 4. Check organization mentions No organization is named in this sentence. **Final answer.** Meera Shah→PERSON; New Delhi→LOCATION; 5 October 2026→DATE. **Question. Explain the entity label of Apple in “Apple opened an office in Mumbai” and apple in “Ravi ate an apple”. Use PERSON, ORGANIZATION and LOCATION only.** #### Given Two contexts contain the same surface word apart from capitalization. The schema contains no FOOD category. #### To find Contextual labels and the ordinary noun case. #### Formula and why it applies NER labels a mention in context, not every occurrence of a dictionary string identically. A company can open an office; the object eaten in the second sentence is a fruit. #### Solution ##### Step 1. Analyze the first sentence Apple refers to the company, so label it ORGANIZATION. Mumbai is LOCATION. ##### Step 2. Analyze the second sentence Ravi is PERSON. apple is an ordinary fruit noun and receives no entity label under this schema. ##### Step 3. State the dictionary limitation Blindly labeling every apple as an organization would produce a false positive. **Final answer.** First: Apple→ORGANIZATION, Mumbai→LOCATION. Second: Ravi→PERSON; apple is outside the entity types. ### M2 14 — Named entities, boundaries and recognition approaches id : nlp-m2-14-named-entity-recognition title : M2 14 — Named entities, boundaries and recognition approaches title\_confirmed : false topic\_title : Named entities, boundaries and recognition approaches mode : m2 kind : focused status : queued duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Prepare a grammar for CYK / CKY Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/u4-forward-cky--cnf/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/u4-forward-cky/cnf Unit 4 PDF p105 names CYK. The supplied papers ask top-down / bottom-up CFG trees, not CKY charts. These traces explain the taught topic. **Faculty topic; not an observed numerical PYQ.** Unit 4 PDF p105 names CYK. The supplied papers ask top-down / bottom-up CFG trees, not CKY charts. These traces explain the taught topic. **CYK and CKY are two names for the same dynamic-programming parsing family.** A chart stores which grammatical categories can generate each contiguous span of the sentence. Small spans are solved first and reused to build larger spans. Basic CKY uses a context-free grammar in Chomsky normal form (CNF): a production has either two nonterminals on its right, A→B C, or one terminal, A→word. Empty-string handling is a separate special case; the examples here have no empty production. | Rule | Basic CKY status | Reason | | --- | --- | --- | | S → NP VP | Allowed | Two nonterminals | | N → cat | Allowed | One terminal | | NP → Det Adj N | Convert first | Three nonterminals | | NP → N | Eliminate or handle unary closure | One nonterminal, not a lexical terminal | | VP → eats NP | Convert first | Mixed terminal and nonterminal | **Question. Convert NP→Det Adj N, NP→N, N→cat and VP→eats NP into basic CNF-compatible productions. Preserve their possible word yields. [Authored worked example]** ### Answer #### Given The symbols NP,Det,Adj,N,VP are nonterminals; cat and eats are terminals. Other needed lexical rules remain as supplied. #### To find Equivalent binary or terminal productions for these cases. #### Formula and why it applies Introduce helper categories for a long right-hand side or a terminal mixed with nonterminals. Eliminate a unary nonterminal rule by inheriting the reachable non-unit productions. #### Solution ##### Step 1. Binarize the three-part NP Introduce X: NP→Det X and X→Adj N. X is a helper, not a word; the terminal order remains Det, Adj, N. ##### Step 2. Remove the unit production NP→N and N→cat allow NP to derive cat. Add NP→cat, retain N→cat where needed, and remove NP→N. With more rules reachable through unit chains, inherit every corresponding non-unit rule, not only the first one. ##### Step 3. Replace the mixed terminal Introduce Vlex→eats and replace VP→eats NP by VP→Vlex NP. ##### Step 4. Read the converted forms NP→Det X | cat; X→Adj N; N→cat; VP→Vlex NP; Vlex→eats. All displayed productions are binary nonterminal or single-terminal rules. **Final answer.** The converted rules preserve the original word sequences while satisfying the displayed basic CKY forms. Helper nodes change tree shape; they can be removed when presenting the original grammar’s tree. General CNF conversion may also require eliminating nullable productions and useless symbols while preserving the intended language. Do not claim that simply splitting long rules handles every arbitrary CFG. An Earley parser instead supports general CFG rules using prediction, scanning and completion; it combines top-down expectations with bottom-up evidence. LR and operator-precedence parsers are other methods named in the faculty list, with grammar restrictions of their own. This worked lesson was added after the video plan. No matching video has been generated or reviewed. --- ## TEE · CYK / CKY: complete charts, backpointers and ambiguity Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/u4-forward-cky--cky/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/u4-forward-cky/cky Unit 4 PDF p105 names CYK. The supplied papers ask top-down / bottom-up CFG trees, not CKY charts. These traces explain the taught topic. **Faculty topic; not an observed numerical PYQ.** Unit 4 PDF p105 names CYK. The supplied papers ask top-down / bottom-up CFG trees, not CKY charts. These traces explain the taught topic. Number the words 1 to n. Let **T[i,j]** contain all categories that derive the span from word i through word j, inclusive. Start with one-word cells from lexical rules. For every longer span try **every split k** between i and j: if B belongs to T[i,k], C belongs to T[k+1,j], and A→B C is a rule, put A in T[i,j]. Store the rule and split as backpointers when a tree is required. $$ A\in T[i,j]\quad\text{if}\quad A\to BC,\ B\in T[i,k],\ C\in T[k+1,j],\ i\le kj); ∅ is an evaluated but empty valid span. Read the chart by increasing span length, not left-to-right across a row. ##### Step 7. Follow the stored backpointers S[1,5]→NP[1,2] VP[3,5]; NP[1,2]→Det[1,1] N[2,2]; VP[3,5]→V[3,3] NP[4,5]; NP[4,5]→Det[4,4] N[5,5]. ``` S ├── NP │ ├── Det: the │ └── N: cat └── VP ├── V: eats └── NP ├── Det: a └── N: fish ``` **Final answer.** Accepted because S∈T[1,5]. The reconstructed tree’s leaves are exactly the five input words in order. **Question. A grammar has S→A A, A→A A and A→a. Use CKY for “a a a” and explain how two parse trees are preserved. [Authored worked example]** ### Answer #### Given Three identical terminal tokens. S is the start symbol. No unit or empty rules are present. #### To find Chart categories and both successful split choices for the root. #### Formula and why it applies A chart cell is a set of possible categories. For each category keep all successful backpointers when the task requires every parse. #### Solution ##### Step 1. Initialize the diagonal Each of T[1,1], T[2,2], T[3,3] is {A}, from A→a. ##### Step 2. Fill two-word spans T[1,2] and T[2,3] each contain {A,S}: A+A can use A→A A or S→A A. ##### Step 3. Try both root splits k=1: A[1,1] + A[2,3] permits S. k=2: A[1,2] + A[3,3] also permits S. The complete cell is {A,S}, but S has two distinct backpointer alternatives. ##### Step 4. Draw both bracketed trees [S [A a] [A [A a] [A a]]] and [S [A [A a] [A a]] [A a]]. Both yield a a a; the constituent grouping differs. **Final answer.** Accepted, with two S-rooted parses. A category appearing once in a set does not mean there is only one derivation. **Question. With S→NP VP, NP→Det N, VP→V NP; Det→the|a; N→cat|fish; V→eats, test “the eats cat” using CKY. [Authored worked example]** ### Answer #### Given Tokens are the/eats/cat, with no punctuation. Use exactly the supplied grammar. #### To find Whether the grammar accepts the whole input. #### Formula and why it applies A parse exists only if the complete chart cell contains the start symbol. #### Solution ##### Step 1. Initialize the diagonal T[1,1]={Det}; T[2,2]={V}; T[3,3]={N}. ##### Step 2. Try length-two combinations Det+V matches no binary rule; V+N also matches no rule (VP requires V+NP). Both two-word cells are empty. ##### Step 3. Try the two full-span splits Each split has an empty child, so T[1,3]=∅ and S is absent. **Final answer.** Rejected by this grammar. Do not invent NP→N to force acceptance. Grammar rejection and a universal judgment about every possible English grammar are different claims. **Yaad rakho:** Diagonal par words ke tags; phir chhote spans jodo. Har split dekho. Full sentence cell mein S aaya toh parse possible. This worked lesson was added after the video plan. No matching video has been generated or reviewed. --- ## TEE · Lexeme, lemma, sense and context Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/u5-meaning--u5-words/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/u5-meaning/u5-words Unit 5 · PDF pages 1–22 **Semantics is the study of meaning in language.** Lexical semantics studies meanings of words and relationships between their meanings. Sentence semantics studies how word meanings and sentence structure combine. A **word form** is what appears in the sentence. A **lexeme** is the underlying vocabulary item whose grammatical forms belong together. A **lemma** is the dictionary form used to name it. A **sense** is one particular meaning. | Term | Example | What to understand | | --- | --- | --- | | Word forms | run, runs, ran, running | Different grammatical forms | | Lexeme | RUN | The shared underlying vocabulary item | | Lemma | run | Dictionary heading | | Sense | run a race / run a business | Different meanings need context | | Lexicon | A language's vocabulary resource | Stores lexical items and information about them | The four forms *run, ran, runs, running* belong to the same lexeme. They are not four unrelated lexemes. “I deposited cash in the bank” and “We sat by the bank of the river” contain the same written form but different senses. word: bank → context: cash, deposit → sense: financial institution **Syntax** asks how words are arranged. **Semantics** asks what the expression means. **Pragmatics** uses situation and speaker intention: “Can you open the window?” is grammatically a question about ability but is normally a request. Form = jo likha; lemma = dictionary naam; sense = iss sentence mein matlab. ### M2 15 — Lexical meaning, semantic relations and WordNet id : nlp-m2-15-lexical-meaning-wordnet title : M2 15 — Lexical meaning, semantic relations and WordNet title\_confirmed : false topic\_title : Lexical meaning, semantic relations and WordNet mode : m2 kind : focused status : queued duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Every lexical relation with examples Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/u5-meaning--u5-relations/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/u5-meaning/u5-relations Unit 5 · PDF pages 1–22 **A lexical relation connects words or senses by a meaning relationship.** State the relation and its direction before giving the example. | Relation | Exam definition | Examples and direction | | --- | --- | --- | | Homonymy | The same form has unrelated meanings. | bank: financial institution / river edge; bat: animal / sports implement | | Polysemy | One lexical item has multiple related senses. | head: body part / head of a department; paper: material / article | | Synonymy | Different words have identical or closely similar meanings in a specified sense. | sofa / couch; automobile / car | | Antonymy | Words have opposite meanings. | hot / cold; alive / dead; buy / sell | | Hyponymy | A more specific concept is a type of a more general one. | rose is a flower; car is a vehicle | | Hypernymy | A broader concept includes a more specific one. | flower is a hypernym of rose | | Meronymy | A concept is a part of another concept. | wheel is part of car; finger is part of hand | | Holonymy | A concept is the whole containing a part. | car is a holonym of wheel | ### How to distinguish the commonly confused pairs For homonymy versus polysemy, ask whether the senses share an identifiable meaning. A river bank and a financial bank are unrelated. A financial bank and a blood bank share a repository/storage idea, illustrating polysemy in the supplied notes. For hyponymy versus meronymy, use the sentence test: **“X is a kind of Y”** indicates hyponymy; **“X is part of Y”** indicates meronymy. A wheel is not a type of car. | Expression | Correct relation | Reason | | --- | --- | --- | | sparrow → bird | Hyponymy | A sparrow is a kind of bird | | bird → sparrow | Hypernymy | Bird is the broader category | | keyboard → computer | Meronymy | Keyboard is a component | | computer → keyboard | Holonymy | Computer is the containing whole | | mouth of a person / mouth of a river | Polysemy | Both senses share the idea of an opening; the river use is a related extension | Synonyms are not always interchangeable in every context or register. “Big mistake” and “large mistake” can sound different even though big and large overlap in meaning. Antonyms can be gradable (hot/cold), complementary (alive/dead) or relational converses (buy/sell). HYPO = specific type; HYPER = bigger category. MERO = part; HOLO = whole. ### M2 15 — Lexical meaning, semantic relations and WordNet id : nlp-m2-15-lexical-meaning-wordnet title : M2 15 — Lexical meaning, semantic relations and WordNet title\_confirmed : false topic\_title : Lexical meaning, semantic relations and WordNet mode : m2 kind : focused status : queued duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · WordNet: synsets, glosses and hierarchy Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/u5-meaning--u5-wordnet/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/u5-meaning/u5-wordnet Unit 5 · PDF pages 1–22 **WordNet is a lexical database that groups word senses into synonym sets called synsets and links them through semantic and lexical relations.** It is a network of meanings, not merely a list of similar spellings. A synset represents one concept. Its **lemmas** are the words that can express that concept. A **gloss** is a short definition, often accompanied by an example. An ambiguous word can occur in several synsets. vehicle (general) → car / automobile (same concept) → sports car (specific) This is a simplified teaching hierarchy, not a claim about every intermediate WordNet node. Going upward reaches hypernyms; going downward reaches hyponyms. Multiple parents can exist, so the structure need not be a single simple tree. | Operation | Meaning | Illustration | | --- | --- | --- | | Get senses of a word | Find candidate synsets | bank → financial sense, river-edge sense, … | | Get lemmas | Find synonyms within one synset | car, automobile, motorcar | | Get definition/examples | Read the gloss and uses | Choose the intended concept | | Get hypernyms | Move to broader categories | car → vehicle | | Get hyponyms | Move to narrower categories | vehicle → particular vehicle categories | | Get meronyms/holonyms | Find part/whole links | car ↔ wheel | ### WordNet versus a thesaurus A thesaurus mainly groups related expressions for vocabulary lookup. WordNet explicitly distinguishes senses and labels relations such as hypernymy, hyponymy and meronymy. This supports WSD, query expansion, lexical similarity and semantic search. ### Three uses explained 1. **Search expansion:** a query for “automobile” can retrieve relevant documents using “car”. Expansion should use the right sense. 2. **WSD:** definitions of the senses of “bank” provide words that can be compared with the sentence context. 3. **Similarity:** concepts near one another in an “is-a” hierarchy tend to be semantically related. Path-based measures depend on which senses were selected. WordNet mein spelling ka group nahi—meaning ka synset, aur meanings ke beech labelled links. ### M2 15 — Lexical meaning, semantic relations and WordNet id : nlp-m2-15-lexical-meaning-wordnet title : M2 15 — Lexical meaning, semantic relations and WordNet title\_confirmed : false topic\_title : Lexical meaning, semantic relations and WordNet mode : m2 kind : focused status : queued duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · What WSD solves and how approaches differ Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/u5-wsd--u5-wsd-basics/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/u5-wsd/u5-wsd-basics Unit 5 · PDF pages 23–48 **Word sense disambiguation (WSD) is the task of selecting the intended sense of an ambiguous word in its context from a set of possible senses.** The output is a sense label, not a different replacement word. Input: “He grilled the bass.” → Candidate senses: fish / low sound → Context: grilled → Output: fish sense The **target word** is the ambiguous word. The **context** is the surrounding evidence. The **sense inventory** lists candidate meanings, often using WordNet. A **feature** is a usable clue: nearby word, position, part-of-speech tag or a phrase. | Approach | What it needs | How it decides | Limitation | | --- | --- | --- | --- | | Knowledge based | Dictionary, thesaurus or semantic graph | Definition overlap, category evidence or graph scores | Short definitions and missing knowledge | | Supervised | Examples already labelled with senses | Learns context → sense mapping | Labelling cost, limited rare-sense examples | | Semi-supervised | Small labelled set plus unlabelled data | Bootstraps from reliable seed examples | Early errors can spread | | Unsupervised sense induction | Unlabelled contextual occurrences | Clusters similar contexts | Clusters are not automatically named dictionary senses | | Neural/contextual | Learned word/context representations | Predicts or compares a context representation | Needs suitable data/model and evaluation | **Bag-of-words features** record nearby words while ignoring exact positions. **Collocational features** preserve a positional phrase such as “river bank”, “bank account”, “chocolate bar” or “bar exam”. ### Why WSD matters Machine translation needs the correct target-language meaning; information retrieval should distinguish river-bank travel from banking services; text-to-speech may need different pronunciations for “bass”; information extraction should avoid connecting entities through the wrong relation. WSD: same spelling ke options mein context dekh ke sahi meaning select. ### M2 16 WSD and Lesk Scoring id : nlp-m2-16-lesk-wsd title : M2 16 WSD and Lesk Scoring title\_confirmed : true topic\_title : Word-sense disambiguation and Lesk overlap scoring mode : m2 kind : focused status : available duration : 10:17 reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — Word-sense disambiguation | Reviewed Lesk and Bayes numericals id : overview-wsd title : NLP M2 — Word-sense disambiguation | Reviewed Lesk and Bayes numericals title\_confirmed : true topic\_title : WSD: Detective Math Toolkit mode : m2 kind : supplemental status : available duration : 6:11 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Lesk variants, conditional Bayes definitions, smoothing and both worked posteriors; corrected spoken claims. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids : - nlp-m2-16-lesk-wsd - nlp-m2-17-walker-graph-wsd - nlp-m2-18-naive-bayes-wsd These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Lesk algorithm, from words to overlap scores Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/u5-wsd--u5-lesk/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/u5-wsd/u5-lesk Unit 5 · PDF pages 23–48 **Lesk is a knowledge-based WSD method that chooses senses whose dictionary descriptions have the greatest overlap with contextual evidence.** Original Lesk compares definitions of neighbouring words' senses. A common simplified version compares each target-sense gloss directly with the sentence context. State which version you use. ### Simplified Lesk procedure 1. Remove the target word from the context; tokenize and normalize the remaining content words. 2. Obtain the gloss for each candidate sense and apply the same normalization. 3. Count shared words between the context and each gloss. 4. Select the sense with maximum overlap. If scores tie, report the tie or use a specified tie-break rule. $$ \operatorname{score}(s)=|C\cap G_s|,\qquad s^*=\arg\max_s\operatorname{score}(s) $$ $$ C $$ is the set of context words, $$ G_s $$ the set of words in sense $$ s $$ 's gloss, $$ \cap $$ means common elements, $$ |\cdot| $$ means count, and $$ \arg\max $$ means “the sense giving the largest score”. This version counts unique matching words once. **Question. Use simplified Lesk to disambiguate “bank” in “The river water rose near the bank.” Use C = {river, water, rose, near}; financial gloss = {institution, money, deposit}; river gloss = {land, river, water, edge}.** #### Given The normalized context and two gloss sets are given. Count exact shared words. #### To find The score of each sense and the selected meaning. #### Formula and why it applies Use $$ \operatorname{score}(s)=|C\cap G_s| $$ because the task specifies context–gloss overlap. #### Step 1. Compare the financial sense No word occurs in both {river, water, rose, near} and {institution, money, deposit}. Overlap = {}. #### Step 2. Count the financial score $$ \operatorname{score}(F)=0 $$ . #### Step 3. Compare the river sense Shared words are {river, water}. “Rose” and “near” are not in the gloss. #### Step 4. Count and select $$ \operatorname{score}(R)=2 $$ . Since $$ 2>0 $$ , choose the river-edge sense. **Answer.** Bank means land beside a river. **Question. Use simplified Lesk for “The dog barked loudly.” Treat barked as bark. Context C = {dog, loudly}; sound gloss = {sharp, cry, dog}; tree gloss = {outer, covering, tree}; ship gloss = {sailing, ship}.** #### Given Three glosses and normalized context {dog, loudly}. #### To find Select the sense of bark. #### Formula and why it applies $$ \operatorname{score}(s)=|C\cap G_s| $$ . Lemmatization lets barked match the target lemma bark; it does not create additional overlap words. #### Step 1. Sound sense Intersection = {dog}; score = 1. #### Step 2. Tree sense Intersection = {}; score = 0. #### Step 3. Ship sense Intersection = {}; score = 0. #### Step 4. Compare The sound sense has the unique maximum: 1 > 0. **Answer.** Bark is the dog's cry. **Question. Apply definition–definition Lesk to “pine cone”. Pine senses: P1 = {evergreen, tree, needle, leaf}; P2 = {waste, sorrow, illness}. Cone senses: C1 = {solid, shape, circular}; C2 = {fruit, evergreen, tree}.** #### Given Two candidate senses for each word. Tokens are already normalized; tree and trees would be reduced to the same form if necessary. #### To find The most compatible pair of senses. #### Formula and why it applies Use $$ |G_{P_i}\cap G_{C_j}| $$ for every pair because this is original-style gloss–gloss comparison. #### Step 1. P1 with C1 No shared word → 0. #### Step 2. P1 with C2 Shared {evergreen, tree} → 2. #### Step 3. P2 with each cone sense P2–C1 has no overlap → 0; P2–C2 has no overlap → 0. #### Step 4. Choose the best pair The four scores are 0, 2, 0, 0. Maximum 2 belongs to P1–C2. **Answer.** Pine is the evergreen tree; cone is its fruit-like reproductive structure. **Limitations:** exact overlap misses synonyms, short glosses may produce zeros, and different preprocessing choices change the score. Extended Lesk may include related synset definitions or longer phrase overlaps; do not silently substitute it for a simple question. Lesk: context aur dictionary mein common words gino; sabse bada overlap jeeta. ### M2 16 WSD and Lesk Scoring id : nlp-m2-16-lesk-wsd title : M2 16 WSD and Lesk Scoring title\_confirmed : true topic\_title : Word-sense disambiguation and Lesk overlap scoring mode : m2 kind : focused status : available duration : 10:17 reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — Word-sense disambiguation | Reviewed Lesk and Bayes numericals id : overview-wsd title : NLP M2 — Word-sense disambiguation | Reviewed Lesk and Bayes numericals title\_confirmed : true topic\_title : WSD: Detective Math Toolkit mode : m2 kind : supplemental status : available duration : 6:11 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Lesk variants, conditional Bayes definitions, smoothing and both worked posteriors; corrected spoken claims. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids : - nlp-m2-16-lesk-wsd - nlp-m2-17-walker-graph-wsd - nlp-m2-18-naive-bayes-wsd These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Thesaurus / Walker approach, with three complete examples Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/u5-wsd--u5-walker/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/u5-wsd/u5-walker Unit 5 · PDF pages 23–48 **The Walker approach uses thesaurus categories as semantic evidence.** Associate each candidate sense with a category. Count or weight context words that support that category. Choose the largest total. $$ \operatorname{score}(s)=\sum_{w\in C} I(w,\operatorname{category}(s)) $$ $$ I=1 $$ if the supplied category mapping says the context word supports the sense, otherwise $$ I=0 $$ . The summation symbol means add all those contributions. Weighted questions may replace 1 with a supplied weight. Category membership must be supplied or justified; it is not guessed from word frequency. **Question. Disambiguate bank using Walker scoring. Context words are money, interest, annum, fetch. The finance category supports money, interest and annum; the location category supports none of these words.** #### Given Two categories: Finance and Location. Each supported word contributes 1. #### To find The two totals and selected bank sense. #### Formula and why it applies Sum the category indicators for each sense. #### Step 1. Make a contribution table | Word | Finance | Location | | --- | --- | --- | | money | 1 | 0 | | interest | 1 | 0 | | annum | 1 | 0 | | fetch | 0 | 0 | #### Step 2. Add Finance $$ 1+1+1+0=3 $$ . #### Step 3. Add Location $$ 0+0+0+0=0 $$ . #### Step 4. Select 3 > 0, so select the financial institution. **Answer.** Finance, with score 3 versus 0. **Question. Use Walker scoring for “The fisherman caught bass beside the lake.” Context {fisherman,caught,lake}. The Fish category contains all three clues; the Music category contains none.** #### Given Fish and Music senses, binary category membership given. #### To find The correct bass sense. #### Formula and why it applies Score = sum of context-category matches. #### Step 1. Record Fish matches fisherman → 1; caught → 1; lake → 1. #### Step 2. Add Fish score $$ 1+1+1=3 $$ . #### Step 3. Add Music score $$ 0+0+0=0 $$ . #### Step 4. Choose Fish has the larger score. **Answer.** Bass refers to a fish. **Question. Score “The player used a bat in the match.” Context {player,match}. Sport supports both words with weights 2 and 1; Animal supports neither. Use the supplied weighted Walker scheme.** #### Given Sport weights {player:2, match:1}; Animal weights {player:0, match:0}. #### To find Weighted sense scores. #### Formula and why it applies Use the sum of supplied weights, not an unweighted count, because the question explicitly gives weights. #### Step 1. Sport contribution of player Add 2. #### Step 2. Sport contribution of match Add 1. Sport total = 2 + 1 = 3. #### Step 3. Animal total 0 + 0 = 0. #### Step 4. Compare 3 > 0 selects the sporting implement. **Answer.** Bat is the sports implement. Walker: dictionary ke exact words nahi, context kis topic/category ko support karta hai woh count. ### M2 17 — Walker category scoring and graph-based WSD id : nlp-m2-17-walker-graph-wsd title : M2 17 — Walker category scoring and graph-based WSD title\_confirmed : false topic\_title : Walker category scoring and graph-based WSD mode : m2 kind : focused status : queued duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — Word-sense disambiguation | Reviewed Lesk and Bayes numericals id : overview-wsd title : NLP M2 — Word-sense disambiguation | Reviewed Lesk and Bayes numericals title\_confirmed : true topic\_title : WSD: Detective Math Toolkit mode : m2 kind : supplemental status : available duration : 6:11 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Lesk variants, conditional Bayes definitions, smoothing and both worked posteriors; corrected spoken claims. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids : - nlp-m2-16-lesk-wsd - nlp-m2-17-walker-graph-wsd - nlp-m2-18-naive-bayes-wsd These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Random-walk WSD and supervised alternatives Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/u5-wsd--u5-graph/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/u5-wsd/u5-graph Unit 5 · PDF pages 23–48 **Graph-based WSD represents candidate senses as vertices and semantic relationships as weighted edges.** A random walk repeatedly transfers score along these edges. Senses supported by other strongly connected context senses tend to obtain higher ranks. 1. Create a vertex for every candidate sense of every relevant word. 2. Connect semantically related senses; similarity or gloss overlap can set edge weights. 3. Normalize outgoing edge weights into transition probabilities. 4. Run a graph-ranking method such as PageRank. 5. For each target word, choose its highest-ranked candidate sense. church context → bell: instrument sense ↔ ring: sound sense → mutually supported senses receive rank One common rank update is $$ r_{t+1}(j)=\frac{1-d}{N}+d\sum_i r_t(i)P(i\rightarrow j). $$ $$ r_t(i) $$ is current score of vertex $$ i $$ , $$ P(i\to j) $$ is its outgoing probability to $$ j $$ , $$ N $$ is the number of vertices, and $$ d $$ is the probability of following a graph edge. The remaining probability $$ 1-d $$ gives a uniform restart here. A personalized variant uses a nonuniform restart. The course requires the graph method concept; numerical details depend on supplied ranking rules. **Question. A two-vertex toy sense graph has transition matrix rows A:[0.2,0.8], B:[0.4,0.6]. Starting scores are A=0.5, B=0.5. Find one random-walk update with no restart.** #### Given Rows denote source vertices, columns destination vertices. d = 1. #### To find New scores at A and B. #### Formula and why it applies Use $$ r_{t+1}(j)=\sum_i r_t(i)P(i\to j) $$ : incoming contributions are added. #### Step 1. Contribution to A from A $$ 0.5\times0.2=0.1 $$ . #### Step 2. Contribution to A from B $$ 0.5\times0.4=0.2 $$ , hence $$ r_1(A)=0.1+0.2=0.3 $$ . #### Step 3. Contribution to B from A and B $$ 0.5\times0.8=0.4 $$ ; $$ 0.5\times0.6=0.3 $$ . #### Step 4. Add B and verify total $$ r_1(B)=0.4+0.3=0.7 $$ ; $$ 0.3+0.7=1 $$ . **Answer.** A=0.3, B=0.7 after one update. This is one iteration, not a converged PageRank result. ### Other approaches explained without assuming ML knowledge **A supervised classifier** learns from examples such as “cash … bank” labelled FINANCE and “river … bank” labelled RIVER. A decision list applies ranked context rules; a decision tree branches on clues; an SVM separates feature representations; a neural model learns contextual patterns. Naive Bayes, developed next, multiplies class-specific context probabilities. **Semi-supervised bootstrapping** begins with a few labelled examples or high-confidence rules, labels suitable unlabelled cases, and retrains. **Unsupervised clustering** groups usages by context similarity without gold sense labels; an additional mapping is needed to name the discovered groups. Graph WSD: meanings ko jodo; jo meaning context ke strong meanings se supported hai uska rank badhega. ### M2 17 — Walker category scoring and graph-based WSD id : nlp-m2-17-walker-graph-wsd title : M2 17 — Walker category scoring and graph-based WSD title\_confirmed : false topic\_title : Walker category scoring and graph-based WSD mode : m2 kind : focused status : queued duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — Word-sense disambiguation | Reviewed Lesk and Bayes numericals id : overview-wsd title : NLP M2 — Word-sense disambiguation | Reviewed Lesk and Bayes numericals title\_confirmed : true topic\_title : WSD: Detective Math Toolkit mode : m2 kind : supplemental status : available duration : 6:11 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Lesk variants, conditional Bayes definitions, smoothing and both worked posteriors; corrected spoken claims. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids : - nlp-m2-16-lesk-wsd - nlp-m2-17-walker-graph-wsd - nlp-m2-18-naive-bayes-wsd These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Prior, likelihood and posterior—before the formula Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/u5-naive-bayes--u5-bayes-meaning/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/u5-naive-bayes/u5-bayes-meaning Unit 5 · PDF pages 49–57 **A probability is a number from 0 to 1 measuring how likely an event is under a stated model.** A class probability of $$ 3/6=0.5 $$ means 3 of 6 training examples belong to that class. **Conditional probability** means probability inside a restricted group. $$ P(\text{loan}\mid F) $$ reads “probability of the word loan given the finance sense”. The vertical bar means “given”; it is not division. | Symbol | Name | Plain meaning | | --- | --- | --- | | $$ P(F) $$ | Prior | How common finance is before reading the new context | | $$ P(C\mid F) $$ | Likelihood | How compatible the observed context is with finance | | $$ P(F\mid C) $$ | Posterior | How likely finance is after seeing the context | | $$ P(C) $$ | Evidence | Total probability of this context under all senses | $$ P(F\mid C)=\frac{P(C\mid F)P(F)}{P(C)}. $$ Bayes' theorem reverses a conditional direction using the prior. Do not swap $$ P(F\mid C) $$ and $$ P(C\mid F) $$ : one asks about the sense, the other about the evidence. The **naive conditional independence assumption** says that, once the sense is fixed, the context-word features can be multiplied as if they were independent. It is an approximation: natural-language words are not actually independent in general. Training examples → count classes → Prior per sense → Count words within each sense → Likelihood of query clues → Multiply → compare → normalize if asked Prior = pehle kitna common; likelihood = clue kitna fit; posterior = clue dekhne ke baad meaning ki chance. ### Naive Bayes WSD Numerical id : nlp-m2-18-naive-bayes-wsd title : Naive Bayes WSD Numerical title\_confirmed : true topic\_title : Naive Bayes WSD: a complete corpus numerical mode : m2 kind : focused status : available duration : 11:35 reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — Word-sense disambiguation | Reviewed Lesk and Bayes numericals id : overview-wsd title : NLP M2 — Word-sense disambiguation | Reviewed Lesk and Bayes numericals title\_confirmed : true topic\_title : WSD: Detective Math Toolkit mode : m2 kind : supplemental status : available duration : 6:11 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Lesk variants, conditional Bayes definitions, smoothing and both worked posteriors; corrected spoken claims. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids : - nlp-m2-16-lesk-wsd - nlp-m2-17-walker-graph-wsd - nlp-m2-18-naive-bayes-wsd These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Multinomial Naive Bayes: the exact solving method Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/u5-naive-bayes--u5-bayes-steps/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/u5-naive-bayes/u5-bayes-steps Unit 5 · PDF pages 49–57 Naive Bayes selects the sense with the largest prior × likelihood product. The context evidence denominator is identical across candidate senses, so it can be omitted for comparison. $$ \operatorname{score}(s)=P(s)\prod_j P(w_j\mid s). $$ For add-one multinomial smoothing: $$ P(w\mid s)=\frac{C(w,s)+1}{N_s+V}. $$ $$ C(w,s) $$ counts occurrences of word $$ w $$ in training contexts of sense $$ s $$ ; $$ N_s $$ is the total number of word occurrences in those contexts; $$ V $$ is the shared vocabulary size. It is not the number of training sentences. Repeated query words contribute repeated factors. ### Always keep these counts separate Class priors use **number of examples**. Word likelihoods in this multinomial model use **number of word occurrences**. The vocabulary counts **unique words across all classes**. 1. List the class examples and calculate priors. 2. Write one shared vocabulary and count its size. 3. Count total word occurrences in each class. 4. Count each query word in each class. 5. Apply the given smoothing rule. 6. Multiply each prior by all its query likelihoods. 7. Compare scores; divide each score by their sum only if a normalized posterior is requested. $$ P(s\mid C)=\frac{\operatorname{score}(s)}{\sum_{s'}\operatorname{score}(s')}. $$ **Why smoothing is needed:** an unseen clue gives $$ 0/N_s=0 $$ ; one zero makes the whole product zero. Add-one smoothing redistributes probability across every vocabulary word and prevents these zero factors. Use it only when requested, taught for the task, or state the assumption. **Different model warning:** a Bernoulli Naive Bayes question counts presence/absence per document and may include absence factors. Do not use its denominator in a multinomial word-count question. Pehle sentences se prior; phir words se likelihood; dono counts mix mat kar. ### Naive Bayes WSD Numerical id : nlp-m2-18-naive-bayes-wsd title : Naive Bayes WSD Numerical title\_confirmed : true topic\_title : Naive Bayes WSD: a complete corpus numerical mode : m2 kind : focused status : available duration : 11:35 reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — Word-sense disambiguation | Reviewed Lesk and Bayes numericals id : overview-wsd title : NLP M2 — Word-sense disambiguation | Reviewed Lesk and Bayes numericals title\_confirmed : true topic\_title : WSD: Detective Math Toolkit mode : m2 kind : supplemental status : available duration : 6:11 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Lesk variants, conditional Bayes definitions, smoothing and both worked posteriors; corrected spoken claims. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids : - nlp-m2-16-lesk-wsd - nlp-m2-17-walker-graph-wsd - nlp-m2-18-naive-bayes-wsd These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Mam's bank numerical—every count and every multiplication Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/u5-naive-bayes--u5-bayes-mam/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/u5-naive-bayes/u5-bayes-mam Unit 5 · PDF pages 49–57 **Question. Classify the sense of “bank” using the training table below. The sentence is: “As the stream flowed gently past the shore, we set up camp by the bank of the river, discussing the possibility of borrowing a loan to buy a boat for future trips.” Use only the highlighted feature words stream, shore and loan, and add-one multinomial smoothing.** #### Given | Training row | Context words | Sense | | --- | --- | --- | | 1 | money, deposit, loan | Finance | | 2 | river, water, shore | Riverbank | | 3 | account, savings, investment | Finance | | 4 | boat, river, stream | Riverbank | | 5 | branch, deposit, check | Finance | | 6 | fishing, shore, river | Riverbank | Query feature set: {stream, shore, loan}. Other sentence words are not included as features in this supplied example. #### To find The most probable bank sense and normalized posteriors. #### Formula and why it applies Naive Bayes selects the sense with the largest prior × likelihood product. The context evidence denominator is identical across candidate senses, so it can be omitted for comparison. $$ \operatorname{score}(s)=P(s)\prod_j P(w_j\mid s). $$ For add-one multinomial smoothing: $$ P(w\mid s)=\frac{C(w,s)+1}{N_s+V}. $$ $$ C(w,s) $$ counts occurrences of word $$ w $$ in training contexts of sense $$ s $$ ; $$ N_s $$ is the total number of word occurrences in those contexts; $$ V $$ is the shared vocabulary size. It is not the number of training sentences. Repeated query words contribute repeated factors. #### Step 1. Calculate priors Finance appears in 3 of the 6 rows; Riverbank appears in 3 of 6. $$ P(F)=\frac36=\frac12,\qquad P(R)=\frac36=\frac12. $$ #### Step 2. Construct the vocabulary {money, deposit, loan, river, water, shore, account, savings, investment, boat, stream, fishing, branch, check}. Thus V=14. Repeated deposit or river is counted once in vocabulary. #### Step 3. Count word occurrences per class There are 3 finance rows × 3 words = 9 word occurrences. There are 3 river rows × 3 words = 9. Thus $$ N_F=N_R=9 $$ , and each smoothed denominator is $$ 9+14=23 $$ . #### Step 4. Count the three query words | Word | Finance count | Riverbank count | | --- | --- | --- | | stream | 0 | 1 | | shore | 0 | 2 | | loan | 1 | 0 | #### Step 5. Calculate finance likelihoods $$ P(\text{stream}\mid F)=\frac{0+1}{23}=\frac1{23} $$ $$ P(\text{shore}\mid F)=\frac{0+1}{23}=\frac1{23} $$ $$ P(\text{loan}\mid F)=\frac{1+1}{23}=\frac2{23} $$ #### Step 6. Multiply the finance score $$ \operatorname{score}(F)=\frac12\times\frac1{23}\times\frac1{23}\times\frac2{23}=\frac1{23^3}=\frac1{12167}\approx0.00008219. $$ #### Step 7. Calculate river likelihoods $$ P(\text{stream}\mid R)=\frac{1+1}{23}=\frac2{23} $$ $$ P(\text{shore}\mid R)=\frac{2+1}{23}=\frac3{23} $$ $$ P(\text{loan}\mid R)=\frac{0+1}{23}=\frac1{23} $$ #### Step 8. Multiply the river score $$ \operatorname{score}(R)=\frac12\times\frac2{23}\times\frac3{23}\times\frac1{23}=\frac3{12167}\approx0.00024657. $$ #### Step 9. Compare scores 3/12167 is larger than 1/12167. Riverbank wins even though “loan” is financial evidence: the combined clues matter. #### Step 10. Normalize if a posterior is requested $$ P(F\mid C)=\frac{1/12167}{4/12167}=\frac14=0.25 $$ $$ P(R\mid C)=\frac{3/12167}{4/12167}=\frac34=0.75. $$ **Answer.** Bank is the riverbank sense. Scores are 1/12167 and 3/12167; normalized model posteriors are 0.25 and 0.75. **Without smoothing:** stream and shore are absent in Finance; loan is absent in Riverbank. Both unsmoothed scores become zero, so there is no valid winning class from those products. Also, unnormalized scores should not be labelled as final posterior probabilities. Zero word? Add-one: count+1 upar, total words+vocabulary neeche. Last mein score bada wala sense. ### Naive Bayes WSD Numerical id : nlp-m2-18-naive-bayes-wsd title : Naive Bayes WSD Numerical title\_confirmed : true topic\_title : Naive Bayes WSD: a complete corpus numerical mode : m2 kind : focused status : available duration : 11:35 reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — Word-sense disambiguation | Reviewed Lesk and Bayes numericals id : overview-wsd title : NLP M2 — Word-sense disambiguation | Reviewed Lesk and Bayes numericals title\_confirmed : true topic\_title : WSD: Detective Math Toolkit mode : m2 kind : supplemental status : available duration : 6:11 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Lesk variants, conditional Bayes definitions, smoothing and both worked posteriors; corrected spoken claims. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids : - nlp-m2-16-lesk-wsd - nlp-m2-17-walker-graph-wsd - nlp-m2-18-naive-bayes-wsd These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · More WSD numericals: given likelihoods, unequal priors and repeated clues Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/u5-naive-bayes--u5-bayes-practice/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/u5-naive-bayes/u5-bayes-practice Unit 5 · PDF pages 49–57 **Question. For bass, let P(Fish)=0.6 and P(Music)=0.4. Given Fish likelihoods P(caught|Fish)=0.5, P(lake|Fish)=0.4 and Music likelihoods P(caught|Music)=0.05, P(lake|Music)=0.1, classify context {caught,lake} and normalize.** #### Given Priors and likelihoods are provided directly; no smoothing or training-count estimation is requested. #### To find The winning sense and its posterior. #### Formula and why it applies $$ \operatorname{score}(s)=P(s)P(\text{caught}\mid s)P(\text{lake}\mid s) $$ , because the two features are treated as conditionally independent. #### Step 1. Fish multiplication $$ 0.6\times0.5=0.3 $$ , then $$ 0.3\times0.4=0.12 $$ . #### Step 2. Music multiplication $$ 0.4\times0.05=0.02 $$ , then $$ 0.02\times0.1=0.002 $$ . #### Step 3. Total evidence $$ 0.12+0.002=0.122 $$ . #### Step 4. Normalize $$ P(Fish\mid C)=0.12/0.122\approx0.983607 $$ $$ P(Music\mid C)=0.002/0.122\approx0.016393. $$ **Answer.** Fish, with model posterior approximately 0.983607. **Question. Disambiguate bank from the repeated context tokens {loan, loan, river}. Priors are Finance=2/3 and River=1/3. Vocabulary V=4. Finance has N=8 total tokens, loan count 3 and river count 0; River has N=4, loan count 0 and river count 2. Apply add-one smoothing.** #### Given Multinomial counts are supplied. The two appearances of loan must both contribute. #### To find Scores and normalized Finance posterior. #### Formula and why it applies Naive Bayes selects the sense with the largest prior × likelihood product. The context evidence denominator is identical across candidate senses, so it can be omitted for comparison. $$ \operatorname{score}(s)=P(s)\prod_j P(w_j\mid s). $$ For add-one multinomial smoothing: $$ P(w\mid s)=\frac{C(w,s)+1}{N_s+V}. $$ $$ C(w,s) $$ counts occurrences of word $$ w $$ in training contexts of sense $$ s $$ ; $$ N_s $$ is the total number of word occurrences in those contexts; $$ V $$ is the shared vocabulary size. It is not the number of training sentences. Repeated query words contribute repeated factors. #### Step 1. Finance denominator $$ 8+4=12 $$ . $$ P(loan\mid F)=(3+1)/12=1/3 $$ ; $$ P(river\mid F)=(0+1)/12=1/12 $$ . #### Step 2. River denominator $$ 4+4=8 $$ . $$ P(loan\mid R)=(0+1)/8=1/8 $$ ; $$ P(river\mid R)=(2+1)/8=3/8 $$ . #### Step 3. Finance product $$ \operatorname{score}(F)=\frac23\times\frac13\times\frac13\times\frac1{12}=\frac1{162}\approx0.00617284. $$ #### Step 4. River product $$ \operatorname{score}(R)=\frac13\times\frac18\times\frac18\times\frac38=\frac1{512}\approx0.001953125. $$ #### Step 5. Compare and normalize $$ P(F\mid C)=\frac{1/162}{1/162+1/512}=\frac{512}{674}=\frac{256}{337}\approx0.759644. $$ **Answer.** Finance wins. Repeated loan contributes a squared likelihood; do not reduce the query to a unique-word set. ### Answer-writing order in an exam Write the class priors, vocabulary size, class token totals, a likelihood table, both score products and the final class. Keep fractions until the comparison where possible. If the question only asks classification, normalization is optional; if it asks posterior probabilities, normalization is required. Repeated word aaye toh factor bhi repeat. Sirf word-set bana diya toh multinomial answer badal jayega. ### Naive Bayes WSD Numerical id : nlp-m2-18-naive-bayes-wsd title : Naive Bayes WSD Numerical title\_confirmed : true topic\_title : Naive Bayes WSD: a complete corpus numerical mode : m2 kind : focused status : available duration : 11:35 reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — Word-sense disambiguation | Reviewed Lesk and Bayes numericals id : overview-wsd title : NLP M2 — Word-sense disambiguation | Reviewed Lesk and Bayes numericals title\_confirmed : true topic\_title : WSD: Detective Math Toolkit mode : m2 kind : supplemental status : available duration : 6:11 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Lesk variants, conditional Bayes definitions, smoothing and both worked posteriors; corrected spoken claims. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids : - nlp-m2-16-lesk-wsd - nlp-m2-17-walker-graph-wsd - nlp-m2-18-naive-bayes-wsd These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Distributional hypothesis and a co-occurrence matrix Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/u5-vectors--u5-distribution/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/u5-vectors/u5-distribution TEE · Unit 5 PDF pages 58–118 **The distributional hypothesis states that words occurring in similar contexts tend to have similar meanings or semantic behaviour.** “Cat” and “dog” often occur near pet, feed and animal. Their context patterns can therefore be represented and compared numerically. A **vector** is an ordered list of numbers. Each position must have a fixed meaning. For context order [pet, food, road], cat=[4,3,0] means four co-occurrences with pet, three with food, none with road. A **matrix** stacks vectors as rows in a table. A **word–context co-occurrence matrix** records how often a target word occurs with each context word under a specified context definition. A window may be a sentence, document, or a fixed number of neighbouring tokens. State its width, direction and whether targets count themselves. Choose target words → Choose context/window → Count co-occurrences → Store each row as a vector → Compare vectors **Question. Build the symmetric co-occurrence matrix for “cats eat fish” using a window of one token on either side, no self-counts and vocabulary [cats,eat,fish].** #### Given One sentence, one occurrence of each token. Left and right adjacent tokens are contexts. #### To find A 3×3 target-by-context count matrix. #### Formula and why it applies $$ X_{ij} $$ counts the number of times context word $$ j $$ occurs within the chosen window of target $$ i $$ . #### Step 1. Target cats Only eat is adjacent. Row cats = [0,1,0]. #### Step 2. Target eat cats is left and fish is right. Row eat = [1,0,1]. #### Step 3. Target fish Only eat is adjacent. Row fish = [0,1,0]. #### Step 4. Assemble the matrix | Target / context | cats | eat | fish | | --- | --- | --- | --- | | cats | 0 | 1 | 0 | | eat | 1 | 0 | 1 | | fish | 0 | 1 | 0 | **Answer.** Rows are [0,1,0], [1,0,1], [0,1,0]. The symmetry follows from the symmetric window. **Question. Build a target–context matrix for sentences “cats eat fish” and “dogs eat fish”. Target words are cats and dogs; context columns [eat,fish]; use the entire sentence excluding the target.** #### Given Each sentence contains one target. Count context token occurrences. #### To find Two context vectors. #### Formula and why it applies Count each context word in every sentence containing the target. #### Step 1. Count cats contexts In its sentence: eat occurs once, fish once → [1,1]. #### Step 2. Count dogs contexts In its sentence: eat occurs once, fish once → [1,1]. #### Step 3. Compare Both targets have identical observed context patterns in this very small corpus. **Answer.** cats=[1,1], dogs=[1,1]. This corpus supplies no context evidence to distinguish them. **Question. For “a b a”, use a right-only window of one token. Build the matrix with rows/columns [a,b].** #### Given Count directed target→next-word events, not both directions. #### To find The co-occurrence matrix. #### Formula and why it applies $$ X_{ij}=C(\text{target }i\text{ immediately followed by }j) $$ . #### Step 1. First target a Next token is b → add 1 to X(a,b). #### Step 2. Target b Next token is a → add 1 to X(b,a). #### Step 3. Final a No following token → no addition. #### Step 4. Write matrix | Target / next | a | b | | --- | --- | --- | | a | 0 | 1 | | b | 1 | 0 | **Answer.** a=[0,1], b=[1,0]. A different context window would produce a different matrix. Word ka meaning: uske aas-paas kaunse words kitni baar aaye, uski number-list. ### TEE 09 — Distributional semantics, co-occurrence matrices and sentence vectors id : tee-09 title : TEE 09 — Distributional semantics, co-occurrence matrices and sentence vectors title\_confirmed : false topic\_title : Distributional semantics, co-occurrence matrices and sentence vectors mode : tee kind : focused status : queued duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Sparse, dense, distributional and distributed Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/u5-vectors--u5-representation/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/u5-vectors/u5-representation TEE · Unit 5 PDF pages 58–118 **A sparse representation** has many zero entries and often one coordinate per vocabulary item. BoW and TF–IDF are common sparse document representations. **A dense embedding** has comparatively few learned real-valued coordinates; one word's properties are spread across many coordinates. | Representation | What coordinates mean | Strength | Limitation | | --- | --- | --- | --- | | One-hot | One identity position per vocabulary word | Unambiguous ID | Different words are orthogonal; no learned similarity | | Count-based context vector | Context-word co-occurrence counts | Transparent corpus evidence | Large, sparse; frequent words may dominate | | TF–IDF document vector | Weighted vocabulary terms | Useful retrieval weighting | Word order and contextual meaning are limited | | Dense word embedding | Learned latent features | Compact semantic/syntactic relationships | Coordinates are not generally human-labelled features | **Distributional** describes learning from patterns of linguistic context. **Distributed** describes representing an item across multiple coordinates rather than a single active identity bit. These are related but different ideas. A one-hot ID alone does not learn distributional similarity. ### Model design choices Choose contexts (document, sentence, fixed window), weighting (raw counts, normalization, TF–IDF-like weighting or association scores), dimensionality reduction or learned embeddings, and a similarity measure. Cosine compares direction and is less affected by length than a raw dot product; Euclidean distance also reflects magnitude. Sparse = bahut zero. Dense = chhoti learned number-list. Similar context se useful similarity seekhte hain. ### TEE 09 — Distributional semantics, co-occurrence matrices and sentence vectors id : tee-09 title : TEE 09 — Distributional semantics, co-occurrence matrices and sentence vectors title\_confirmed : false topic\_title : Distributional semantics, co-occurrence matrices and sentence vectors mode : tee kind : focused status : queued duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Word2Vec: CBOW and skip-gram from training pairs Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/u5-vectors--u5-word2vec/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/u5-vectors/u5-word2vec TEE · Unit 5 PDF pages 58–118 **Word2Vec learns dense word embeddings by predicting words from nearby context.** Its two common architectures are CBOW and skip-gram. A vocabulary ID or one-hot vector is the input representation; the learned embedding table converts that identity into a short trainable vector. | Architecture | Input | Prediction | Memory cue | | --- | --- | --- | --- | | CBOW | Surrounding context words | Centre/target word | Context → centre | | Skip-gram | Centre/target word | Surrounding words | Centre → context | CBOW: the + sat → Average context embeddings → Predict: cat Skip-gram: cat → Embedding of cat → Predict: the and sat For vocabulary size $$ V $$ and embedding dimension $$ d $$ , an input one-hot row is $$ 1\times V $$ ; the input embedding matrix is $$ V\times d $$ ; the hidden row is $$ 1\times d $$ ; the output matrix is $$ d\times V $$ ; output scores are $$ 1\times V $$ . Multiplying a one-hot row by the embedding matrix simply selects one row. $$ P(w_o\mid w_i)=\frac{\exp(u_{w_o}^{T}v_{w_i})}{\sum_{w\in V}\exp(u_w^{T}v_{w_i})}. $$ $$ v_{w_i} $$ is the input word embedding, $$ u_w $$ an output-context embedding, and the dot product is a compatibility score. Softmax turns all scores into probabilities that sum to one. Training increases the probability of observed context words. Negative sampling or hierarchical softmax can reduce computation in larger vocabularies; use the algorithm specified by the question. **Question. For the sentence “the cat sat on mat”, generate CBOW and skip-gram training data for centre word sat with a window of one token on each side.** #### Given Tokens indexed 1 to 5: the, cat, sat, on, mat. Centre = sat. #### To find The CBOW example and the skip-gram pairs. #### Formula and why it applies CBOW predicts the centre from its neighbours; skip-gram predicts each neighbour from the centre. #### Step 1. Identify neighbours Left neighbour = cat; right neighbour = on. “the” and “mat” are outside this window. #### Step 2. CBOW example Input [cat,on] → target sat. #### Step 3. Skip-gram examples (sat,cat) and (sat,on), with first item as input and second as predicted context. **Answer.** CBOW: [cat,on]→sat. Skip-gram: sat→cat and sat→on. **Question. The embeddings of context words cat and on are [1,2] and [3,0]. Compute the CBOW hidden vector using mean aggregation. Output vectors for [sat,ran] are [1,0] and [0,1]. Compute softmax probabilities.** #### Given Context embeddings [1,2], [3,0]; mean aggregation; two candidate output vectors. #### To find The hidden vector, logits and probabilities. #### Formula and why it applies Mean context vector $$ h=(v_1+v_2)/2 $$ . Logit $$ z_w=u_w^Th $$ . Softmax $$ P(w)=e^{z_w}/\sum_j e^{z_j} $$ . #### Step 1. Add coordinates $$ [1+3,\,2+0]=[4,2] $$ . #### Step 2. Divide by number of words $$ h=[4/2,2/2]=[2,1] $$ . #### Step 3. Compute logits $$ z_{sat}=1(2)+0(1)=2 $$ . $$ z_{ran}=0(2)+1(1)=1 $$ . #### Step 4. Exponentials $$ e^2\approx7.389056 $$ , $$ e^1\approx2.718282 $$ ; sum $$ \approx10.107338 $$ . #### Step 5. Divide $$ P(sat)\approx0.731059 $$ , $$ P(ran)\approx0.268941 $$ . **Answer.** h=[2,1]; sat is more probable (0.731059). **Question. A skip-gram centre embedding is v=[1,−1]. Output embeddings for [eat,sleep,drive] are [1,1], [0,1], [1,0]. Calculate the predicted context probabilities.** #### Given One centre vector and three output vectors; full softmax over these three words. #### To find Three logits and three probabilities. #### Formula and why it applies $$ z_j=u_j^Tv $$ , then $$ P(j)=e^{z_j}/\sum_k e^{z_k} $$ . #### Step 1. Eat dot product $$ 1(1)+1(-1)=0 $$ . #### Step 2. Sleep and drive dot products $$ z_{sleep}=0(1)+1(-1)=-1 $$ . $$ z_{drive}=1(1)+0(-1)=1 $$ . #### Step 3. Exponentiate and sum $$ e^0=1 $$ , $$ e^{-1}\approx0.367879 $$ , $$ e^1\approx2.718282 $$ . Sum $$ \approx4.086161 $$ . #### Step 4. Normalize $$ P(eat)\approx0.244728 $$ ; $$ P(sleep)\approx0.090031 $$ ; $$ P(drive)\approx0.665241 $$ . **Answer.** drive has the highest probability in this toy learned model. **Limitations:** a standard static Word2Vec vector merges a word's senses; it has limited representation of subword morphology and out-of-vocabulary words; its local prediction objective does not directly fit a full global co-occurrence table. Similarity is learned from data and is not a guarantee of factual or causal understanding. CBOW = neighbours se missing centre. Skip-gram = centre se neighbours. ### TEE 10 — Word2Vec: CBOW and skip-gram, training pairs and softmax numericals id : tee-10 title : TEE 10 — Word2Vec: CBOW and skip-gram, training pairs and softmax numericals title\_confirmed : false topic\_title : Word2Vec: CBOW and skip-gram, training pairs and softmax numericals mode : tee kind : focused status : queued duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Word analogies and sentence embeddings Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/u5-vectors--u5-sentence/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/u5-vectors/u5-sentence TEE · Unit 5 PDF pages 58–118 **A sentence embedding** represents an entire sentence by one fixed-length vector. A basic method averages word embeddings. More advanced encoders learn a sentence representation from context and task objectives. $$ v_{\text{sentence}}=\frac1n\sum_{i=1}^{n}v_{w_i}. $$ This means add corresponding coordinates and divide each coordinate by the number of words. Averaging ignores order: “dog bites man” and “man bites dog” have the same average if their words are identical. **Question. Average the word vectors [1,0], [0,2] and [2,1] to represent a three-word sentence.** #### Given Three two-dimensional vectors with equal weights. #### To find The two-dimensional sentence vector. #### Formula and why it applies Use coordinate-wise mean $$ v=(v_1+v_2+v_3)/3 $$ . #### Step 1. First coordinate $$ 1+0+2=3 $$ ; $$ 3/3=1 $$ . #### Step 2. Second coordinate $$ 0+2+1=3 $$ ; $$ 3/3=1 $$ . **Answer.** Sentence vector = [1,1]. **Question. Toy word vectors are king=[3,2], man=[1,0], woman=[1,1]. Find king−man+woman and compare to queen=[3,3].** #### Given All vectors use the same two-coordinate space. #### To find The analogy vector and its distance from queen. #### Formula and why it applies Vector arithmetic is coordinate by coordinate. Euclidean distance is the square root of the sum of squared coordinate differences. #### Step 1. Subtract man $$ [3-1,2-0]=[2,2] $$ . #### Step 2. Add woman $$ [2+1,2+1]=[3,3] $$ . #### Step 3. Compare with queen $$ \sqrt{(3-3)^2+(3-3)^2}=0 $$ . **Answer.** The toy analogy exactly equals queen. Real learned embeddings only approximate some analogies. **Contextual encoders** can assign “bank” different representations in different sentences. BERT and RoBERTa produce contextual token representations; obtaining strong sentence similarity vectors requires an appropriate pooling/training method, not simply assuming every raw token vector is a sentence embedding. Universal Sentence Encoder is designed to encode sentence-level meaning. Skip-thought predicts surrounding sentences; InferSent learns from natural-language-inference supervision. Sentence average: har column add karo, words ki count se divide. Word order ka loss yaad rakh. ### TEE 09 — Distributional semantics, co-occurrence matrices and sentence vectors id : tee-09 title : TEE 09 — Distributional semantics, co-occurrence matrices and sentence vectors title\_confirmed : false topic\_title : Distributional semantics, co-occurrence matrices and sentence vectors mode : tee kind : focused status : queued duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Why GloVe uses global statistics Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/u5-glove--u5-glove-theory/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/u5-glove/u5-glove-theory TEE · Unit 5 PDF pages 119–130 **GloVe (Global Vectors) learns word embeddings by fitting weighted relationships in a global word–context co-occurrence matrix.** Word2Vec uses local prediction examples; GloVe uses aggregate counts directly in its objective. $$ X_i=\sum_k X_{ik},\qquad P_{ik}=\frac{X_{ik}}{X_i}. $$ $$ X_{ik} $$ is how often context word $$ k $$ appears around target $$ i $$ . $$ X_i $$ is the total of row $$ i $$ . $$ P_{ik} $$ is the resulting context probability. The ratio $$ P(k\mid i)/P(k\mid j) $$ helps identify how a probe context differs between two words. For targets ice and steam, a probe such as solid should favour ice; gas should favour steam; a probe shared similarly by both gives a ratio near one. These are data tendencies, not universal numerical identities. **Question. For context solid, ice has count 12 out of row total 20; steam has count 2 out of row total 40. Compute P(solid|ice), P(solid|steam) and their ratio.** #### Given X(ice,solid)=12, X(ice)=20; X(steam,solid)=2, X(steam)=40. #### To find Two conditional probabilities and the ice/steam ratio. #### Formula and why it applies $$ P(k\mid i)=X_{ik}/X_i $$ , and $$ R=P(k\mid ice)/P(k\mid steam) $$ . #### Step 1. Ice probability $$ 12/20=0.6 $$ . #### Step 2. Steam probability $$ 2/40=0.05 $$ . #### Step 3. Ratio $$ 0.6/0.05=12 $$ . **Answer.** The probe solid is 12 times as probable in ice's context under these counts. **Question. For probe water, ice and steam both have row total 50. Counts are 10 and 10. Calculate the ratio.** #### Given Both probabilities use a denominator of 50. #### To find The probability ratio. #### Formula and why it applies Use row-normalized counts before taking the ratio. #### Step 1. Ice probability $$ 10/50=0.2 $$ . #### Step 2. Steam probability $$ 10/50=0.2 $$ . #### Step 3. Divide $$ 0.2/0.2=1 $$ . **Answer.** Ratio 1: this probe does not distinguish the targets in this toy data. **Question. For probe gas, counts are 1/100 around ice and 20/100 around steam. Compute the ice-to-steam ratio.** #### Given Conditional counts have the same row denominator. #### To find The ratio and interpretation. #### Formula and why it applies $$ R=P(gas\mid ice)/P(gas\mid steam) $$ . #### Step 1. Ice probability $$ 1/100=0.01 $$ . #### Step 2. Steam probability $$ 20/100=0.2 $$ . #### Step 3. Ratio $$ 0.01/0.2=0.05 $$ . **Answer.** Ratio 0.05, so gas is much more strongly associated with steam. Ratio >1: pehla word favoured. <1: doosra. ≈1: probe dono mein similar. ### GloVe: global counts, probability ratios and weighted loss id : tee-11 title : GloVe: global counts, probability ratios and weighted loss title\_confirmed : false topic\_title : GloVe: global counts, probability ratios and weighted loss mode : tee kind : focused status : not\_generated duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · The GloVe objective and numerical interpretation Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/u5-glove--u5-glove-loss/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/u5-glove/u5-glove-loss TEE · Unit 5 PDF pages 119–130 $$ J=\sum_{i,j:X_{ij}>0}f(X_{ij})\left(w_i^T\widetilde w_j+b_i+\widetilde b_j-\ln X_{ij}\right)^2. $$ **What each piece means:** $$ w_i $$ is a target vector; $$ \widetilde w_j $$ is a separate context vector; the dot product estimates their compatibility; biases adjust word-level baselines. The desired score is the natural log of the observed count. Squaring penalizes both positive and negative errors. Weighting prevents unreliable rare pairs or extremely frequent pairs from dominating inappropriately. $$ f(x)=\begin{cases}(x/x_{\max})^\alpha,&0 youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Reference, pragmatics and discourse Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/u6--reference/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/u6/reference TEE only · Unit 6 · syllabus-based supplement; faculty slide pack not supplied **Pragmatic analysis** interprets an utterance using its situation and the speaker’s intended meaning. **Discourse analysis** studies how utterances connect across a text or conversation. Syntax and literal word meanings alone cannot settle every interpretation. | Text | Literal content | Contextual interpretation | | --- | --- | --- | | “Can you close the window?” | A question about ability. | Usually a polite request. | | “Meera bought a laptop. It was expensive.” | Two sentences. | “It” connects the second sentence to the laptop. | | “Meet me here tomorrow.” | An instruction. | Me, here and tomorrow depend on speaker, place and date. | ### Reference, mention and antecedent A **referring expression** points to an entity; a **mention** is its occurrence in the text. Its **referent** is the entity being discussed. An **antecedent** is a preceding expression used to interpret a later referring expression. Reference resolution identifies the intended referent. Coreference resolution groups mentions that refer to the same entity. “Meera”name mention → Person entitysame referent → “she” / “the engineer”other mentions | Concept | Definition | Example | | --- | --- | --- | | Anaphora | Reference back to something introduced earlier. | “Riya arrived. She sat down.” She → Riya. | | Cataphora | Reference forward to a later expression. | “Before she spoke, Riya checked the microphone.” she → Riya. | | Exophora | Reference to the situation outside the text. | “Put it there”, said while pointing. | | Coreference | Two or more mentions refer to the same entity. | “Dr Rao”, “the lecturer”, “she”. | | Pronoun resolution | Find the referent of a pronoun. | Resolve “it” in “The laptop stopped because it overheated.” | Coreference is broader than pronoun resolution: repeated names and descriptions also form a chain. Anaphora describes direction/dependence; coreference describes identity. In “I bought a car. The engine is noisy”, engine is related to the car but is not the same entity. **Yaad rakho:** Reference = kiski baat? Coreference = alag mentions, same entity. Pronoun resolution = he/she/it kis se linked? ### Cohesion, coherence and discourse markers **Cohesion** is the explicit linguistic linking of a text through reference, repeated or related words, conjunctions and other devices. In “Rita bought a book. She read it”, she and it connect the second sentence to Rita and the book. **Coherence** is the conceptual unity that makes a discourse meaningful. “The road flooded, so the match was cancelled” presents a plausible cause and consequence. A reader relates the propositions using context and knowledge. **Discourse markers** signal relationships or organize speech: however signals contrast; therefore signals a result; first and next signal sequence; for example introduces an illustration. These concepts differ: “I like apples. Therefore the moon is square” has a linking word but no sensible inference. A connector alone does not guarantee coherence. **Yaad rakho:** Cohesion = words se link; coherence = meaning se link; marker = relation ka signal. ### Reference and coreference: mentions, antecedents and pronoun resolution id : tee-12 title : Reference and coreference: mentions, antecedents and pronoun resolution title\_confirmed : false topic\_title : Reference and coreference: mentions, antecedents and pronoun resolution mode : tee kind : focused status : not\_generated duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Pronoun resolution: methods and examples Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/u6--resolution/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/u6/resolution TEE only · Unit 6 · syllabus-based supplement; faculty slide pack not supplied ### A practical resolution procedure First detect the pronoun and candidate noun phrases. Eliminate candidates inconsistent with person, number, grammatical binding or relevant semantic constraints. Rank the surviving candidates using syntactic prominence, recency and discourse context. Preserve ambiguity if the available evidence does not decide. | Approach | How it works | Strength | Limitation | | --- | --- | --- | --- | | Rule based / syntactic | Use parse trees, agreement and explicit search order; Hobbs is a classic example. | Can explain each decision without labelled training data. | Parsing errors and world knowledge gaps cause errors. | | Salience based | Score candidates using recency, subjecthood and other discourse features; select the strongest compatible candidate. | Captures the tendency to refer again to prominent entities. | Feature weights and competing candidates can mislead. | | Supervised statistical | Learn candidate/pronoun compatibility from labelled examples. | Combines many features from data. | Needs representative annotated examples. | | Neural / contextual | Encode the surrounding text and score mention pairs or antecedents. | Can learn contextual patterns and long-range dependencies. | Needs data/compute and can still make confident mistakes. | ### Worked analysis 1 · number and semantics Resolve “they” in: “Riya put the books on the shelf because they were heavy.” **Given** Candidates: Riya (singular person), books (plural objects), shelf (singular object); pronoun they is plural here. **To find** The antecedent and a justified resolution procedure. ### Formula and why it applies Use number agreement first, then semantic compatibility. There is no multiplication formula in this linguistic task. ### Solution #### Step 1. List candidates Riya, the books, the shelf. #### Step 2. Apply number agreement Riya and shelf are singular. Books is plural, matching they. #### Step 3. Check meaning Books can be heavy; the causal explanation fits carrying/placing books. “They” refers to “the books”. ### Worked analysis 2 · a complete coreference chain Identify the coreference chain in: “Dr Sen entered. The surgeon greeted Ravi. She opened her notebook.” **Given** Mentions: Dr Sen, the surgeon, Ravi, she, her, her notebook. Assume the surgeon is Dr Sen and she/her refers to that surgeon. **To find** The person chain, and a mention that must not be placed in it. ### Formula and why it applies Group expressions by identity of referent; a possessive expression and the owned object are different entities. ### Solution #### Step 1. Link the name and description Dr Sen and the surgeon denote the same person under the given context. #### Step 2. Link the pronouns She and her refer to the surgeon. #### Step 3. Keep the notebook separate “Her notebook” is an object owned by the person; it is not the person herself. Person chain: {Dr Sen, the surgeon, She, her}. “Her notebook” is a separate object mention. ### Worked analysis 3 · honest ambiguity Discuss the pronoun in: “Asha called Nisha after she finished work.” **Given** Both named candidates are singular; the sentence gives no further context. **To find** Whether one referent is logically guaranteed. ### Formula and why it applies Agreement filters incompatible candidates but cannot always select between equally compatible candidates. ### Solution #### Step 1. Check agreement Both Asha and Nisha are compatible with she. #### Step 2. Check the event Either person could have finished work. #### Step 3. State the limitation A heuristic may prefer a candidate, but grammar and this sentence alone do not prove the intended referent. Ambiguous without further context. State any heuristic and parse assumption before giving its preferred candidate. ### Reference and coreference: mentions, antecedents and pronoun resolution id : tee-12 title : Reference and coreference: mentions, antecedents and pronoun resolution title\_confirmed : false topic\_title : Reference and coreference: mentions, antecedents and pronoun resolution mode : tee kind : focused status : not\_generated duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Hobbs’ algorithm, step by step Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/u6--hobbs/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/u6/hobbs TEE only · Unit 6 · syllabus-based supplement; faculty slide pack not supplied **Hobbs’ algorithm is a syntax-based procedure that searches constituent parse trees in a fixed order for a compatible noun-phrase antecedent of a pronoun.** The tree determines which candidates are proposed first. Agreement and other relevant constraints determine whether a proposed candidate can be accepted. It is not simply the rule “choose the closest noun”. A nearby noun may have the wrong number, may be syntactically ineligible, or may be visited after a structurally more prominent noun. The algorithm produces a preference under the supplied parse; ambiguous language can still require context. ### The tree vocabulary you need | Term | Meaning | Example | | --- | --- | --- | | NP | Noun phrase: a phrase referring to an entity | the student; Anita; they | | S | A sentence/clause node | the node containing a subject and predicate | | VP | Verb phrase | told Ravi that she should revise | | X | The NP or S node currently being searched | An algorithm label, not a word | | Path p | The branch travelled upward from the pronoun/previous X | Used to distinguish left and right subtrees | | N-bar (N′) | The noun-centred part of an NP in a grammar using that label | The nominal-child test depends on the given tree | | Breadth-first | Visit shallower nodes before deeper ones | Level 1 before level 2 | | Left-to-right | Visit equally deep nodes in their written/tree order | Earlier left branch before right branch | Locate pronoun NP → Climb to NP/S → Propose candidates in tree order → Check compatibility → Accept or continue searching ### The complete search procedure #### Step 1. Start at the pronoun Locate the NP immediately dominating the pronoun. “Dominating” means containing it as a descendant in the tree. #### Step 2. Climb to the first NP or S Move upward to the first NP or S above that starting NP. Call this node X. Keep the upward path p. #### Step 3. Search the left branches initially Below X, traverse branches to the left of p breadth-first and left-to-right. At this initial search, propose an NP only if another NP or S lies between that candidate and X. Test proposed candidates for the required agreement and constraints. #### Step 4. If X is the highest sentence node, inspect earlier sentences Search previous sentence trees from most recent to oldest. Within each, use breadth-first, left-to-right order to propose NPs. Accept the first eligible compatible candidate, or report that no candidate was found in the available context. #### Step 5. Otherwise climb again Move from X to the next NP or S above it. Record this new X and the path followed during the climb. #### Step 6. Apply the higher-NP test If the new X is an NP, propose X when the path did not pass through its nominal N-bar child. Use the given parse labels; do not invent an N-bar branch missing from the question's structure. #### Step 7. Search its left branches Traverse branches below the new X to the left of the path, breadth-first and left-to-right, proposing NPs. The special intervening-NP/S restriction belongs to the initial search in Step 3. #### Step 8. If X is S, search the permitted right branches Search branches to the right of the path breadth-first and left-to-right. Do not descend below an encountered NP or S. Propose eligible NP candidates there. #### Step 9. Repeat or stop Return to the highest-S check and continue the climb/search procedure until a compatible candidate is accepted or the available trees are exhausted. Record the candidate order rather than only writing an unexplained name. ### Why the first left search has an extra condition In a simple clause such as “John saw him”, the subject NP John is directly under the clause S. There is no extra NP or S between John and that S. The initial-search restriction therefore prevents casually selecting John as the ordinary local antecedent of him. Compare the reflexive “John saw himself”, which is a different grammatical case; do not treat every pronoun type as interchangeable. ### Worked tree 1 · move to the previous sentence **Question. Apply Hobbs’ search to “The student arrived. She smiled.” Assume the student is compatible with she.** #### Given Previous tree: [S [NP The student] [VP arrived]]. Current tree: [S [NP She] [VP smiled]]. #### To find The selected antecedent and the search path. #### Method and why it applies Use the supplied constituent trees. The current sentence has no earlier candidate, so the highest-S rule activates previous-sentence search. ### Solution #### Step 1. Draw both sentence structures ``` Previous sentence Current sentence S S ├── NP: The student ├── NP: She ← start └── VP: arrived └── VP: smiled ``` #### Step 2. Locate and climb Start at NP She. Its first higher NP/S is the current S. Set X to that S. #### Step 3. Search left of the path She is the leftmost subject. No NP lies to its left in the current sentence. #### Step 4. Use the highest-S rule This X is the highest S. Inspect the immediately preceding sentence tree. #### Step 5. Propose its first NP Breadth-first, left-to-right finds NP The student. #### Step 6. Check compatibility and accept The question stipulates that the student agrees with she. Accept this candidate. **Answer.** She refers to the student under the supplied assumptions. ### Worked tree 2 · reject an incompatible subject **Question. Resolve they in “Riya spoke to the students. They laughed.” Use a previous-sentence search and interpret they as plural.** #### Given Previous tree has subject NP Riya, then VP spoke [PP to [NP the students]]. Current subject is NP They. #### To find The candidate order and accepted antecedent. #### Method and why it applies Breadth-first search favours a shallow subject, but agreement must still be applied; a first candidate is not automatically a valid answer. ### Solution #### Step 1. Draw the useful part of the previous tree ``` S ├── NP: Riya ← candidate 1 └── VP ├── V: spoke └── PP ├── P: to └── NP: the students ← candidate 2 ``` #### Step 2. Search the current sentence From NP They climb to its S; there is no earlier NP in “They laughed”. This is the highest S, so move to the previous tree. #### Step 3. Visit Riya first Riya is the shallower leftmost NP. Under the stated plural interpretation, this singular candidate fails number agreement. #### Step 4. Visit the students Continue breadth-first. The next relevant NP is the students. #### Step 5. Apply number agreement Students is plural and is an eligible candidate. It matches plural they. #### Step 6. State the resolution Accept the students. The method did not simply accept the first noun it saw. **Answer.** They → the students. Candidate order: Riya rejected; the students accepted. ### Worked tree 3 · climb out of an embedded clause **Question. Resolve she in “Anita told Ravi that she should revise.” Assume Anita is compatible with she and Ravi is not. Use the displayed parse.** #### Given The pronoun is the subject of an embedded S inside the main VP. #### To find The preferred antecedent found by the climb and left-branch search. #### Method and why it applies The embedded clause has no left candidate. Climbing to the main S exposes its subject Anita as the first suitable higher-level candidate. ### Solution #### Step 1. Show the clause structure ``` S (main) ├── NP: Anita ← accepted candidate └── VP ├── V: told ├── NP: Ravi └── SBAR: that └── S (embedded) ← first X ├── NP: she ← start └── VP: should revise ``` #### Step 2. Start inside the embedded clause From NP she climb to the embedded S. There is no candidate to the left of she within this embedded S. #### Step 3. Climb to the next NP/S The embedded S is not the highest S. Move upward through SBAR and VP to the main S. These intermediate nodes are not NP/S search nodes. #### Step 4. Search left of the upward path At the main S, the subject NP Anita is a shallow left candidate. The object Ravi is deeper within the VP. Breadth-first order proposes Anita first. #### Step 5. Check the stated compatibility Anita is compatible with she by the question's assumption. Accept it; no need to choose a later incompatible candidate. #### Step 6. Give the result and its limit This is the algorithm's result under the displayed parse and agreement assumptions. It does not prove the intention of every speaker using an otherwise ambiguous sentence. **Answer.** She → Anita under the stated tree and compatibility information. ### Advantages, limitations and a complete exam answer | Advantages | Limitations | | --- | --- | | Explicit candidate order makes the decision traceable. | Wrong parse structure can produce the wrong search order. | | Requires no labelled coreference-training corpus for its basic rules. | World knowledge and discourse intention may override syntactic preference. | | Uses structure rather than only surface distance. | Some ambiguous, long-distance or unusual constructions need additional treatment. | For a longer answer, write the definition, inputs, tree notation, ordered algorithm, one labelled tree with a candidate table, final antecedent, and limitations. In a worked question show separately: candidate → why proposed → agreement result → accept/reject. Do not equate the selected candidate with a logically guaranteed interpretation when the sentence is ambiguous. **Yaad rakho:** Tree order candidate deta hai; agreement usko allow/reject karta hai. Nearest noun automatically answer nahi. ### Hobbs algorithm: parse-tree traversal with three complete traces id : tee-13 title : Hobbs algorithm: parse-tree traversal with three complete traces title\_confirmed : false topic\_title : Hobbs algorithm: parse-tree traversal with three complete traces mode : tee kind : focused status : not\_generated duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Text classification and its full pipeline Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/u7--classification/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/u7/classification TEE only · Unit 7 · current syllabus, including spelling correction; faculty slide pack not supplied **Text classification** assigns a document to one or more predefined categories. Spam detection usually uses two classes; news classification may use several topics. In single-label multiclass classification, choose one of several categories. In multilabel classification, several labels may apply to the same document. Define labels → Prepare labelled examples → Train representation + model → Evaluate on unseen text → Predict new labels ### An exam-ready pipeline | Stage | What you do | Why it matters | | --- | --- | --- | | Task definition | Decide what counts as spam or which news topics exist. | Unclear labels create inconsistent training data. | | Data collection and annotation | Collect representative examples and assign correct labels. | The model learns from these examples. | | Split data | Separate training, validation and test data. | Test examples must not influence training choices. | | Preprocess | Tokenize and normalise where suitable; retain meaningful negation. | Reduce irrelevant variation without deleting signal. | | Represent | Use BoW, TF-IDF or learned embeddings. | Models need numerical features. | | Train | Fit Naive Bayes, logistic regression or another appropriate model. | Learn how features predict categories. | | Evaluate | Measure relevant precision, recall and F1 on held-out examples. | Accuracy alone may hide poor minority-class performance. | | Deploy and monitor | Apply the same preprocessing and monitor changing language. | Incoming data can differ from old examples. | Naive Bayes is especially useful for hand calculations. With a bag-of-words representation, its class score is the prior multiplied by the conditional probability of each token; repeated tokens repeat the factor. The “naive” assumption is conditional independence of features given the class, not that words are independent in real language. $$ s(c)=P(c)\prod_{w\in V}P(w\mid c)^{C(w,d)} $$ $$ P(w\mid c)=\frac{C(w,c)+1}{N_c+|V|} $$ C(w,d) counts word w in document d. C(w,c) counts it in training documents belonging to class c. Nc is the total token count for class c. Add-one smoothing adds one to every vocabulary count and therefore adds |V| to the denominator. This is the same arithmetic taught for WSD, with document classes instead of senses. ### Worked classification · two-token message A spam classifier has vocabulary {win, prize, meeting}. Priors are P(spam)=P(ham)=0.5. Spam token counts are [3,2,0]; ham counts are [0,0,5]. With add-one smoothing, classify “win prize”. **Given** |V|=3; N(spam)=5; N(ham)=5; message has win once and prize once. **To find** Class scores and predicted label. ### Formula and why it applies $$ s(c)=P(c)P(\text{win}\mid c)P(\text{prize}\mid c) $$ $$ P(w\mid c)=\frac{C(w,c)+1}{N_c+|V|} $$ The model is multinomial Naive Bayes; multiply the two token likelihoods and the class prior. ### Solution #### Step 1. Compute the smoothed denominators $$ 5+3=8 $$ #### Step 2. Compute spam likelihoods $$ P(\text{win}\mid S)=\frac{3+1}{8}=\frac12 $$ $$ P(\text{prize}\mid S)=\frac{2+1}{8}=\frac38 $$ #### Step 3. Compute ham likelihoods $$ P(\text{win}\mid H)=P(\text{prize}\mid H)=\frac{0+1}{8}=\frac18 $$ #### Step 4. Multiply the spam score $$ s(S)=\frac12\times\frac12\times\frac38=\frac{3}{32}=0.09375 $$ #### Step 5. Multiply the ham score $$ s(H)=\frac12\times\frac18\times\frac18=\frac{1}{128}=0.0078125 $$ #### Step 6. Compare 0.09375 > 0.0078125. These are unnormalised scores; the label is chosen by the larger score. Predicted label: spam. **Yaad rakho:** Class prior × har word ki class-likelihood; sabse bada score wins. ### Text classification, sentiment and evaluation numericals id : tee-14 title : Text classification, sentiment and evaluation numericals title\_confirmed : false topic\_title : Text classification, sentiment and evaluation numericals mode : tee kind : focused status : not\_generated duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Sentiment analysis: meaning, methods, example Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/u7--sentiment/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/u7/sentiment TEE only · Unit 7 · current syllabus, including spelling correction; faculty slide pack not supplied **Sentiment analysis** identifies expressed opinion or attitude, often as positive, negative or neutral. Document-level analysis labels a complete text; sentence-level analysis labels a sentence; aspect-level analysis attaches sentiment to a specific feature or entity. | Review | Useful interpretation | | --- | --- | | “The display is excellent, but the battery is poor.” | Display: positive; battery: negative. One overall label loses this distinction. | | “The service is not good.” | Negative: negation changes good. Removing not can reverse the intended meaning. | | “Great, another two-hour delay.” | Likely sarcasm: literal positive word alone is insufficient. | ### Approaches **Lexicon based:** look up sentiment scores, combine them, and apply stated handling for negation/intensifiers. It is interpretable but fragile with context and sarcasm. **Supervised:** learn a mapping from labelled text features to sentiment labels. **Neural/contextual:** learn or use contextual representations to capture word interactions, then fine-tune a sentiment classifier. None guarantees correct interpretation of every review. ### A specified lexicon calculation Use good = +2, excellent = +3 and slow = −2. A “not” immediately before a sentiment word reverses its score. Score “The support was not good but the delivery was excellent and slow.” **Given** Three sentiment words; not modifies good only; sum word scores under the stated rule. **To find** Total sentiment score and label, taking positive totals as positive. ### Formula and why it applies $$ S=\sum_i s_i $$ Apply the given negation rule before summing. This is a simplified specified model, not a universal linguistic rule. ### Solution #### Step 1. Identify sentiment words good, excellent and slow. #### Step 2. Apply negation $$ s(\text{not good})=-2 $$ #### Step 3. Add the contributions $$ S=-2+3-2=-1 $$ #### Step 4. Map score to label The sum is negative. Total score = −1; negative under this rule. ### Text classification, sentiment and evaluation numericals id : tee-14 title : Text classification, sentiment and evaluation numericals title\_confirmed : false topic\_title : Text classification, sentiment and evaluation numericals mode : tee kind : focused status : not\_generated duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Precision, recall, F1 and accuracy Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/u7--evaluation/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/u7/evaluation TEE only · Unit 7 · current syllabus, including spelling correction; faculty slide pack not supplied Evaluation compares predicted results with known answers. A **true positive** is a correctly predicted positive; a **false positive** is a negative item predicted positive; a **false negative** is a missed positive; a **true negative** is a correctly rejected negative. $$ \text{Precision}=\frac{TP}{TP+FP} $$ $$ \text{Recall}=\frac{TP}{TP+FN} $$ $$ F_1=\frac{2PR}{P+R}=\frac{2TP}{2TP+FP+FN} $$ $$ \text{Accuracy}=\frac{TP+TN}{TP+TN+FP+FN} $$ Precision asks how many predicted positives were correct. Recall asks how many actual positives were found. F1 balances them by a harmonic mean. If a denominator is zero, state the convention rather than dividing by zero. ### Example 1 · evaluate a classifier A binary text classifier gives TP=6, FP=2, FN=4, TN=8. Compute precision, recall, F1 and accuracy. **Given** TP=6; FP=2; FN=4; TN=8. **To find** Four evaluation measures. ### Formula and why it applies $$ P=\frac{TP}{TP+FP},\quad R=\frac{TP}{TP+FN} $$ $$ F_1=\frac{2TP}{2TP+FP+FN},\quad A=\frac{TP+TN}{TP+FP+FN+TN} $$ Use predicted positives in precision, actual positives in recall, and all examples in accuracy. ### Solution #### Step 1. Precision $$ P=\frac{6}{6+2}=0.750000 $$ #### Step 2. Recall $$ R=\frac{6}{6+4}=0.600000 $$ #### Step 3. F1 $$ F_1=\frac{12}{12+2+4}=0.666667 $$ #### Step 4. Accuracy $$ A=\frac{6+8}{6+2+4+8}=0.700000 $$ Precision 0.7500; recall 0.6000; F1 0.6667; accuracy 0.7000. ### Example 2 · evaluate a classifier A binary text classifier gives TP=9, FP=3, FN=1, TN=7. Compute precision, recall, F1 and accuracy. **Given** TP=9; FP=3; FN=1; TN=7. **To find** Four evaluation measures. ### Formula and why it applies $$ P=\frac{TP}{TP+FP},\quad R=\frac{TP}{TP+FN} $$ $$ F_1=\frac{2TP}{2TP+FP+FN},\quad A=\frac{TP+TN}{TP+FP+FN+TN} $$ Use predicted positives in precision, actual positives in recall, and all examples in accuracy. ### Solution #### Step 1. Precision $$ P=\frac{9}{9+3}=0.750000 $$ #### Step 2. Recall $$ R=\frac{9}{9+1}=0.900000 $$ #### Step 3. F1 $$ F_1=\frac{18}{18+3+1}=0.818182 $$ #### Step 4. Accuracy $$ A=\frac{9+7}{9+3+1+7}=0.800000 $$ Precision 0.7500; recall 0.9000; F1 0.8182; accuracy 0.8000. ### Example 3 · evaluate a classifier A binary text classifier gives TP=4, FP=0, FN=4, TN=12. Compute precision, recall, F1 and accuracy. **Given** TP=4; FP=0; FN=4; TN=12. **To find** Four evaluation measures. ### Formula and why it applies $$ P=\frac{TP}{TP+FP},\quad R=\frac{TP}{TP+FN} $$ $$ F_1=\frac{2TP}{2TP+FP+FN},\quad A=\frac{TP+TN}{TP+FP+FN+TN} $$ Use predicted positives in precision, actual positives in recall, and all examples in accuracy. ### Solution #### Step 1. Precision $$ P=\frac{4}{4+0}=1.000000 $$ #### Step 2. Recall $$ R=\frac{4}{4+4}=0.500000 $$ #### Step 3. F1 $$ F_1=\frac{8}{8+0+4}=0.666667 $$ #### Step 4. Accuracy $$ A=\frac{4+12}{4+0+4+12}=0.800000 $$ Precision 1.0000; recall 0.5000; F1 0.6667; accuracy 0.8000. ### Text classification, sentiment and evaluation numericals id : tee-14 title : Text classification, sentiment and evaluation numericals title\_confirmed : false topic\_title : Text classification, sentiment and evaluation numericals mode : tee kind : focused status : not\_generated duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Spelling correction and edit distance Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/u7--spelling/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/u7/spelling TEE only · Unit 7 · current syllabus, including spelling correction; faculty slide pack not supplied **Spelling correction** proposes the intended word or sentence when the observed text contains an error. A non-word error produces an invalid vocabulary item (“recieve”); a real-word error produces an existing word in the wrong context (“I will meat you”). Dictionary lookup can detect many non-words but cannot reliably detect real-word errors. Detect suspicious text → Generate candidates → Score edit + context fit → Choose correction ### Minimum edit distance Levenshtein distance is the minimum total cost of insertions, deletions and substitutions needed to transform one string into another. In the examples below, each edit costs 1; matching characters cost 0. Transposition is not a single Levenshtein operation. State different costs explicitly if the question gives them. $$ D(i,j)=\min\begin{cases}D(i-1,j)+1\\D(i,j-1)+1\\D(i-1,j-1)+[a_i\ne b_j]\end{cases} $$ Rows represent prefixes of the source; columns represent prefixes of the target. D(i,j) is the best cost for transforming the first i source characters to the first j target characters. Initialise D(i,0)=i and D(0,j)=j. The three choices are deletion, insertion and diagonal match/substitution. ### Example 1 · cat → cut Calculate the unit-cost Levenshtein distance from “cat” to “cut”. Show the dynamic-programming table. **Given** Source=cat; target=cut; insertion=deletion=substitution=1; match=0. **To find** Minimum edit cost. ### Formula and why it applies $$ D(i,j)=\min(D(i-1,j)+1,D(i,j-1)+1,D(i-1,j-1)+c) $$ c = 0 for matching characters, 1 otherwise. Compare all three possible ways to reach each cell. ### Solution #### Step 1. Initialise empty prefixes First row counts insertions; first column counts deletions. #### Step 2. Fill row 1: source character “c” $$ D(1,1)=\min(1+1,\ 1+1,\ 0+0)=0 $$ $$ D(1,2)=\min(2+1,\ 0+1,\ 1+1)=1 $$ $$ D(1,3)=\min(3+1,\ 1+1,\ 2+1)=2 $$ #### Step 3. Fill row 2: source character “a” $$ D(2,1)=\min(0+1,\ 2+1,\ 1+1)=1 $$ $$ D(2,2)=\min(1+1,\ 1+1,\ 0+1)=1 $$ $$ D(2,3)=\min(2+1,\ 1+1,\ 1+1)=2 $$ #### Step 4. Fill row 3: source character “t” $$ D(3,1)=\min(1+1,\ 3+1,\ 2+1)=2 $$ $$ D(3,2)=\min(1+1,\ 2+1,\ 1+1)=2 $$ $$ D(3,3)=\min(2+1,\ 2+1,\ 1+0)=1 $$ #### Step 5. Read the completed matrix | Source / target | ∅ | c | u | t | | --- | --- | --- | --- | --- | | ∅ | 0 | 1 | 2 | 3 | | c | 1 | 0 | 1 | 2 | | a | 2 | 1 | 1 | 2 | | t | 3 | 2 | 2 | 1 | Minimum edit distance = 1. ### Example 2 · at → cat Calculate the unit-cost Levenshtein distance from “at” to “cat”. Show the dynamic-programming table. **Given** Source=at; target=cat; insertion=deletion=substitution=1; match=0. **To find** Minimum edit cost. ### Formula and why it applies $$ D(i,j)=\min(D(i-1,j)+1,D(i,j-1)+1,D(i-1,j-1)+c) $$ c = 0 for matching characters, 1 otherwise. Compare all three possible ways to reach each cell. ### Solution #### Step 1. Initialise empty prefixes First row counts insertions; first column counts deletions. #### Step 2. Fill row 1: source character “a” $$ D(1,1)=\min(1+1,\ 1+1,\ 0+1)=1 $$ $$ D(1,2)=\min(2+1,\ 1+1,\ 1+0)=1 $$ $$ D(1,3)=\min(3+1,\ 1+1,\ 2+1)=2 $$ #### Step 3. Fill row 2: source character “t” $$ D(2,1)=\min(1+1,\ 2+1,\ 1+1)=2 $$ $$ D(2,2)=\min(1+1,\ 2+1,\ 1+1)=2 $$ $$ D(2,3)=\min(2+1,\ 2+1,\ 1+0)=1 $$ #### Step 4. Read the completed matrix | Source / target | ∅ | c | a | t | | --- | --- | --- | --- | --- | | ∅ | 0 | 1 | 2 | 3 | | a | 1 | 1 | 1 | 2 | | t | 2 | 2 | 2 | 1 | Minimum edit distance = 1. ### Example 3 · cats → cat Calculate the unit-cost Levenshtein distance from “cats” to “cat”. Show the dynamic-programming table. **Given** Source=cats; target=cat; insertion=deletion=substitution=1; match=0. **To find** Minimum edit cost. ### Formula and why it applies $$ D(i,j)=\min(D(i-1,j)+1,D(i,j-1)+1,D(i-1,j-1)+c) $$ c = 0 for matching characters, 1 otherwise. Compare all three possible ways to reach each cell. ### Solution #### Step 1. Initialise empty prefixes First row counts insertions; first column counts deletions. #### Step 2. Fill row 1: source character “c” $$ D(1,1)=\min(1+1,\ 1+1,\ 0+0)=0 $$ $$ D(1,2)=\min(2+1,\ 0+1,\ 1+1)=1 $$ $$ D(1,3)=\min(3+1,\ 1+1,\ 2+1)=2 $$ #### Step 3. Fill row 2: source character “a” $$ D(2,1)=\min(0+1,\ 2+1,\ 1+1)=1 $$ $$ D(2,2)=\min(1+1,\ 1+1,\ 0+0)=0 $$ $$ D(2,3)=\min(2+1,\ 0+1,\ 1+1)=1 $$ #### Step 4. Fill row 3: source character “t” $$ D(3,1)=\min(1+1,\ 3+1,\ 2+1)=2 $$ $$ D(3,2)=\min(0+1,\ 2+1,\ 1+1)=1 $$ $$ D(3,3)=\min(1+1,\ 1+1,\ 0+0)=0 $$ #### Step 5. Fill row 4: source character “s” $$ D(4,1)=\min(2+1,\ 4+1,\ 3+1)=3 $$ $$ D(4,2)=\min(1+1,\ 3+1,\ 2+1)=2 $$ $$ D(4,3)=\min(0+1,\ 2+1,\ 1+1)=1 $$ #### Step 6. Read the completed matrix | Source / target | ∅ | c | a | t | | --- | --- | --- | --- | --- | | ∅ | 0 | 1 | 2 | 3 | | c | 1 | 0 | 1 | 2 | | a | 2 | 1 | 0 | 1 | | t | 3 | 2 | 1 | 0 | | s | 4 | 3 | 2 | 1 | Minimum edit distance = 1. ### Noisy-channel correction $$ \hat w=\arg\max_w P(x\mid w)P(w) $$ x is the observed misspelling and w a candidate intended word. P(x | w) is the error-model likelihood; P(w) is the language-model prior. In context, replace the isolated-word prior with an appropriate contextual probability. A frequent word with a plausible typo may beat a rarer candidate, but context can reverse the ranking. ### Candidate ranking For observed “teh”, candidate “the” has prior 0.05 and error likelihood 0.02. Candidate “ten” has prior 0.01 and likelihood 0.01. Select the intended word. **Given** Two candidates and their prior/error-model likelihoods are supplied. **To find** Largest noisy-channel score. ### Formula and why it applies $$ s(w)=P(x\mid w)P(w) $$ Multiply likelihood by prior; the same evidence normaliser is irrelevant to ranking. ### Solution #### Step 1. Score the $$ s(\text{the})=0.02\times0.05=0.001 $$ #### Step 2. Score ten $$ s(\text{ten})=0.01\times0.01=0.0001 $$ #### Step 3. Compare 0.001 is ten times 0.0001. Choose “the”. ### TEE 15 — Spelling correction: full edit-distance tables and noisy-channel ranking id : tee-15 title : TEE 15 — Spelling correction: full edit-distance tables and noisy-channel ranking title\_confirmed : false topic\_title : Spelling correction: full edit-distance tables and noisy-channel ranking mode : tee kind : focused status : queued duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Machine translation Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/u7--translation/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/u7/translation TEE only · Unit 7 · current syllabus, including spelling correction; faculty slide pack not supplied **Machine translation (MT) automatically converts a source-language text into a target-language text while aiming to preserve its meaning and produce natural, grammatical output.** The source is the language received; the target is the language generated. Translating words independently is insufficient because languages differ in order, agreement, idioms and word senses. ### What a translation must preserve | Requirement | What to check | Example | | --- | --- | --- | | Participants | Who performed the action and who received it | Asha helped Riya must not become Riya helped Asha | | Event and polarity | What happened, and whether it did not happen | Do not remove not | | Time/aspect | When and how the event unfolds | is working differs from worked | | Entities and quantities | Names, dates and numbers | ₹500 must not become ₹5,000 | | Intended sense | Meaning selected by context | Financial bank versus riverbank | | Target grammar | Natural order and agreement | Hindi object-before-verb often becomes English verb-before-object | Source text → Tokenize and analyse → Resolve structure + sense → Choose target words/structure → Generate target sentence → Check adequacy and fluency ### The three major approaches | Approach | Procedure | Advantages | Limitations | | --- | --- | --- | --- | | Rule-based MT | Analyse source words/grammar → transfer structure or meaning → generate target language using dictionaries and rules. | Explicit rules and terminology can be inspected. | Rule writing is expensive; exceptions and language pairs require maintenance. | | Statistical MT | Learn translation correspondences from parallel text; combine translation adequacy with a target-language model; search candidate translations. | Learns from examples and can use phrase correspondences. | Sparse data, word-order differences and rare phrases cause errors. | | Neural MT | Encode source context; a decoder generates target tokens conditioned on that context, often using attention. | Learns contextual representations and long-range interactions. | Needs suitable data/compute; can omit, distort or generate unsupported content. | **Parallel corpus:** aligned source and target examples expressing corresponding content. A monolingual corpus contains text in only one language. Parallel alignment does not mean every source word maps to exactly one target word: translation may insert, omit or reorder function words to preserve meaning. In a simple noisy-channel view of statistical MT, let f be the observed source sentence and e a candidate target sentence. The decoder compares translation compatibility and target-language fluency: $$ \hat e=\arg\max_e P(f\mid e)P(e). $$ P(f|e) is the translation-model likelihood; P(e) is the target-language-model prior. The common P(f) denominator is omitted for ranking. A modern MT system may use a different objective; this formula explains the stated statistical model rather than every translation system. ### Worked application 1 · translation requires reordering **Question. Translate the simple Hindi sentence “मैं पानी पीता हूँ” into natural English and explain the processing stages.** #### Given Intended subject is I; object is water; predicate expresses drinking in a habitual/simple-present reading. #### To find A natural translation with preserved roles and the stage-by-stage explanation. #### Method and why it applies Identify roles before generating the target order. Use natural English subject–verb–object order for this sentence rather than copying the Hindi order. ### Solution #### Step 1. Identify the source components | Source component | Function | Meaning | | --- | --- | --- | | मैं | Subject | I | | पानी | Object | water | | पीता हूँ | Predicate | drink, habitual/simple-present reading | #### Step 2. Build the meaning relation The speaker performs the action drink; water is what is drunk. The subject and object must stay attached to these roles. #### Step 3. Choose target lexical forms मैं → I; पानी → water; the predicate is naturally realized as drink with subject I in this reading. #### Step 4. Arrange target syntax English order here is subject I + verb drink + object water. #### Step 5. Check adequacy The participant, action and object are retained. No new object, negation or time expression has been invented. #### Step 6. Check fluency “I drink water” is natural English. A literal “I water drink” retains words but fails the required target order. **Answer.** A suitable translation is “I drink water.” The processing preserves meaning while changing structure. ### Worked application 2 · one word, two meanings **Question. Explain how an MT system should distinguish bank in “She deposited money in the bank” and “They sat on the bank of the river” before translation.** #### Given Two English source sentences contain the same written word bank. #### To find The intended sense in each and suitable Hindi phrase-level renderings. #### Method and why it applies Apply word-sense disambiguation using nearby context before choosing the target term. Identical spelling does not force identical translation. ### Solution #### Step 1. Collect the first sentence's clues Deposited and money strongly support the financial-institution sense. #### Step 2. Select its target expression Use बैंक for the financial institution; retain the money-deposit event in the complete translated sentence. #### Step 3. Collect the second sentence's clues River, sat and the location phrase support land beside a river. #### Step 4. Select its target expression Use नदी का किनारा / नदी के किनारे as appropriate to the target sentence. A financial-bank translation would be wrong here. #### Step 5. Generate grammatical context A suitable rendering of the second sentence is “वे नदी के किनारे बैठे।” Grammar adjusts the phrase to its role; do not paste an isolated dictionary form without inflection. #### Step 6. State why a dictionary alone fails A bilingual entry offers candidate words. Context and syntax select and inflect the candidate expressing the intended sense. **Answer.** Financial bank → बैंक; riverbank → नदी का किनारा, with grammatical realization such as नदी के किनारे in the sentence. ### Worked numerical 3 · compare two statistical translations **Question. A statistical MT system compares candidate A with P(f|A)=0.12 and P(A)=0.03, and candidate B with P(f|B)=0.08 and P(B)=0.06. Select the candidate under the noisy-channel product model.** #### Given Two candidate translations; likelihoods and target-language priors are provided. Ignore all other candidates for this ranking exercise. #### To find Each score and the selected translation. #### Method and why it applies Multiply translation likelihood by target-language prior because this specified model values both compatibility and fluency. These products are comparison scores, not normalized posterior probabilities. ### Solution #### Step 1. Write the scoring rule $$ s(e)=P(f\mid e)P(e). $$ #### Step 2. Compute candidate A $$ s(A)=0.12\times0.03=0.0036. $$ #### Step 3. Compute candidate B $$ s(B)=0.08\times0.06=0.0048. $$ #### Step 4. Compare the products 0.0048 > 0.0036. B wins even though A had a larger translation likelihood; B's target-language prior compensates. #### Step 5. Interpret the result The model prefers B under the supplied numbers. The score alone does not establish that B is semantically correct; a poor model can rank a bad translation highly. **Answer.** Select B, with score 0.0048 versus A's 0.0036. ### Evaluation and common failure cases **Adequacy** asks whether the original information and intended meaning were preserved. **Fluency** asks whether the target text is natural and grammatical. A fluent sentence can still mistranslate a negation or number, so assess both. Automatic reference-overlap metrics provide a useful signal, but several valid translations can use different words. | Failure | Why it occurs | What the system needs | | --- | --- | --- | | Word-for-word order | Source and target syntax differ | Structural modelling/reordering | | Wrong bank/bat sense | Multiple senses share a written form | Context-sensitive lexical choice | | Idiomatic phrase translated literally | Meaning is not the sum of isolated words | Phrase/context knowledge | | Rare name or specialist term corrupted | Weak vocabulary/domain coverage | Appropriate data and terminology handling | | Missing negation or number | Omission or generation error | Content-preservation checks | | Low-resource language pair | Few representative parallel examples | Suitable data collection/adaptation | For a 10-mark design explanation, connect the definition, pipeline, three approaches, worked source-to-target example, data requirements, evaluation and limitations. Merely listing “rule based, statistical, neural” does not explain how a translation is produced. **Yaad rakho:** MT: pehle kaun-kya-kisko samjho; phir target language ke words aur order mein wahi meaning banao. ### Machine translation: meaning preservation and worked processing id : tee-16 title : Machine translation: meaning preservation and worked processing title\_confirmed : false topic\_title : Machine translation: meaning preservation and worked processing mode : tee kind : focused status : not\_generated duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Question answering Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/u7--qa/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/u7/qa TEE only · Unit 7 · current syllabus, including spelling correction; faculty slide pack not supplied **Question answering (QA) returns an answer to a natural-language question using available evidence or knowledge.** Information retrieval usually returns ranked documents; QA must identify the requested information and deliver the answer. A relevant document is useful evidence but is not automatically the answer. ### Question types and answer types | Question | Expected answer type | Required operation | | --- | --- | --- | | Who founded the company? | Person or organisation | Find the founder relation | | When did the museum open? | Date/time | Locate the opening date | | Where is the campus? | Location | Locate the campus-location relation | | How many books were issued? | Quantity | Read or compute the stated count | | Why was the train cancelled? | Reason/explanation | Find the causal evidence | | Is the office open on Sunday? | Yes/no with evidence | Check the relevant schedule statement | **Closed-domain QA** operates within a restricted area such as a university handbook. **Open-domain QA** addresses a wider range of subjects and usually needs broad retrieval. **Extractive QA** selects a span from a passage. **Generative QA** constructs answer text; it still must preserve the evidence and avoid unsupported details. Question analysis → Evidence retrieval → Passage/record understanding → Answer selection or composition → Support check → Present answer ### Explain each module | Module | What happens | Why it matters | | --- | --- | --- | | Question analysis | Identify topic, intent, constraints and answer type. | When asks for a time, not a place. | | Retrieval | Find a relevant passage or structured fact. | Reduces the search space. | | Evidence understanding | Parse relations, resolve references and ambiguous words. | The correct date must belong to the correct event. | | Answer formation | Extract a span or compose a short supported response. | Turns a document into an actual answer. | | Verification | Check whether every claimed detail follows from evidence. | Prevents filling gaps by guessing. | | Presentation | Return a concise answer with suitable context. | Avoids dumping unrelated text. | ### Worked application 1 · extract the correct fact **Question. Given “The City Museum opened in 1998 in Pune. It was renovated in 2016”, answer “When did the museum open?” Show the QA stages.** #### Given A two-sentence passage. Dates 1998 and 2016 refer to different events. #### To find The opening date, not the renovation date. #### Method and why it applies Use extractive QA: identify the requested event and expected date type, then choose the span linked to that event. ### Solution #### Step 1. Analyse the question When requires a date/time answer. The event is opening; the entity is the City Museum. #### Step 2. Retrieve the relevant sentence The first sentence mentions the museum and its opening. The second sentence discusses renovation. #### Step 3. Identify event–date relations | Event | Date | | --- | --- | | opened | 1998 | | renovated | 2016 | #### Step 4. Select the answer span 1998 is attached to opened, so select the four-character year span 1998. #### Step 5. Reject the distractor 2016 is also a date, but it answers a different question: when was the museum renovated? #### Step 6. Present the answer “The museum opened in 1998.” **Answer.** 1998. Matching only the answer type date would be insufficient without matching the event. ### Worked application 2 · resolve a pronoun before answering **Question. Given “Mira joined Orion Labs in 2022. She became its research lead in 2024”, answer “Who became Orion Labs' research lead, and when?”** #### Given The passage contains the name Mira, pronoun She, organisation Orion Labs and possessive its. Assume the normal reading that She refers to Mira and its to Orion Labs. #### To find The person and the date of becoming research lead. #### Method and why it applies Combine reference resolution with relation extraction. The question asks for two fields, both tied to the promotion/role event. ### Solution #### Step 1. Identify requested fields Who → person; when → date. The relation is became research lead of Orion Labs. #### Step 2. Resolve mentions | Mention | Referent | | --- | --- | | She | Mira | | its | Orion Labs | #### Step 3. Read the target event After resolution, the second sentence states that Mira became Orion Labs' research lead in 2024. #### Step 4. Distinguish the dates 2022 belongs to joining the organisation. 2024 belongs to becoming research lead. #### Step 5. Compose a supported answer “Mira became Orion Labs' research lead in 2024.” #### Step 6. Verify completeness The answer supplies both requested fields and does not add an unsupported department, qualification or reason. **Answer.** Mira, in 2024. Reference resolution connects the second sentence to the named person and organisation. ### Worked application 3 · detect insufficient evidence **Question. A handbook excerpt states “The library opens at 8 a.m. on weekdays.” A user asks “At what time does it close on Sunday?” Explain the correct QA response.** #### Given Only the opening time for weekdays is supplied. No closing time or Sunday schedule is present. #### To find Whether an answer is supported and how to handle the missing information. #### Method and why it applies Compare every question constraint against the evidence. Topic overlap with library is not sufficient: time type and day must also match. ### Solution #### Step 1. Extract the question constraints | Constraint | Requested | | --- | --- | | Entity | Library | | Event | Closing | | Day | Sunday | | Answer type | Time | #### Step 2. Extract the supplied fact Entity=library; event=opening; day=weekdays; time=8 a.m. #### Step 3. Identify mismatches Opening is not closing. Weekdays do not supply a Sunday schedule. #### Step 4. Avoid the unsupported substitution Do not answer 8 a.m. just because it is the only time in the passage. #### Step 5. Return an evidence-limited response “The excerpt does not state the library's Sunday closing time.” A system with further retrieval can search a relevant schedule. #### Step 6. State the benefit The system avoids converting a relevant-looking number into an incorrect answer. **Answer.** No Sunday closing time is supported by the excerpt. Report insufficient evidence or retrieve the missing schedule. ### Advantages, limits and evaluation | Design choice | Benefit | Limitation | | --- | --- | --- | | Extractive answer | The returned wording comes directly from evidence. | The answer may require combining sentences or may not be present as one span. | | Generative answer | Can combine supported facts into a readable response. | Can introduce unsupported information unless carefully checked. | | Closed-domain collection | Controlled terminology and relevant resources. | Cannot answer facts outside its knowledge source. | | Broad retrieval | Wider question coverage. | Irrelevant, outdated or conflicting sources can be retrieved. | Evaluate **exact match** when a canonical answer string is expected, **token overlap/F1** when wording may vary, and **factual correctness and completeness** when answers are composed. An answer can be grammatically fluent but wrong. Questions with no answer in the evidence should be included in evaluation so the system's restraint is also tested. For a longer exam answer, define QA, distinguish it from retrieval, explain answer types and system variants, draw the pipeline, work through a passage including distractors, and discuss ambiguity, missing evidence and evaluation. A list of chatbot names is not a QA design. **Yaad rakho:** Question ka entity + event + constraint match karo. Passage mein koi bhi number mil gaya matlab answer nahi. ### Question answering: identify, retrieve and justify the answer id : tee-17 title : Question answering: identify, retrieve and justify the answer title\_confirmed : false topic\_title : Question answering: identify, retrieve and justify the answer mode : tee kind : focused status : not\_generated duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Information extraction Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/u7--extraction/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/u7/extraction TEE only · Unit 7 · current syllabus, including spelling correction; faculty slide pack not supplied **Information extraction (IE) converts unstructured language into structured information such as named entities, relations and events.** A structured output has explicit fields or records that can be stored, searched or analysed. IE is broader than NER: finding two names does not yet determine how those entities are related. ### The three levels of extracted information | Level | What it identifies | Example | | --- | --- | --- | | Entity extraction / NER | A span and its entity type | Asha → person; Orbit Labs → organisation | | Relation extraction | A labelled connection between entities | works-for(Asha, Orbit Labs) | | Event extraction | An occurrence with roles and attributes | Joining: employee=Asha, employer=Orbit Labs, date=2025 | An **entity** is a participant such as a person or organisation. A **relation** links participants. An **event** describes something that happened, may happen or did not happen. An **event role** names a participant's function: buyer, seller, employee, employer, location or time. These labels must follow the chosen extraction schema. Raw text → Sentence/token processing → Entity spans + types → Reference resolution → Relations and event roles → Normalize and store records ### An extraction pipeline explained | Step | Operation | Reason | | --- | --- | --- | | Specify schema | Choose fields and allowed relations/events. | A system needs to know what records it is building. | | Preprocess | Segment sentences and tokenize without losing useful structure. | Locate expressions and boundaries. | | Recognise entities | Find spans and assign types. | Names can contain several words. | | Resolve references | Link pronouns/descriptions to the right entities. | Two mentions may denote one participant. | | Extract links/events | Use patterns, syntax or learned models. | Co-occurrence alone does not identify a relationship. | | Handle qualifiers | Preserve negation, time and uncertainty. | Not hired differs from hired. | | Normalize | Standardize dates, names or quantities where justified. | Equivalent forms should be comparable. | | Store/evaluate | Save records and compare them with annotated answers. | Wrong links matter even if entity names are correct. | ### Worked application 1 · turn a sentence into a record **Question. Extract entities and a joining event from “Asha joined Orbit Labs in Mumbai on 3 May 2025.” Use fields person, organisation, location, event and date.** #### Given One affirmative sentence; Orbit Labs is an organisation and Mumbai is a location. #### To find Entity spans/types and one structured joining record. #### Method and why it applies Find the complete entity spans, then assign their roles using the verb joined and its grammatical context. ### Solution #### Step 1. Mark entity spans | Span | Type | | --- | --- | | Asha | Person | | Orbit Labs | Organisation | | Mumbai | Location | | 3 May 2025 | Date | #### Step 2. Identify the event trigger Joined signals a joining event. The subject Asha is the joining person; Orbit Labs is the joined organisation. #### Step 3. Attach contextual fields In Mumbai supplies a location associated with the stated joining event. On 3 May 2025 supplies its date. Do not infer the company's permanent headquarters from this sentence alone. #### Step 4. Normalize the date The unambiguous date can be stored as 2025-05-03 while preserving its original text. #### Step 5. Build the record | Field | Value | | --- | --- | | event | joining | | person | Asha | | organisation | Orbit Labs | | location | Mumbai | | date | 2025-05-03 | #### Step 6. State the information limit The sentence supports a joining event on that date. By itself it does not prove that Asha still works there indefinitely. **Answer.** A joining record for Asha and Orbit Labs at the stated location/date, with complete entity spans preserved. ### Worked application 2 · direction of a relation matters **Question. From “Nova acquired Luma for ₹50 crore in 2025”, extract an acquisition event with buyer, acquired organisation, amount and year.** #### Given Nova and Luma are organisation names. The active verb is acquired; no seller is named. #### To find A correctly directed acquisition record. #### Method and why it applies Use grammatical roles to identify buyer and acquired organisation. Merely returning the unordered pair {Nova,Luma} loses the event direction. ### Solution #### Step 1. Recognise participant entities Nova is the subject organisation; Luma is the object organisation. #### Step 2. Locate the trigger Acquired identifies an acquisition event. #### Step 3. Assign roles | Role | Entity | | --- | --- | | Buyer/acquirer | Nova | | Acquired organisation | Luma | #### Step 4. Extract amount and time Amount text=₹50 crore; year=2025. Currency and magnitude belong together; do not drop crore or replace it by lakh. #### Step 5. Create the record | Field | Value | | --- | --- | | event | acquisition | | buyer | Nova | | acquired\_organisation | Luma | | amount | INR 50 crore | | year | 2025 | | seller | Not stated | #### Step 6. Reject an unsupported inference The sentence does not name the former owner/seller. Luma is the acquired organisation, not automatically the seller. **Answer.** Acquisition(buyer=Nova, acquired organisation=Luma, amount=INR 50 crore, year=2025). Seller remains unspecified. ### Worked application 3 · reference, negation and time **Question. Extract employment-related events from “Ravi joined Vega in 2021. He left the company in 2024. He did not join Orion.” Keep event type, person, organisation, date and polarity.** #### Given He refers to Ravi; the company refers to Vega. The last sentence explicitly negates joining Orion. #### To find A structured event list without claiming an unsupported current employer. #### Method and why it applies Resolve references, identify each event separately and preserve whether it is affirmed or negated. ### Solution #### Step 1. Resolve the repeated mentions | Mention | Referent | | --- | --- | | He in sentence 2 | Ravi | | the company | Vega | | He in sentence 3 | Ravi | #### Step 2. Extract the first event Joining; person=Ravi; organisation=Vega; date=2021; polarity=affirmed. #### Step 3. Extract the second event Leaving; person=Ravi; organisation=Vega; date=2024; polarity=affirmed. #### Step 4. Extract the third statement Joining; person=Ravi; organisation=Orion; date=not stated; polarity=negated. This must not create an affirmative employment relation with Orion. #### Step 5. Write the event table | Event | Person | Organisation | Date | Polarity | | --- | --- | --- | --- | --- | | joining | Ravi | Vega | 2021 | Affirmed | | leaving | Ravi | Vega | 2024 | Affirmed | | joining | Ravi | Orion | Not stated | Negated | #### Step 6. Interpret current status cautiously The text says that Ravi left Vega and did not join Orion. It supplies no other current employer. Do not fill that field by guessing. **Answer.** Three qualified event statements, including a negated Orion joining. No current employer can be established from this text. ### Approaches and their trade-offs | Approach | Mechanism | Advantage | Limitation | | --- | --- | --- | --- | | Dictionary/rule based | Match known names, trigger words and patterns such as person joined organisation. | Transparent and useful in a narrow domain. | New names, paraphrases and complex syntax can escape patterns. | | Supervised statistical | Learn entity/relation/event labels from annotated examples and features. | Combines several cues consistently. | Requires representative labels and careful feature design. | | Neural/contextual | Use contextual representations to classify spans and links or generate a structured record. | Can capture paraphrases and broader context. | May still invent or misassign fields; schema and factual checks remain necessary. | Applications include extracting company events from news, conditions/treatments from clinical text, skills from resumes and attributes from product descriptions. Extraction quality is measured against annotated records: an entity match can require the correct span and type; a relation match additionally requires the correct participants and direction. Precision measures how many extracted records are correct; recall measures how many gold records were found. For a 10-mark answer, begin with the definition and structured output, distinguish NER/relation/event extraction, draw and explain the pipeline, work through a record with roles and qualifiers, compare approaches, then state evaluation and limitations. “Extract keywords” alone is too narrow: keywords do not necessarily represent a factual relation. **Yaad rakho:** NER naam pakadta hai; relation kaun-kisse jodta hai; event kya hua batata hai. Negation aur date mat girao. ### Information extraction and retrieval: records, indexes and precision/recall id : tee-18 title : Information extraction and retrieval: records, indexes and precision/recall title\_confirmed : false topic\_title : Information extraction and retrieval: records, indexes and precision/recall mode : tee kind : focused status : not\_generated duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Information retrieval Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/read/u7--retrieval/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/read/u7/retrieval TEE only · Unit 7 · current syllabus, including spelling correction; faculty slide pack not supplied **Information retrieval (IR)** locates and ranks items relevant to an information need, usually expressed as a query. The collection may contain documents, product descriptions or scientific papers. Retrieval estimates relevance; it does not itself establish that every retrieved claim is true. Documents → cleaning → Build index → Query → same representation → Match & rank → Return relevant results ### The inverted index An inverted index maps a term to the documents containing it, often with positions or counts. For D1 “cats drink milk” and D2 “dogs drink water”, the posting list for drink is {D1,D2}, while milk maps only to D1. This avoids rescanning every document for every query. | Term | Posting list | | --- | --- | | cats | D1 | | drink | D1, D2 | | milk | D1 | | dogs | D2 | | water | D2 | ### Matching and ranking Boolean retrieval combines term membership using AND, OR and NOT. Vector-space retrieval represents documents and queries as weighted vectors, for example with TF-IDF, and compares them with cosine similarity. Learned retrieval can use dense embeddings for meaning-based matching. The representation used at query time must be compatible with the document index. ### Search engines and recommendation A search engine receives an explicit query and ranks matching indexed pages. A recommender may derive an implicit query from a user’s liked items or profile and retrieve similar candidates. Both require candidate retrieval and ranking, but the source of the information need differs. ### Evaluate the results For retrieval, TP is the number of retrieved relevant documents, FP retrieved irrelevant documents, and FN relevant documents not retrieved. Precision and recall use the same formulas as classification. Ranking measures also consider the positions of relevant items; a relevant result at rank 1 is more useful than one far below. ### Retrieval evaluation A search system retrieves 8 documents; 6 are relevant. There are 10 relevant documents in the collection. Find precision, recall and F1. **Given** Retrieved=8; relevant retrieved=6; all relevant=10. **To find** P, R and F1. ### Formula and why it applies $$ P=\frac{\text{relevant retrieved}}{\text{retrieved}},\quad R=\frac{\text{relevant retrieved}}{\text{all relevant}} $$ $$ F_1=\frac{2PR}{P+R} $$ The denominator is 8 for precision and 10 for recall because they ask different questions. ### Solution #### Step 1. Identify the confusion counts TP=6; FP=8−6=2; FN=10−6=4. #### Step 2. Precision $$ P=\frac68=0.75 $$ #### Step 3. Recall $$ R=\frac6{10}=0.6 $$ #### Step 4. F1 $$ F_1=\frac{2(0.75)(0.6)}{0.75+0.6}=\frac{0.9}{1.35}=\frac23\approx0.666667 $$ Precision=0.75, recall=0.60, F1≈0.666667. **Yaad rakho:** Precision: jo laaye, unmein kitne sahi? Recall: jo sahi the, unmein kitne laaye? ### Information extraction and retrieval: records, indexes and precision/recall id : tee-18 title : Information extraction and retrieval: records, indexes and precision/recall title\_confirmed : false topic\_title : Information extraction and retrieval: records, indexes and precision/recall mode : tee kind : focused status : not\_generated duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Core formula sheet Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/formulas/formula-index--all/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/formulas/formula-index/all M2 · formulas with meaning · full examples follow Read each formula with its meaning. The remaining lessons in this tab contain the complete worked examples for these methods. Use the same vocabulary, boundary, case and token-count conventions throughout one solution. ### N-gram count $$ \max(T-n+1,0) $$ Unpadded sequence of T tokens. Count only valid contiguous windows. ### Unigram MLE $$ P(w)=C(w)/N $$ N is the training-token total under the stated boundary convention. ### Bigram MLE $$ P(w_i\mid w_{i-1})=C(w_{i-1},w_i)/C(w_{i-1}) $$ Denominator = history count. ### Trigram MLE $$ P(w_i\mid w_{i-2},w_{i-1})=C(w_{i-2},w_{i-1},w_i)/C(w_{i-2},w_{i-1}) $$ Denominator = two-word history count. ### Add-one / Laplace $$ P(w\mid h)=\frac{C(hw)+1}{C(h)+|V|} $$ Add one to every possible vocabulary continuation. ### Sentence probability $$ P(W)\approx\prod_iP(w_i\mid w_{i-1}) $$ Multiply in order; include an end factor if specified. ### Perplexity $$ PP=P(W)^{-1/N} $$ N is the number of predicted tokens/events used by the question. ### Emission $$ P(w\mid t)=C(w,t)/C(t) $$ Word given tag; count tokens with that tag. ### Transition $$ P(t_j\mid t_i)=C(t_i\to t_j)/\sum_k C(t_i\to t_k) $$ Next tag given previous tag; count outgoing transitions. ### HMM joint path $$ P(W,T)=\pi_{t_1}b_{t_1}(w_1)\prod_{i=2}^{n}a_{t_{i-1},t_i}b_{t_i}(w_i) $$ Multiply initial, emission and transition factors; add end probability if used. ### Tag sequences $$ K^T $$ K tags, T word positions; count before constraints. ### Viterbi $$ \delta_i(t)=b_t(w_i)\max_s[\delta_{i-1}(s)a_{s,t}] $$ Choose the largest predecessor score; store a backpointer. ### Lesk $$ score(s)=|context\cap gloss(s)| $$ Use the specified normalisation and overlap convention. ### Naive Bayes class prior $$ P(c)=D_c/D $$ D counts labelled examples; this is not the word-token total. ### Smoothed NB likelihood $$ P(w\mid c)=\frac{C(w,c)+1}{N_c+|V|} $$ N\_c counts training tokens in class c. ### NB decision $$ \hat c=\arg\max_c P(c)\prod_iP(w_i\mid c) $$ Select label with largest score; normalise only for posterior values. **Yaad rakho:** Condition jis par hai, denominator usi ka; sequence mein multiply; options mein maximum. --- ## TEE · Unigrams, bigrams and trigrams: construct them correctly Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/formulas/f-counts--n-grams-construction/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/formulas/f-counts/n-grams-construction Formulas explained through full worked examples **Definition.** An n-gram is a contiguous sequence of n items. For word n-grams, the items are consecutive word tokens. A unigram contains one word; a bigram contains two; a trigram contains three. Character n-grams use characters instead, so always identify the unit being counted. Tokens: we | study | natural | language → Move a window one token at a time → Bigrams: we study / study natural / natural language For m tokens without padding, the number of n-gram *positions* is max(0, m−n+1). Repeated positions may contain the same n-gram type. A bag-of-n-grams representation counts each n-gram type and loses the overall sentence order beyond the local order inside each n-gram. $$ \text{Number of unpadded n-gram positions}=\max(0,m-n+1) $$ **Question. List all word unigrams, bigrams and trigrams for “the prime minister of our country”. Do not add boundary tokens.** #### Given Six tokens: the, prime, minister, of, our, country. n = 1, 2 and 3. #### To find All three lists and their counts. #### Formula and why it applies $$ \#\text{positions}=m-n+1 $$ Each window must contain consecutive words and move one position. No wraparound or invented final word is allowed. #### Solution ##### Step 1. Take one word at a time Unigrams: (the), (prime), (minister), (of), (our), (country). Count = 6−1+1 = 6. ##### Step 2. Take two consecutive words Bigrams: (the, prime), (prime, minister), (minister, of), (of, our), (our, country). Count = 6−2+1 = 5. ##### Step 3. Take three consecutive words Trigrams: (the, prime, minister), (prime, minister, of), (minister, of, our), (of, our, country). Count = 6−3+1 = 4. **Final answer.** 6 unigrams, 5 bigrams and 4 trigrams. **Question. Construct bigrams for “I like tea” using one start and one end marker.** #### Given Original tokens: I, like, tea. Add and once each. #### To find The boundary-aware bigram sequence. #### Formula and why it applies $$ \#\text{bigrams}=(m+2)-1=m+1 $$ The start context predicts the first word, and the last word predicts that the sentence ends. #### Solution ##### Step 1. Write the padded sequence I like tea ##### Step 2. Move a two-token window (, I), (I, like), (like, tea), (tea, ). ##### Step 3. Count the windows $$ 3+1=4 $$ **Final answer.** There are four boundary-aware bigrams. **Question. Construct trigrams for “I like tea” using two start markers and one end marker.** #### Given Original sentence has three words. For a trigram model each predicted token needs two prior context tokens. #### To find All trigram prediction events. #### Formula and why it applies $$ P(w_i\mid w_{i-2},w_{i-1}) $$ Two start markers provide a complete context even for the first word. #### Solution ##### Step 1. Pad with two starts I like tea ##### Step 2. Slide a three-token window (, , I), (, I, like), (I, like, tea), (like, tea, ). ##### Step 3. Count predicted events The model predicts I, like, tea and the end marker: four predictions. **Final answer.** Four trigrams. Start markers are context; they are not ordinary predicted words. **Question. Give the bag-of-bigrams counts for “a b a b” without padding.** #### Given Four tokens in their given order. #### To find Bigram positions, distinct bigram types and their counts. #### Formula and why it applies $$ \#\text{positions}=4-2+1=3 $$ A bag stores frequencies; repeated bigrams must not be discarded. #### Solution ##### Step 1. Enumerate all positions (a,b), (b,a), (a,b). ##### Step 2. Group identical pairs | Bigram type | Count | | --- | --- | | a b | 2 | | b a | 1 | **Final answer.** Three bigram positions but two distinct bigram types. **Yaad rakho:** n-gram mein n words; context mein n−1 words. --- ## TEE · Unigram estimation and complete worked examples Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/formulas/f-counts--unigram-estimation/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/formulas/f-counts/unigram-estimation Formulas explained through full worked examples **Maximum likelihood estimation (MLE)** chooses probabilities equal to observed relative frequencies. In a unigram model, every word is predicted independently of its surrounding words. If a word occurs c times in N training tokens, its estimate is c/N. To score m specified words, multiply their m unigram probabilities. $$ P_{\mathrm{MLE}}(w)=\frac{C(w)}{N}\qquad P(w_1,\ldots,w_m)\approx\prod_{i=1}^{m}P(w_i) $$ Choose a tokenization convention before counting. Lowercasing, punctuation handling and boundary markers can change the vocabulary and counts. Never count words with one convention and form the denominator with another. A unigram model gives the same product to rearrangements of the same word multiset; that is one of its limitations. **Question. Train an unpadded, case-insensitive unigram model on “I like tea” and “I like coffee”. Find the probability of “I like tea”.** #### Given Training sentences: I like tea; I like coffee. Use lowercase words; exclude punctuation and sentence boundaries. #### To find The vocabulary, word probabilities and test-sentence probability. #### Formula and why it applies $$ P(w)=C(w)/N,\qquad P(I\ like\ tea)=P(I)P(like)P(tea) $$ The unigram approximation ignores order and uses total token count for every denominator. #### Solution ##### Step 1. Count every training token Each sentence contains three words, so N = 3+3 = 6. ##### Step 2. Count types and frequencies | Word | Count | MLE probability | | --- | --- | --- | | i | 2 | 2/6 = 1/3 | | like | 2 | 2/6 = 1/3 | | tea | 1 | 1/6 | | coffee | 1 | 1/6 | Vocabulary size V = 4. ##### Step 3. Write one factor per test word $$ P=\frac{2}{6}\times\frac{2}{6}\times\frac{1}{6} $$ ##### Step 4. Multiply the fractions $$ P=\frac{4}{216}=\frac{1}{54}\approx0.018519 $$ **Final answer.** The unigram sentence probability is approximately 0.018519. **Question. A corpus contains counts a: 4, b: 3 and c: 1. Find the unigram probability of “a b a”.** #### Given C(a)=4, C(b)=3, C(c)=1. No boundary markers. #### To find P(a b a). #### Formula and why it applies $$ P(a\,b\,a)=P(a)P(b)P(a) $$ A repeated test word contributes a probability factor at every occurrence; do not include it just once. #### Solution ##### Step 1. Find the total $$ N=4+3+1=8 $$ ##### Step 2. Estimate the required probabilities $$ P(a)=4/8=1/2,\qquad P(b)=3/8 $$ ##### Step 3. Use both occurrences of a $$ P=\frac12\times\frac38\times\frac12 $$ ##### Step 4. Simplify $$ P=\frac3{32}=0.09375 $$ **Final answer.** P(a b a) = 0.09375. **Yaad rakho:** Repeated word ka probability bhi repeat multiply hoga. **Question. An unsmoothed unigram model is trained on “red blue red”. Find the probability of “red green”.** #### Given N=3. Counts: red=2, blue=1, green=0. The test contains red and green. #### To find The unsmoothed test probability and the reason for its value. #### Formula and why it applies $$ P(red\ green)=P(red)P(green) $$ MLE assigns zero to an unseen outcome. Smoothing is needed if an unseen in-vocabulary item must receive probability. #### Solution ##### Step 1. Estimate red $$ P(red)=2/3 $$ ##### Step 2. Estimate the unseen word $$ P(green)=0/3=0 $$ ##### Step 3. Multiply $$ (2/3)\times0=0 $$ **Final answer.** Unsmoothed probability = 0. This reflects missing training evidence, not proof that the phrase is impossible. --- ## TEE · Bigram counts: build the table before multiplying Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/formulas/f-counts--bigram-count-table/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/formulas/f-counts/bigram-count-table Formulas explained through full worked examples A **bigram probability table** uses the previous word as its row and the next word as its column. Divide each cell count by its row's total outgoing count. With one end marker on every sentence, each ordinary training word has a continuation, so its outgoing count equals its token frequency. Do not create a bigram that crosses from one sentence into the next. $$ P(w\mid h)=\frac{C(h,w)}{\sum_{v\in V}C(h,v)}=\frac{C(h,w)}{C(h)} $$ Add boundaries to each sentence separately → List adjacent pairs → Count repeated pairs → Divide each row by its total → Follow the test sentence through the table **Question. Train a bigram model on “I like tea”, “I like coffee” and “you like tea”. Use one start and one end marker. Find the probability of “I like tea”.** #### Given Corpus: I like tea ; I like coffee ; you like tea . Preserve these tokens. #### To find All nonzero bigram estimates and the test probability. #### Formula and why it applies $$ P(w\mid h)=C(h,w)/C(h) $$ Every sentence starts and ends independently. Four probability factors are required for the three test words plus end. #### Solution ##### Step 1. List the training pairs Sentence 1: (,I), (I,like), (like,tea), (tea,). Sentence 2: (,I), (I,like), (like,coffee), (coffee,). Sentence 3: (,you), (you,like), (like,tea), (tea,). ##### Step 2. Create the nonzero count table | History → next | Pair count | History count | Probability | | --- | --- | --- | --- | | → I | 2 | 3 | 2/3 | | → you | 1 | 3 | 1/3 | | I → like | 2 | 2 | 1 | | you → like | 1 | 1 | 1 | | like → tea | 2 | 3 | 2/3 | | like → coffee | 1 | 3 | 1/3 | | tea → | 2 | 2 | 1 | | coffee → | 1 | 1 | 1 | ##### Step 3. Write the test path $$ P=P(I\mid\langle s\rangle)P(like\mid I)P(tea\mid like)P(\langle/s\rangle\mid tea) $$ ##### Step 4. Substitute each matching row $$ P=\frac23\times1\times\frac23\times1 $$ ##### Step 5. Multiply $$ P=\frac49\approx0.444444 $$ **Final answer.** P(I like tea) = 4/9 ≈ 0.444444. **Question. Using the same three-sentence corpus given here—“I like tea”, “I like coffee”, “you like tea”—find the bigram probabilities of “you like coffee” and “tea like I”. Include boundaries.** #### Given Corpus with one start/end marker per sentence. Counts: start→you=1/3; you→like=1; like→coffee=1/3; coffee→end=1. No training sentence starts with tea. #### To find The two test probabilities. #### Formula and why it applies $$ P(\text{sentence})=\prod P(\text{next token}\mid\text{previous token}) $$ A model can assign a nonzero probability to a sentence it never saw if all of its local bigrams were seen. #### Solution ##### Step 1. Score you like coffee $$ P=\frac13\times1\times\frac13\times1=\frac19\approx0.111111 $$ ##### Step 2. Inspect tea like I $$ P(tea\mid\langle s\rangle)=0/3=0 $$ ##### Step 3. Apply the zero factor The full product is zero, regardless of the remaining factors. **Final answer.** P(you like coffee) = 1/9; P(tea like I) = 0. A new sentence is not necessarily a zero-probability sentence. **Question. Obtain the unsmoothed bigram probability of “students are from Vellore” from this corpus: “I am from Vellore”; “I am a teacher”; “students are good and are from various cities”; “students from Vellore do engineering”. Use sentence boundaries.** #### Given Four training sentences, each padded with one and . Test sequence: students are from Vellore . #### To find The five required conditional probabilities and their product. #### Formula and why it applies $$ P(w\mid h)=C(h,w)/C(h) $$ Count all occurrences in the entire corpus. In particular, are occurs twice inside the third sentence, and Vellore is followed by end once and do once. #### Solution ##### Step 1. Count the histories | History | Occurrences with continuation | | --- | --- | | | 4 | | students | 2 | | are | 2 | | from | 3 | | Vellore | 2 | ##### Step 2. Count the required pairs | Required pair | Count | Probability | | --- | --- | --- | | students | 2 | 2/4 | | students are | 1 | 1/2 | | are from | 1 | 1/2 | | from Vellore | 2 | 2/3 | | Vellore | 1 | 1/2 | ##### Step 3. Write the complete product $$ P=\frac24\times\frac12\times\frac12\times\frac23\times\frac12 $$ ##### Step 4. Multiply progressively $$ \frac24\times\frac12=\frac14,\quad \frac14\times\frac12=\frac18 $$ $$ \frac18\times\frac23=\frac1{12},\quad \frac1{12}\times\frac12=\frac1{24} $$ **Final answer.** P = 1/24 ≈ 0.0416667. **Yaad rakho:** Same sentence mein word do baar aaye toh count bhi do. --- ## TEE · Trigram estimates: two-word history, not two denominators Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/formulas/f-counts--trigram-estimation/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/formulas/f-counts/trigram-estimation Formulas explained through full worked examples A trigram model predicts the next word from the two preceding words. The numerator is the count of a triple. The denominator is the count of its two-word history with an available next token. Do not multiply two independent bigram probabilities and call that a trigram probability. $$ P(w_i\mid w_{i-2},w_{i-1})=\frac{C(w_{i-2},w_{i-1},w_i)}{C(w_{i-2},w_{i-1})} $$ **Question. A corpus contains 12 occurrences of “prime minister”. Eight are followed by “of”, three by “said”, and one by “visited”. Estimate the three trigram probabilities.** #### Given History h = (prime, minister). Count of this history = 12. Triple counts for of, said, visited are 8, 3, 1. #### To find The conditional distribution of the next word. #### Formula and why it applies $$ P(w\mid prime,minister)=C(prime,minister,w)/12 $$ All three candidates share one two-word history, so they share denominator 12. #### Solution ##### Step 1. Calculate of $$ P(of\mid prime,minister)=8/12=2/3 $$ ##### Step 2. Calculate said $$ P(said\mid prime,minister)=3/12=1/4 $$ ##### Step 3. Calculate visited $$ P(visited\mid prime,minister)=1/12 $$ ##### Step 4. Confirm normalization $$ 8/12+3/12+1/12=12/12=1 $$ **Final answer.** The probabilities are 2/3, 1/4 and 1/12. **Question. Train a trigram model on “I like tea”, “I like coffee” and “you like tea”. Use two start markers and one end marker. Score “I like tea”.** #### Given Three sentences. Padded first sentence is I like tea ; pad the other two in the same way. #### To find Four trigram prediction probabilities and the sentence probability. #### Formula and why it applies $$ P=\prod_i P(w_i\mid w_{i-2},w_{i-1}) $$ Each prediction uses a two-token context; the two start markers supply the first history. #### Solution ##### Step 1. Count the first-word event The history (,) occurs three times, and two are followed by I. $$ P(I\mid\langle s\rangle,\langle s\rangle)=2/3 $$ ##### Step 2. Count the second-word event (,I) occurs twice; both are followed by like. $$ P(like\mid\langle s\rangle,I)=2/2=1 $$ ##### Step 3. Count the third-word event (I,like) occurs twice; one is followed by tea. $$ P(tea\mid I,like)=1/2 $$ ##### Step 4. Count the end event (like,tea) occurs twice; both are followed by end. $$ P(\langle/s\rangle\mid like,tea)=2/2=1 $$ ##### Step 5. Multiply all four $$ P=\frac23\times1\times\frac12\times1=\frac13 $$ **Final answer.** Trigram P(I like tea) = 1/3. It differs from the bigram estimate because the context and denominator changed. **Question. A trigram model supplies P(we|,)=0.4, P(study|,we)=0.5, P(NLP|we,study)=0.8 and P(|study,NLP)=0.25. Score “we study NLP”.** #### Given Four trigram conditional factors. The end marker is included. #### To find The complete sentence probability. #### Formula and why it applies $$ P=P(we\mid s,s)P(study\mid s,we)P(NLP\mid we,study)P(end\mid study,NLP) $$ Use each supplied two-word history exactly; do not swap its word order. #### Solution ##### Step 1. Insert the values $$ P=0.4\times0.5\times0.8\times0.25 $$ ##### Step 2. Multiply the first pair $$ 0.4\times0.5=0.2 $$ ##### Step 3. Include NLP $$ 0.2\times0.8=0.16 $$ ##### Step 4. Include the end $$ 0.16\times0.25=0.04 $$ **Final answer.** P = 0.04. **Yaad rakho:** Trigram: teen-word count upar, pehle do-word count neeche. **Why larger n is not automatically better.** More context can capture local order more precisely, but each longer history occurs less often. Parameter count, storage and sparsity grow. A model trained on Shakespeare may assign poor probabilities to financial news because training and test distributions differ. Keep a held-out test set; do not evaluate only on memorized training sentences. --- ## TEE · Laplace smoothing: why both numerator and denominator change Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/formulas/f-smoothing--laplace-basics/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/formulas/f-smoothing/laplace-basics Formulas explained through full worked examples **Smoothing** reallocates some probability mass to events that received zero or unreliable estimates from finite training data. **Laplace, or add-one, smoothing** adds one pseudo-count to every possible next outcome. If there are V possible outcomes, the total added count is V, so adding 1 only to the numerator would produce probabilities that do not sum to 1. $$ P_{\mathrm{Laplace}}(w)=\frac{C(w)+1}{N+V} $$ $$ P_{\mathrm{Laplace}}(w\mid h)=\frac{C(h,w)+1}{C(h)+V} $$ For unigram smoothing, N is the total training token count. For bigram smoothing, C(h) is the count of the particular history. In both cases V is the number of outcomes being predicted, not the number of training sentences and not the number of distinct bigrams. When end is a possible prediction, include it in V; the start marker is usually context only. Observed counts: 3, 1, 0 → Add one to every outcome: 4, 2, 1 → New total: 7 → Smoothed probabilities: 4/7, 2/7, 1/7 **Question. A fixed unigram vocabulary is {a,b,c}. The training counts are 3,1,0. Compute unsmoothed and Laplace-smoothed probabilities.** #### Given C(a)=3, C(b)=1, C(c)=0; N=4; V=3. #### To find The complete distributions. #### Formula and why it applies $$ P(w)=C(w)/N,\qquad P_L(w)=(C(w)+1)/(N+V) $$ The zero-count c is still in the fixed vocabulary, so it must receive one pseudo-count and contribute to V. #### Solution ##### Step 1. Compute unsmoothed values | Word | Unsmoothed | | --- | --- | | a | 3/4 = 0.75 | | b | 1/4 = 0.25 | | c | 0/4 = 0 | ##### Step 2. Calculate the new denominator $$ N+V=4+3=7 $$ ##### Step 3. Add one to each numerator $$ C(a)+1=4,\quad C(b)+1=2,\quad C(c)+1=1 $$ ##### Step 4. Divide | Word | Smoothed | | --- | --- | | a | 4/7 ≈ 0.571429 | | b | 2/7 ≈ 0.285714 | | c | 1/7 ≈ 0.142857 | ##### Step 5. Confirm the total $$ 4/7+2/7+1/7=1 $$ **Final answer.** Smoothed probabilities: a=4/7, b=2/7, c=1/7. Common a loses mass; unseen c gains mass. **Question. After history I, a corpus records like three times, study once and sleep zero times. The allowed next vocabulary is {like,study,sleep}. Compute all Laplace-smoothed bigram probabilities.** #### Given C(I,like)=3; C(I,study)=1; C(I,sleep)=0. History count C(I)=4. V=3. #### To find P\_L(like|I), P\_L(study|I), P\_L(sleep|I). #### Formula and why it applies $$ P_L(w\mid I)=\frac{C(I,w)+1}{C(I)+V} $$ This is smoothing within the I row; use its total 4, not the global corpus token count. #### Solution ##### Step 1. Form the common denominator $$ C(I)+V=4+3=7 $$ ##### Step 2. Calculate like $$ P_L(like\mid I)=(3+1)/7=4/7 $$ ##### Step 3. Calculate study $$ P_L(study\mid I)=(1+1)/7=2/7 $$ ##### Step 4. Calculate sleep $$ P_L(sleep\mid I)=(0+1)/7=1/7 $$ **Final answer.** The row is (4/7, 2/7, 1/7). An unseen bigram is no longer forced to zero. **Question. A fixed vocabulary has five possible next tokens. A history h is unseen in training. What is its add-one probability for each next token?** #### Given C(h)=0; C(h,w)=0 for each outcome; V=5. #### To find P\_L(w|h) for each token. #### Formula and why it applies $$ P_L(w\mid h)=\frac{0+1}{0+V} $$ All outcomes receive equal pseudo-counts because the training data gives no preference for this history. #### Solution ##### Step 1. Add the pseudo-count $$ 0+1=1 $$ ##### Step 2. Form the denominator $$ 0+5=5 $$ ##### Step 3. Divide and sum $$ P_L(w\mid h)=1/5=0.2,\qquad5(0.2)=1 $$ **Final answer.** Every outcome gets 0.2 under this fixed-vocabulary model. **Vocabulary warning.** Add-one does not automatically define probabilities for every word in the world. Unknown words require a declared vocabulary/UNK convention. If a question supplies V, use that stated V rather than inventing additional tokens. **Yaad rakho:** Add-one: upar +1, neeche +V; bigram mein neeche history ka count. --- ## TEE · A complete unseen-bigram question from corpus to final answer Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/formulas/f-smoothing--smoothed-sentence/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/formulas/f-smoothing/smoothed-sentence Formulas explained through full worked examples **Question. Consider the corpus “I love to code in Python. I love to learn new things. I code every day.” Obtain the bigram probabilities needed for “I love Python”. Explain the zero-probability issue, then use Laplace smoothing to score it. Treat each sentence separately, preserve case, remove punctuation, and include an end token among predicted outcomes.** #### Given Three sentences padded with one start/end marker. Distinct words: I, love, to, code, in, Python, learn, new, things, every, day. With , V=12. Start is context only. #### To find Unsmoothed and smoothed P(I love Python | ). #### Formula and why it applies $$ P(w\mid h)=\frac{C(h,w)}{C(h)},\qquad P_L(w\mid h)=\frac{C(h,w)+1}{C(h)+12} $$ The test contains an unseen transition love→Python. Smoothing assigns probability to that transition and renormalizes every required history row. #### Solution ##### Step 1. Count the contexts | History | Count | | --- | --- | | | 3 | | I | 3 | | love | 2 | | Python | 1 | ##### Step 2. Count each required pair | Pair | Count | | --- | --- | | I | 3 | | I love | 2 | | love Python | 0 | | Python | 1 | ##### Step 3. Compute the unsmoothed product $$ P=\frac33\times\frac23\times\frac02\times\frac11=0 $$ Both words love and Python were seen, but their adjacent pair was not. This is an unseen bigram, not an unknown-word case. ##### Step 4. Smooth the first event $$ P_L(I\mid\langle s\rangle)=\frac{3+1}{3+12}=\frac4{15} $$ ##### Step 5. Smooth the second event $$ P_L(love\mid I)=\frac{2+1}{3+12}=\frac3{15} $$ ##### Step 6. Smooth the unseen event $$ P_L(Python\mid love)=\frac{0+1}{2+12}=\frac1{14} $$ ##### Step 7. Smooth the end event $$ P_L(\langle/s\rangle\mid Python)=\frac{1+1}{1+12}=\frac2{13} $$ ##### Step 8. Multiply the smoothed factors $$ P_L=\frac4{15}\times\frac3{15}\times\frac1{14}\times\frac2{13} $$ $$ P_L=\frac{24}{40950}=\frac4{6825}\approx0.000586081 $$ **Final answer.** Unsmoothed P=0. With the stated V=12 and boundaries, Laplace-smoothed P≈0.000586081. If an exam states a different convention—for example excluding boundary tokens from the entire calculation—recompute both V and the set of probability factors consistently. Do not combine a boundary-aware numerator with a boundary-free denominator. --- ## TEE · Adjusted counts and relative discounting Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/formulas/f-smoothing--adjusted-count-discount/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/formulas/f-smoothing/adjusted-count-discount Formulas explained through full worked examples The slides also express smoothing through an **adjusted count** c\*. Instead of writing the smoothed probability with denominator N+V, they rescale its numerator so the denominator can remain N. Algebraically these are equivalent: $$ c^*=(c+1)\frac{N}{N+V},\qquad P_L(w)=\frac{c^*}{N} $$ $$ d=\frac{c^*}{c}\quad(c>0) $$ The relative discount d compares adjusted and original counts. It is undefined when c=0 because division by zero is undefined. Add-one smoothing moves the distribution toward uniformity: frequent events are reduced, but some rare seen events may increase. It is not correct to claim that every nonzero count must decrease under Laplace smoothing. **Question. For N=20 and V=5, a word has original count c=10. Find its adjusted count, smoothed probability and relative discount.** #### Given N=20; V=5; c=10. #### To find c\*, P\_L and d. #### Formula and why it applies $$ c^*=(c+1)N/(N+V),\quad P_L=c^*/N,\quad d=c^*/c $$ These expressions rewrite add-one smoothing without changing its probability. #### Solution ##### Step 1. Form the new total $$ N+V=20+5=25 $$ ##### Step 2. Find the adjusted count $$ c^*=(10+1)\frac{20}{25}=11(0.8)=8.8 $$ ##### Step 3. Find the probability $$ P_L=8.8/20=0.44 $$ ##### Step 4. Find the relative discount $$ d=8.8/10=0.88 $$ **Final answer.** c\*=8.8, P\_L=0.44, d=0.88. The original probability 0.5 is reduced. **Question. Under N=20, V=5, a rare word occurs once. Compute its adjusted count and compare the original and smoothed probabilities.** #### Given N=20; V=5; c=1. #### To find c\*, original P and smoothed P\_L. #### Formula and why it applies $$ c^*=(c+1)N/(N+V) $$ The same smoothing rule may increase a rare event's probability. #### Solution ##### Step 1. Original probability $$ P=1/20=0.05 $$ ##### Step 2. Adjusted count $$ c^*=(1+1)(20/25)=1.6 $$ ##### Step 3. Smoothed probability $$ P_L=1.6/20=0.08 $$ **Final answer.** c\*=1.6 and P\_L=0.08, which exceeds the original 0.05. **Question. For N=20 and V=5, an in-vocabulary word has count zero. Find its adjusted count and smoothed probability. Can a relative discount be calculated?** #### Given c=0; N=20; V=5. #### To find c\*, P\_L and whether d exists. #### Formula and why it applies $$ c^*=(c+1)N/(N+V),\qquad P_L=c^*/N $$ The pseudo-count creates positive mass for the unseen vocabulary item. #### Solution ##### Step 1. Adjusted count $$ c^*=(0+1)(20/25)=0.8 $$ ##### Step 2. Smoothed probability $$ P_L=0.8/20=0.04 $$ ##### Step 3. Relative discount d would be 0.8/0. That is undefined, so do not report a finite discount. **Final answer.** c\*=0.8 and P\_L=0.04; relative discount is undefined for c=0. --- ## TEE · Perplexity: meaning, formula and four complete examples Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/formulas/f-smoothing--evaluation-perplexity/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/formulas/f-smoothing/evaluation-perplexity Formulas explained through full worked examples **Intrinsic evaluation** evaluates a model directly on a held-out test corpus, for example using perplexity. **Extrinsic evaluation** places it inside a downstream application, such as a speech recognizer or translation system, and measures the application's performance. Extrinsic testing is closer to actual use but often slower and more expensive. A lower perplexity is not a guarantee of improvement on every downstream task. **Perplexity** is the inverse probability of a test sequence, normalized by the number of prediction events. Intuitively, it is the model's effective average number of equally plausible choices. A model that always gives the correct next word probability 1/4 has perplexity 4. Lower is better when the models are tested on the same data, vocabulary, tokenization and boundary convention. $$ \operatorname{PP}(W)=P(W)^{-1/N}=\sqrt[N]{\frac1{P(W)}} $$ | Symbol | Meaning | | --- | --- | | W | The complete test sequence. | | P(W) | Product of the probabilities assigned to the actual next tokens. | | N | Number of scored prediction events under the stated convention. | | Exponent −1/N | Take the reciprocal, then the Nth root. | For calculator entry, use parentheses: P raised to (−1 ÷ N), or (1 ÷ P) raised to (1 ÷ N). Retain several digits in intermediate products. The ordinary start marker is usually given context, not a predicted event; the end marker counts if its probability is included. **Question. A model assigns probability 0.0625 to a test sequence containing four scored tokens. Compute its perplexity.** #### Given P(W)=0.0625=1/16; N=4. #### To find PP(W). #### Formula and why it applies $$ \operatorname{PP}(W)=\left(\frac1{P(W)}\right)^{1/N} $$ The root removes the effect of sequence length; the reciprocal makes lower surprise correspond to lower perplexity. #### Solution ##### Step 1. Take the reciprocal $$ 1/P(W)=1/0.0625=16 $$ ##### Step 2. Use the fourth root $$ \operatorname{PP}=16^{1/4} $$ ##### Step 3. Evaluate $$ 2^4=16\quad\Rightarrow\quad\operatorname{PP}=2 $$ **Final answer.** Perplexity = 2. **Question. Two language models score the same three-token test sequence. Model A gives next-token probabilities 0.5, 0.5, 0.5. Model B gives 0.5, 0.25, 0.25. Compute their perplexities and select the better intrinsic result.** #### Given Same sequence and N=3 for both models. A factors: 0.5,0.5,0.5. B factors: 0.5,0.25,0.25. #### To find Both sequence probabilities and perplexities. #### Formula and why it applies $$ P(W)=\prod_i p_i,\qquad\operatorname{PP}=P(W)^{-1/3} $$ A fair comparison uses identical test tokens and scoring conventions. #### Solution ##### Step 1. Calculate A's probability $$ P_A=0.5\times0.5\times0.5=0.125=1/8 $$ ##### Step 2. Calculate A's perplexity $$ \operatorname{PP}_A=8^{1/3}=2 $$ ##### Step 3. Calculate B's probability $$ P_B=0.5\times0.25\times0.25=0.03125=1/32 $$ ##### Step 4. Calculate B's perplexity $$ \operatorname{PP}_B=32^{1/3}\approx3.174802 $$ ##### Step 5. Compare 2 is lower than 3.174802; A places more probability on the observed sequence. **Final answer.** Model A is better by perplexity on this test: 2 versus about 3.174802. **Question. The bigram training corpus is “This is a dog”, “This is a cat”, “I love my cat”, “This is my name”. Score “This is my cat”, including an end marker, and compute perplexity using five predicted events.** #### Given Each training sentence has one start and end marker. Test predictions: This, is, my, cat, . Thus N=5. #### To find Bigram sentence probability and perplexity. #### Formula and why it applies $$ P(w\mid h)=C(h,w)/C(h),\qquad\operatorname{PP}=P(W)^{-1/5} $$ Start is given context. Four words plus the end marker produce five probability factors. #### Solution ##### Step 1. Count the required transitions | Transition | Pair count / history count | | --- | --- | | →This | 3/4 | | This→is | 3/3 | | is→my | 1/3 | | my→cat | 1/2 | | cat→ | 2/2 | ##### Step 2. Form the product $$ P=\frac34\times1\times\frac13\times\frac12\times1=\frac18 $$ ##### Step 3. Take the reciprocal $$ 1/P=8 $$ ##### Step 4. Take the fifth root $$ \operatorname{PP}=8^{1/5}\approx1.515717 $$ **Final answer.** P(W)=1/8 and perplexity≈1.515717 using five predicted events. **If the question explicitly states N = 6:** use that supplied convention and report PP = 81/6 ≈ 1.414214. The class slide counts both boundary markers in N. Write the chosen N beside the formula; do not silently switch between 5 and 6. Also, This occurs three times and is follows it three times, so P(is|This)=3/3, not 3/4. **Question. A uniform model assigns probability 1/10 to each actual next word in a five-token test. Find its perplexity.** #### Given Five scored tokens, each with assigned probability 0.1. N=5. #### To find The sentence probability and perplexity. #### Formula and why it applies $$ P(W)=(1/10)^5,\qquad \operatorname{PP}=P(W)^{-1/5} $$ Uniform equal probabilities make the effective number of choices especially clear. #### Solution ##### Step 1. Multiply the five factors $$ P(W)=0.1^5=0.00001=10^{-5} $$ ##### Step 2. Invert $$ 1/P(W)=10^5 $$ ##### Step 3. Take the fifth root $$ \operatorname{PP}=(10^5)^{1/5}=10 $$ **Final answer.** Perplexity = 10. **Yaad rakho:** Probability high → surprise low → perplexity low. --- ## TEE · Log probabilities: replace long products with sums Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/formulas/f-smoothing--log-probabilities/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/formulas/f-smoothing/log-probabilities Formulas explained through full worked examples Probabilities are at most 1, so multiplying many of them produces very small numbers. Computers can round extremely small positive products to zero: **numerical underflow**. Log probabilities convert multiplication into addition and are more stable for long sequences. Natural logs use base e; log₂ uses base 2. Choose one base consistently. $$ \ln P(W)=\sum_{i=1}^{N}\ln p_i $$ $$ P(W)=e^{\ln P(W)},\qquad \operatorname{PP}(W)=\exp\left(-\frac1N\sum_i\ln p_i\right) $$ The log of a probability below 1 is negative. A value closer to zero means a larger probability: ln(0.5) ≈ −0.693 is larger than ln(0.1) ≈ −2.303. A zero probability has log −∞; taking logs does not solve the unseen-event problem. Smoothing handles that separate problem. **Question. The next-word probabilities are 0.5, 0.25 and 0.5. Calculate the natural log probability, ordinary probability and perplexity.** #### Given p₁=0.5; p₂=0.25; p₃=0.5; N=3. #### To find ln P(W), P(W), and PP. #### Formula and why it applies $$ \ln P=\sum_i\ln p_i,\quad P=e^{\ln P},\quad\operatorname{PP}=e^{-\ln P/N} $$ Logarithms preserve products as sums and the exponent converts the result back. #### Solution ##### Step 1. Compute individual logs $$ \ln0.5\approx-0.693147,\quad\ln0.25\approx-1.386294 $$ ##### Step 2. Add all three terms $$ \ln P=-0.693147-1.386294-0.693147\approx-2.772589 $$ ##### Step 3. Recover the probability $$ P=e^{-2.772589}\approx0.0625 $$ ##### Step 4. Find average negative log probability $$ -\ln P/3\approx2.772589/3=0.924196 $$ ##### Step 5. Exponentiate $$ \operatorname{PP}=e^{0.924196}\approx2.519842 $$ **Final answer.** ln P≈−2.772589, P=0.0625, PP≈2.519842. **Question. A model has a total natural log probability −12 over six scored tokens. Calculate its perplexity.** #### Given ln P(W)=−12; N=6. #### To find PP without forming the tiny ordinary probability. #### Formula and why it applies $$ \operatorname{PP}=\exp(-\ln P(W)/N) $$ The logarithmic formula directly uses the supplied log score. #### Solution ##### Step 1. Negate the log score $$ -\ln P=12 $$ ##### Step 2. Divide by token count $$ 12/6=2 $$ ##### Step 3. Apply exponential $$ \operatorname{PP}=e^2\approx7.389056 $$ **Final answer.** Perplexity≈7.389056. **Question. A language model's average negative log₂ probability is 3 bits per token. Find its perplexity.** #### Given Cross-entropy H=3 bits/token. Log base is 2. #### To find PP. #### Formula and why it applies $$ \operatorname{PP}=2^H $$ The inverse operation of log₂ is raising 2 to that power; use e only for natural logs. #### Solution ##### Step 1. Identify the log base Bits mean base 2. ##### Step 2. Raise the base to H $$ \operatorname{PP}=2^3=8 $$ **Final answer.** Perplexity = 8. **Yaad rakho:** ln use kiya toh e se wapas; log₂ use kiya toh 2 se wapas. --- ## TEE · Markov chains: read a transition table and multiply a path Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/formulas/f-hmm--markov-chain/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/formulas/f-hmm/markov-chain Formulas explained through full worked examples **A Markov chain** is a probabilistic model of a sequence of states. Under the first-order Markov assumption, the next state's probability depends on the current state rather than the complete previous history. The **initial probability** tells us how likely each first state is. A **transition probability** tells us how likely a move from one state to another is. $$ P(s_t\mid s_1,\ldots,s_{t-1})=P(s_t\mid s_{t-1}) $$ $$ P(s_1,\ldots,s_T)=P(s_1)\prod_{t=2}^{T}P(s_t\mid s_{t-1}) $$ For T states in a path, use one initial factor and T−1 transition factors. A row-oriented transition matrix must have each complete row sum to 1. Always read the row/column labels; some exam diagrams use a transposed arrangement. | Current → next | Rain | Dry | | --- | --- | --- | | Rain | 0.3 | 0.7 | | Dry | 0.2 | 0.8 | Initial probabilities: P(Rain)=0.4, P(Dry)=0.6. Rows describe the current state; columns describe the next state. **Question. For the Rain/Dry Markov chain with initial P(Rain)=0.4 and P(Dry)=0.6, and transitions Rain→Rain=0.3, Rain→Dry=0.7, Dry→Rain=0.2, Dry→Dry=0.8, find the probability of Dry, Dry, Rain, Rain.** #### Given | Current → next | Rain | Dry | | --- | --- | --- | | Rain | 0.3 | 0.7 | | Dry | 0.2 | 0.8 | Initial probabilities: P(Rain)=0.4, P(Dry)=0.6. Rows describe the current state; columns describe the next state. #### To find P(Dry,Dry,Rain,Rain). #### Formula and why it applies $$ P(D,D,R,R)=P(D)P(D\mid D)P(R\mid D)P(R\mid R) $$ Each step conditions on the immediately preceding state. The sequence contains four states, so it needs one initial plus three transition factors. #### Solution ##### Step 1. Start in Dry $$ P(D)=0.6 $$ ##### Step 2. Move Dry to Dry $$ 0.6\times0.8=0.48 $$ ##### Step 3. Move Dry to Rain $$ 0.48\times0.2=0.096 $$ ##### Step 4. Move Rain to Rain $$ 0.096\times0.3=0.0288 $$ **Final answer.** Path probability = 0.0288. **Question. For initial probabilities Rain=0.4, Dry=0.6 and transition rows Rain=(0.3,0.7), Dry=(0.2,0.8) in column order Rain,Dry, find the probability of Rain, Dry, Rain.** #### Given | Current → next | Rain | Dry | | --- | --- | --- | | Rain | 0.3 | 0.7 | | Dry | 0.2 | 0.8 | Initial probabilities: P(Rain)=0.4, P(Dry)=0.6. Rows describe the current state; columns describe the next state. #### To find P(Rain,Dry,Rain). #### Formula and why it applies $$ P(R,D,R)=P(R)P(D\mid R)P(R\mid D) $$ Do not replace Dry→Rain with Rain→Dry: the two directional probabilities differ. #### Solution ##### Step 1. Initial Rain $$ P(R)=0.4 $$ ##### Step 2. First transition $$ P(D\mid R)=0.7,\quad0.4(0.7)=0.28 $$ ##### Step 3. Second transition $$ P(R\mid D)=0.2,\quad0.28(0.2)=0.056 $$ **Final answer.** Path probability = 0.056. **Question. For the same fully specified chain—initial Rain=0.4, Dry=0.6; Rain→Rain=0.3, Rain→Dry=0.7, Dry→Rain=0.2, Dry→Dry=0.8—find P(Dry,Dry). Also find the conditional probability of next Dry given current Dry.** #### Given | Current → next | Rain | Dry | | --- | --- | --- | | Rain | 0.3 | 0.7 | | Dry | 0.2 | 0.8 | Initial probabilities: P(Rain)=0.4, P(Dry)=0.6. Rows describe the current state; columns describe the next state. #### To find The two-state joint probability and the one-step conditional probability. #### Formula and why it applies $$ P(D,D)=P(D)P(D\mid D) $$ The joint path includes an uncertain initial state. The conditional transition assumes the current state is already Dry. #### Solution ##### Step 1. Read the conditional transition $$ P(D\mid D)=0.8 $$ ##### Step 2. Include the initial state for the joint path $$ P(D,D)=0.6\times0.8=0.48 $$ **Final answer.** P(Dry,Dry)=0.48; P(next Dry|current Dry)=0.8. These answer different questions. **Yaad rakho:** Path: pehle start probability, phir har arrow ki probability. --- ## TEE · Estimate transition and emission probabilities from tagged data Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/formulas/f-hmm--transition-emission-estimation/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/formulas/f-hmm/transition-emission-estimation Formulas explained through full worked examples $$ P(t'\mid t)=\frac{C(t,t')}{C(t)\text{ with a following tag/end event}} $$ $$ P(w\mid t)=\frac{C(w,t)}{C(t)} $$ In a transition denominator, count the previous tag's outgoing events. In an emission denominator, count all tokens bearing that tag. With explicit end transitions the totals align conveniently. Keep the two numerator types separate: tag→tag counts are not word→tag counts. **Question. A tagged corpus contains 116454 determiner tags DT, each counted with an outgoing event. DT is followed by NN 56509 times. Estimate P(NN|DT).** #### Given C(DT)=116454; C(DT,NN)=56509. #### To find The DT→NN transition probability. #### Formula and why it applies $$ P(NN\mid DT)=C(DT,NN)/C(DT) $$ The denominator is the number of previous DT contexts because DT is the condition. #### Solution ##### Step 1. Identify the transition count DT followed by NN occurs 56509 times. ##### Step 2. Identify the conditional total There are 116454 DT contexts. ##### Step 3. Divide $$ P(NN\mid DT)=\frac{56509}{116454}\approx0.485247 $$ **Final answer.** P(NN|DT)≈0.485247. **Question. The tag VBZ occurs 21627 times, and 10073 of those tokens are the word “is”. Estimate P(is|VBZ).** #### Given C(VBZ)=21627; C(is,VBZ)=10073. #### To find The emission probability of is from VBZ. #### Formula and why it applies $$ P(is\mid VBZ)=C(is,VBZ)/C(VBZ) $$ The condition is the tag VBZ; we ask which word it emits. #### Solution ##### Step 1. Choose the word-tag count The required word is is with tag VBZ: count 10073. ##### Step 2. Choose the tag total All VBZ-tagged word tokens: count 21627. ##### Step 3. Divide $$ P(is\mid VBZ)=\frac{10073}{21627}\approx0.465760 $$ **Final answer.** P(is|VBZ)≈0.465760. This is different from P(VBZ|is). **Question. Use the tagged sentences “dogs/N chase/V cats/N” and “cats/N sleep/V”. Estimate the initial tag probabilities, all nonzero transitions including END, and emissions.** #### Given Corpus: START dogs/N chase/V cats/N END; START cats/N sleep/V END. #### To find Initial distribution, transition rows and emission rows. #### Formula and why it applies $$ \pi_t=C(START,t)/2,\quad a_{tu}=C(t,u)/C(t),\quad b_t(w)=C(w,t)/C(t) $$ The corpus explicitly tells us the tags, so we can count training frequencies directly. #### Solution ##### Step 1. Count tag tokens N occurs 3 times: dogs, cats, cats. V occurs 2 times: chase, sleep. ##### Step 2. Count sentence starts Both sentences start with N. $$ \pi_N=2/2=1,\quad \pi_V=0/2=0 $$ ##### Step 3. Count transitions | Pair | Count | | --- | --- | | N→V | 2 | | N→END | 1 | | V→N | 1 | | V→END | 1 | ##### Step 4. Normalize transition rows | Current | To N | To V | To END | | --- | --- | --- | --- | | N | 0 | 2/3 | 1/3 | | V | 1/2 | 0 | 1/2 | ##### Step 5. Count and normalize emissions | Tag | dogs | cats | chase | sleep | | --- | --- | --- | --- | --- | | N | 1/3 | 2/3 | 0 | 0 | | V | 0 | 0 | 1/2 | 1/2 | **Final answer.** π=(1,0); transitions and emissions are given in the completed tables. Each complete probability row sums to 1. **Yaad rakho:** Tag→tag table aur tag→word table ko mix mat karna. --- ## TEE · HMM path probabilities: three complete numerical examples Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/formulas/f-hmm--hot-cold-paths/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/formulas/f-hmm/hot-cold-paths Formulas explained through full worked examples The following model uses hidden weather states Hot (H) and Cold (C). Observations are numbers 1, 2 and 3. These numbers are observation labels, not quantities to multiply directly. Their emission probabilities come from the table. | Initial state | Probability | | --- | --- | | H | 0.8 | | C | 0.2 | | From → to | H | C | | --- | --- | --- | | H | 0.7 | 0.3 | | C | 0.4 | 0.6 | | Emission state | Observation 1 | Observation 2 | Observation 3 | | --- | --- | --- | --- | | H | 0.2 | 0.4 | 0.4 | | C | 0.5 | 0.4 | 0.1 | **Question. An HMM has π(H)=0.8, π(C)=0.2; H→H=0.7, H→C=0.3, C→H=0.4, C→C=0.6. Emissions for observations (1,2,3) are H:(0.2,0.4,0.4) and C:(0.5,0.4,0.1). Find the joint probability of hidden path H,C,H and observations 1,3,1.** #### Given Hidden path H→C→H. Observations 1→3→1. Model probabilities are specified in the question. There is no END factor. #### To find P(H,C,H; 1,3,1). #### Formula and why it applies $$ P=\pi_H b_H(1)a_{HC}b_C(3)a_{CH}b_H(1) $$ There are three hidden states and observations: three emissions, two transitions, and one initial probability. #### Solution ##### Step 1. Start Hot and emit 1 $$ 0.8\times0.2=0.16 $$ ##### Step 2. Transition Hot to Cold $$ 0.16\times0.3=0.048 $$ ##### Step 3. Cold emits 3 $$ 0.048\times0.1=0.0048 $$ ##### Step 4. Transition Cold to Hot $$ 0.0048\times0.4=0.00192 $$ ##### Step 5. Hot emits the last 1 $$ 0.00192\times0.2=0.000384 $$ **Final answer.** Joint probability = 0.000384 = 3.84×10⁻⁴. **Question. For π(H)=0.8, π(C)=0.2; H→H=0.7, H→C=0.3, C→H=0.4, C→C=0.6; emissions H:(0.2,0.4,0.4), C:(0.5,0.4,0.1) for labels (1,2,3), find the joint probability of path H,H,H and observations 1,3,1.** #### Given State path H→H→H, observations 1→3→1; no END factor. #### To find P(H,H,H;1,3,1). #### Formula and why it applies $$ P=\pi_H b_H(1)a_{HH}b_H(3)a_{HH}b_H(1) $$ The state remains Hot; each transition still contributes its probability 0.7. #### Solution ##### Step 1. Initial state and first emission $$ 0.8(0.2)=0.16 $$ ##### Step 2. First self-transition $$ 0.16(0.7)=0.112 $$ ##### Step 3. Emit observation 3 $$ 0.112(0.4)=0.0448 $$ ##### Step 4. Second self-transition $$ 0.0448(0.7)=0.03136 $$ ##### Step 5. Emit final observation 1 $$ 0.03136(0.2)=0.006272 $$ **Final answer.** Joint probability = 0.006272. A self-transition is not automatically probability 1. **Question. For π(H)=0.8, π(C)=0.2; transitions H:(0.7,0.3), C:(0.4,0.6) in next-state order H,C; emissions H:(0.2,0.4,0.4), C:(0.5,0.4,0.1) for observations 1,2,3, find the score of path C,C,H with observations 1,3,1.** #### Given Hidden path C→C→H. Observation sequence 1→3→1. No end-state factor. #### To find The joint path-and-observation probability. #### Formula and why it applies $$ P=\pi_C b_C(1)a_{CC}b_C(3)a_{CH}b_H(1) $$ The initial state is Cold, so use π(C), not π(H). Each emission is selected from the state at the same position. #### Solution ##### Step 1. Start Cold and emit 1 $$ 0.2(0.5)=0.1 $$ ##### Step 2. Stay Cold $$ 0.1(0.6)=0.06 $$ ##### Step 3. Emit 3 from Cold $$ 0.06(0.1)=0.006 $$ ##### Step 4. Move Cold to Hot $$ 0.006(0.4)=0.0024 $$ ##### Step 5. Emit 1 from Hot $$ 0.0024(0.2)=0.00048 $$ **Final answer.** Joint probability = 0.00048. **Yaad rakho:** Har column par: state ka emission. Columns ke beech: transition. --- ## TEE · POS HMM worked corpus: Mary, Jane, can, will and spot Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/formulas/f-hmm--tagging-corpus-case/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/formulas/f-hmm/tagging-corpus-case Formulas explained through full worked examples The class example contains words that can be either names or ordinary verbs/modal auxiliaries. The term is **modal** (M), not “model”. To avoid guessing tags from spelling, the training corpus below supplies them explicitly and lowercases words for the emission vocabulary. | Training sentence | Gold tags | | --- | --- | | mary jane can see will | N N M V N | | spot will see mary | N M V N | | will jane spot mary | M N V N | | mary will pat spot | N M V N | There are 9 noun tokens, 4 modal tokens and 4 verb tokens. Each sentence includes a start and end transition. For nouns, the next event can be another noun, modal, verb or END. | Current → next | N | M | V | END | | --- | --- | --- | --- | --- | | START | 3/4 | 1/4 | 0 | 0 | | N | 1/9 | 3/9 | 1/9 | 4/9 | | M | 1/4 | 0 | 3/4 | 0 | | V | 4/4 | 0 | 0 | 0 | | Tag → word | mary | jane | can | see | will | spot | pat | | --- | --- | --- | --- | --- | --- | --- | --- | | N | 4/9 | 2/9 | 0 | 0 | 1/9 | 2/9 | 0 | | M | 0 | 0 | 1/4 | 0 | 3/4 | 0 | 0 | | V | 0 | 0 | 0 | 2/4 | 0 | 1/4 | 1/4 | **Question. Using the fully tagged four-sentence corpus above, calculate the joint score of “will jane spot mary” assigned tags M,N,V,N, including END.** #### Given Required factors from the tables: START→M=1/4; emit will|M=3/4; M→N=1/4; jane|N=2/9; N→V=1/9; spot|V=1/4; V→N=1; mary|N=4/9; N→END=4/9. #### To find The specified tag-path score, including words and sentence end. #### Formula and why it applies $$ P=\pi_M b_M(will)a_{MN}b_N(jane)a_{NV}b_V(spot)a_{VN}b_N(mary)a_{N,END} $$ Both transitions and emissions are needed. A tag transition alone cannot tell us whether a particular word is plausible. #### Solution ##### Step 1. Start and emit will $$ \frac14\times\frac34=\frac3{16} $$ ##### Step 2. Move to N and emit jane $$ \frac3{16}\times\frac14\times\frac29=\frac1{96} $$ ##### Step 3. Move to V and emit spot $$ \frac1{96}\times\frac19\times\frac14=\frac1{3456} $$ ##### Step 4. Move to N and emit mary $$ \frac1{3456}\times1\times\frac49=\frac1{7776} $$ ##### Step 5. End the sentence $$ \frac1{7776}\times\frac49=\frac1{17496}\approx0.0000571559 $$ **Final answer.** The joint score is 1/17496 ≈ 0.0000571559. **Question. For the same tagged corpus and tables, explain the unsmoothed score of “will can spot mary” assigned tags M,V,N,N.** #### Given The proposed second word is can with hidden tag V. In the training corpus, can occurs only with M, so b\_V(can)=0. #### To find Whether the complete path score is zero and what this means. #### Formula and why it applies $$ P(\text{path,words})=(\text{other required factors})\times b_V(can) $$ A product containing one zero factor is zero even if every other factor is positive. #### Solution ##### Step 1. Find the critical emission $$ b_V(can)=C(can,V)/C(V)=0/4=0 $$ ##### Step 2. Use the full-product consequence $$ P(\text{path,words})=0 $$ ##### Step 3. Interpret correctly This unsmoothed corpus model has no evidence for can as a verb. It does not prove that a verb usage of can is impossible in English. **Final answer.** The unsmoothed path score is zero. --- ## TEE · Why exhaustive search grows exponentially Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/formulas/f-hmm--sequence-count/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/formulas/f-hmm/sequence-count Formulas explained through full worked examples If each of T observed words can receive any of N tags, there are N choices at each position. Thus the total number of complete tag sequences is N multiplied by itself T times. This counts possible assignments before grammar rules or zero probabilities eliminate any. $$ \text{Number of tag sequences}=N^T $$ **Question. A four-word sentence is tagged using three possible POS tags per word. How many complete tag sequences are possible?** #### Given N=3 possible tags per position; T=4 word positions. #### To find The number of tag sequences. #### Formula and why it applies $$ M=N^T $$ Every independent position contributes a factor of three choices. #### Solution ##### Step 1. Write one factor per position $$ M=3\times3\times3\times3 $$ ##### Step 2. Multiply $$ 3\times3=9,\quad9\times3=27,\quad27\times3=81 $$ **Final answer.** 81 possible tag sequences. **Question. A five-word sentence has two candidate tags per word. Count its tag sequences.** #### Given N=2; T=5. #### To find M. #### Formula and why it applies $$ M=N^T $$ The length is the exponent, not a multiplier. #### Solution ##### Step 1. Substitute $$ M=2^5 $$ ##### Step 2. Evaluate $$ 2\times2\times2\times2\times2=32 $$ **Final answer.** 32 possible sequences. **Question. Using 45 possible tags, compare the number of tag sequences for sentences of length 2 and 3.** #### Given N=45. First T=2, then T=3. #### To find Both counts and the growth factor. #### Formula and why it applies $$ M_2=45^2,\quad M_3=45^3 $$ Adding one position multiplies the possibilities by the number of tag choices. #### Solution ##### Step 1. Two words $$ 45^2=2025 $$ ##### Step 2. Three words $$ 45^3=2025\times45=91125 $$ ##### Step 3. Compare $$ 91125/2025=45 $$ **Final answer.** 2 words: 2025 paths. 3 words: 91125 paths. One additional word multiplies the count by 45. The slides introduce Viterbi to avoid enumerating every path. It reuses the best prefix score for each ending state. For a dense first-order HMM, it needs O(TN²) recurrence work instead of enumerating NT complete paths. **Yaad rakho:** Choices per word ki power = sentence length. --- ## TEE · Viterbi decoding: compare incoming paths, keep the best, trace back Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/formulas/f-hmm--viterbi-explained/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/formulas/f-hmm/viterbi-explained Formulas explained through full worked examples **The Viterbi algorithm** finds the most likely hidden-state sequence for a fixed observation sequence. At each position, it keeps the best path ending in each state. This is safe because, under the HMM assumptions, future extensions depend on that ending state, not the discarded details of the prefix. $$ \delta_1(j)=\pi_jb_j(o_1) $$ $$ \delta_t(j)=b_j(o_t)\max_i[\delta_{t-1}(i)a_{ij}] $$ $$ \psi_t(j)=\arg\max_i[\delta_{t-1}(i)a_{ij}] $$ | Symbol | Meaning in ordinary words | | --- | --- | | δₜ(j) | Score of the best prefix that ends in state j at position t. | | ψₜ(j) | Previous state that supplied that best prefix; keep it as a backpointer. | | max | Choose the larger candidate score; do not add. | | argmax | Remember which previous state won, rather than only its value. | Initialize: start × first emission → For each next state: compare incoming scores → Multiply winner by this state's emission → Keep score + winning predecessor → Choose best final state and trace arrows backward Do not choose the best state independently at each word. The best last state can come from a different best predecessor than a local guess. Backpointers recover one consistent complete path. If scores tie, either tied path is optimal under the model; state the tie. **Question. Use an HMM with π(H)=0.8, π(C)=0.2, transitions H:(0.7,0.3), C:(0.4,0.6) in H,C order, and emissions H:(0.2,0.4,0.4), C:(0.5,0.4,0.1) for observations 1,2,3. Find the most likely hidden path for observations 1,3,1 using Viterbi.** #### Given Initial probabilities: π(H)=0.8, π(C)=0.2. Transition matrix: H→H=0.7, H→C=0.3, C→H=0.4, C→C=0.6. Emissions for labels (1,2,3): H=(0.2,0.4,0.4), C=(0.5,0.4,0.1). There is no END-state factor. #### To find The best hidden-state sequence and its joint score. #### Formula and why it applies $$ \delta_1(j)=\pi_jb_j(o_1),\quad\delta_t(j)=b_j(o_t)\max_i[\delta_{t-1}(i)a_{ij}] $$ Viterbi keeps two best prefix scores per column, one ending H and one ending C. #### Solution ##### Step 1. Initialize at observation 1 $$ \delta_1(H)=0.8(0.2)=0.16 $$ $$ \delta_1(C)=0.2(0.5)=0.10 $$ ##### Step 2. At observation 3, calculate the H cell $$ H\to H:\ 0.16(0.7)=0.112 $$ $$ C\to H:\ 0.10(0.4)=0.040 $$ $$ \delta_2(H)=\max(0.112,0.040)(0.4)=0.0448 $$ Backpointer for this H cell: H. ##### Step 3. At observation 3, calculate the C cell $$ H\to C:\ 0.16(0.3)=0.048 $$ $$ C\to C:\ 0.10(0.6)=0.060 $$ $$ \delta_2(C)=\max(0.048,0.060)(0.1)=0.006 $$ Backpointer for this C cell: C. ##### Step 4. At final observation 1, calculate H $$ H\to H:\ 0.0448(0.7)=0.03136 $$ $$ C\to H:\ 0.006(0.4)=0.0024 $$ $$ \delta_3(H)=0.03136(0.2)=0.006272 $$ Backpointer: H. ##### Step 5. At final observation 1, calculate C $$ H\to C:\ 0.0448(0.3)=0.01344 $$ $$ C\to C:\ 0.006(0.6)=0.0036 $$ $$ \delta_3(C)=0.01344(0.5)=0.00672 $$ Backpointer: H. ##### Step 6. Select the final state 0.00672 > 0.006272, so the best final state is C. ##### Step 7. Trace back Final C came from H at position2. That H came from H at position1. Reading forward gives H,H,C. ##### Step 8. Verify the recovered complete path $$ 0.8(0.2)(0.7)(0.4)(0.3)(0.5)=0.00672 $$ **Final answer.** Best path H→H→C with joint score 0.00672. **Question. For π(H)=0.8, π(C)=0.2; transitions H:(0.7,0.3), C:(0.4,0.6); emissions H:(0.2,0.4,0.4), C:(0.5,0.4,0.1) for labels1,2,3, decode observations 3,1.** #### Given Initial probabilities: π(H)=0.8, π(C)=0.2. Transition matrix: H→H=0.7, H→C=0.3, C→H=0.4, C→C=0.6. Emissions for labels (1,2,3): H=(0.2,0.4,0.4), C=(0.5,0.4,0.1). There is no END-state factor. #### To find Best two-state hidden path and score. #### Formula and why it applies $$ \delta_1(j)=\pi_jb_j(3),\quad\delta_2(j)=b_j(1)\max_i[\delta_1(i)a_{ij}] $$ The same recurrence applies even though the observation order changed. #### Solution ##### Step 1. Initialize observation 3 $$ \delta_1(H)=0.8(0.4)=0.32,\quad\delta_1(C)=0.2(0.1)=0.02 $$ ##### Step 2. Calculate the final H $$ \max(0.32(0.7),0.02(0.4))(0.2) $$ $$ =\max(0.224,0.008)(0.2)=0.0448 $$ Winning predecessor H. ##### Step 3. Calculate the final C $$ \max(0.32(0.3),0.02(0.6))(0.5) $$ $$ =\max(0.096,0.012)(0.5)=0.048 $$ Winning predecessor H. ##### Step 4. Select and trace Final C wins with 0.048 and came from H. The path is H,C. **Final answer.** Best path H→C, score 0.048. **Question. For π(H)=0.8, π(C)=0.2; transitions H:(0.7,0.3), C:(0.4,0.6); emissions H:(0.2,0.4,0.4), C:(0.5,0.4,0.1) for labels1,2,3, decode observations 2,2.** #### Given Initial probabilities: π(H)=0.8, π(C)=0.2. Transition matrix: H→H=0.7, H→C=0.3, C→H=0.4, C→C=0.6. Emissions for labels (1,2,3): H=(0.2,0.4,0.4), C=(0.5,0.4,0.1). There is no END-state factor. #### To find Best path and score. #### Formula and why it applies $$ \delta_1(j)=\pi_jb_j(2),\quad\delta_2(j)=b_j(2)\max_i[\delta_1(i)a_{ij}] $$ Both states emit2 with probability 0.4, but their initial and transition probabilities still differ. #### Solution ##### Step 1. Initialize $$ \delta_1(H)=0.8(0.4)=0.32,\quad\delta_1(C)=0.2(0.4)=0.08 $$ ##### Step 2. Final H $$ \max(0.32(0.7),0.08(0.4))(0.4) $$ $$ =\max(0.224,0.032)(0.4)=0.0896 $$ Predecessor H. ##### Step 3. Final C $$ \max(0.32(0.3),0.08(0.6))(0.4) $$ $$ =\max(0.096,0.048)(0.4)=0.0384 $$ Predecessor H. ##### Step 4. Select and trace The final H score 0.0896 is larger. Its predecessor is H. **Final answer.** Best path H→H with score 0.0896. **Yaad rakho:** Har destination ke liye incoming paths compare; winner ka arrow yaad. --- ## TEE · A path score is not the probability of the observed sentence Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/formulas/f-hmm--joint-vs-posterior/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/formulas/f-hmm/joint-vs-posterior Formulas explained through full worked examples **Question. An HMM has π(H)=0.8, π(C)=0.2; transitions H:(0.7,0.3), C:(0.4,0.6); emissions H:(0.2,0.4,0.4), C:(0.5,0.4,0.1) for labels1,2,3. For observations 1,3, calculate the probability of the observations by summing all four two-state paths.** #### Given Initial probabilities: π(H)=0.8, π(C)=0.2. Transition matrix: H→H=0.7, H→C=0.3, C→H=0.4, C→C=0.6. Emissions for labels (1,2,3): H=(0.2,0.4,0.4), C=(0.5,0.4,0.1). There is no END-state factor. #### To find P(observations 1,3), not just the best-path score. #### Formula and why it applies $$ P(o_1,o_2)=\sum_{s_1,s_2}\pi_{s_1}b_{s_1}(o_1)a_{s_1s_2}b_{s_2}(o_2) $$ The hidden path is unspecified, so its mutually exclusive alternatives must be summed. #### Solution ##### Step 1. Score H,H $$ 0.8(0.2)(0.7)(0.4)=0.0448 $$ ##### Step 2. Score H,C $$ 0.8(0.2)(0.3)(0.1)=0.0048 $$ ##### Step 3. Score C,H $$ 0.2(0.5)(0.4)(0.4)=0.016 $$ ##### Step 4. Score C,C $$ 0.2(0.5)(0.6)(0.1)=0.006 $$ ##### Step 5. Add all paths $$ P(1,3)=0.0448+0.0048+0.016+0.006=0.0716 $$ **Final answer.** Observation probability =0.0716. The best single-path score 0.0448 is a different quantity. If asked for the posterior of path H,H given observations 1,3, normalize: 0.0448/0.0716 ≈0.625698. Viterbi chooses the best path using joint scores because the observation denominator is common to every candidate. It does not compute that denominator by taking a maximum. --- ## TEE · Lesk algorithm, from words to overlap scores Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/formulas/f-wsd--u5-lesk/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/formulas/f-wsd/u5-lesk Formulas explained through full worked examples **Lesk is a knowledge-based WSD method that chooses senses whose dictionary descriptions have the greatest overlap with contextual evidence.** Original Lesk compares definitions of neighbouring words' senses. A common simplified version compares each target-sense gloss directly with the sentence context. State which version you use. ### Simplified Lesk procedure 1. Remove the target word from the context; tokenize and normalize the remaining content words. 2. Obtain the gloss for each candidate sense and apply the same normalization. 3. Count shared words between the context and each gloss. 4. Select the sense with maximum overlap. If scores tie, report the tie or use a specified tie-break rule. $$ \operatorname{score}(s)=|C\cap G_s|,\qquad s^*=\arg\max_s\operatorname{score}(s) $$ $$ C $$ is the set of context words, $$ G_s $$ the set of words in sense $$ s $$ 's gloss, $$ \cap $$ means common elements, $$ |\cdot| $$ means count, and $$ \arg\max $$ means “the sense giving the largest score”. This version counts unique matching words once. **Question. Use simplified Lesk to disambiguate “bank” in “The river water rose near the bank.” Use C = {river, water, rose, near}; financial gloss = {institution, money, deposit}; river gloss = {land, river, water, edge}.** #### Given The normalized context and two gloss sets are given. Count exact shared words. #### To find The score of each sense and the selected meaning. #### Formula and why it applies Use $$ \operatorname{score}(s)=|C\cap G_s| $$ because the task specifies context–gloss overlap. #### Step 1. Compare the financial sense No word occurs in both {river, water, rose, near} and {institution, money, deposit}. Overlap = {}. #### Step 2. Count the financial score $$ \operatorname{score}(F)=0 $$ . #### Step 3. Compare the river sense Shared words are {river, water}. “Rose” and “near” are not in the gloss. #### Step 4. Count and select $$ \operatorname{score}(R)=2 $$ . Since $$ 2>0 $$ , choose the river-edge sense. **Answer.** Bank means land beside a river. **Question. Use simplified Lesk for “The dog barked loudly.” Treat barked as bark. Context C = {dog, loudly}; sound gloss = {sharp, cry, dog}; tree gloss = {outer, covering, tree}; ship gloss = {sailing, ship}.** #### Given Three glosses and normalized context {dog, loudly}. #### To find Select the sense of bark. #### Formula and why it applies $$ \operatorname{score}(s)=|C\cap G_s| $$ . Lemmatization lets barked match the target lemma bark; it does not create additional overlap words. #### Step 1. Sound sense Intersection = {dog}; score = 1. #### Step 2. Tree sense Intersection = {}; score = 0. #### Step 3. Ship sense Intersection = {}; score = 0. #### Step 4. Compare The sound sense has the unique maximum: 1 > 0. **Answer.** Bark is the dog's cry. **Question. Apply definition–definition Lesk to “pine cone”. Pine senses: P1 = {evergreen, tree, needle, leaf}; P2 = {waste, sorrow, illness}. Cone senses: C1 = {solid, shape, circular}; C2 = {fruit, evergreen, tree}.** #### Given Two candidate senses for each word. Tokens are already normalized; tree and trees would be reduced to the same form if necessary. #### To find The most compatible pair of senses. #### Formula and why it applies Use $$ |G_{P_i}\cap G_{C_j}| $$ for every pair because this is original-style gloss–gloss comparison. #### Step 1. P1 with C1 No shared word → 0. #### Step 2. P1 with C2 Shared {evergreen, tree} → 2. #### Step 3. P2 with each cone sense P2–C1 has no overlap → 0; P2–C2 has no overlap → 0. #### Step 4. Choose the best pair The four scores are 0, 2, 0, 0. Maximum 2 belongs to P1–C2. **Answer.** Pine is the evergreen tree; cone is its fruit-like reproductive structure. **Limitations:** exact overlap misses synonyms, short glosses may produce zeros, and different preprocessing choices change the score. Extended Lesk may include related synset definitions or longer phrase overlaps; do not silently substitute it for a simple question. Lesk: context aur dictionary mein common words gino; sabse bada overlap jeeta. --- ## TEE · Thesaurus / Walker approach, with three complete examples Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/formulas/f-wsd--u5-walker/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/formulas/f-wsd/u5-walker Formulas explained through full worked examples **The Walker approach uses thesaurus categories as semantic evidence.** Associate each candidate sense with a category. Count or weight context words that support that category. Choose the largest total. $$ \operatorname{score}(s)=\sum_{w\in C} I(w,\operatorname{category}(s)) $$ $$ I=1 $$ if the supplied category mapping says the context word supports the sense, otherwise $$ I=0 $$ . The summation symbol means add all those contributions. Weighted questions may replace 1 with a supplied weight. Category membership must be supplied or justified; it is not guessed from word frequency. **Question. Disambiguate bank using Walker scoring. Context words are money, interest, annum, fetch. The finance category supports money, interest and annum; the location category supports none of these words.** #### Given Two categories: Finance and Location. Each supported word contributes 1. #### To find The two totals and selected bank sense. #### Formula and why it applies Sum the category indicators for each sense. #### Step 1. Make a contribution table | Word | Finance | Location | | --- | --- | --- | | money | 1 | 0 | | interest | 1 | 0 | | annum | 1 | 0 | | fetch | 0 | 0 | #### Step 2. Add Finance $$ 1+1+1+0=3 $$ . #### Step 3. Add Location $$ 0+0+0+0=0 $$ . #### Step 4. Select 3 > 0, so select the financial institution. **Answer.** Finance, with score 3 versus 0. **Question. Use Walker scoring for “The fisherman caught bass beside the lake.” Context {fisherman,caught,lake}. The Fish category contains all three clues; the Music category contains none.** #### Given Fish and Music senses, binary category membership given. #### To find The correct bass sense. #### Formula and why it applies Score = sum of context-category matches. #### Step 1. Record Fish matches fisherman → 1; caught → 1; lake → 1. #### Step 2. Add Fish score $$ 1+1+1=3 $$ . #### Step 3. Add Music score $$ 0+0+0=0 $$ . #### Step 4. Choose Fish has the larger score. **Answer.** Bass refers to a fish. **Question. Score “The player used a bat in the match.” Context {player,match}. Sport supports both words with weights 2 and 1; Animal supports neither. Use the supplied weighted Walker scheme.** #### Given Sport weights {player:2, match:1}; Animal weights {player:0, match:0}. #### To find Weighted sense scores. #### Formula and why it applies Use the sum of supplied weights, not an unweighted count, because the question explicitly gives weights. #### Step 1. Sport contribution of player Add 2. #### Step 2. Sport contribution of match Add 1. Sport total = 2 + 1 = 3. #### Step 3. Animal total 0 + 0 = 0. #### Step 4. Compare 3 > 0 selects the sporting implement. **Answer.** Bat is the sports implement. Walker: dictionary ke exact words nahi, context kis topic/category ko support karta hai woh count. --- ## TEE · Multinomial Naive Bayes: the exact solving method Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/formulas/f-wsd--u5-bayes-steps/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/formulas/f-wsd/u5-bayes-steps Formulas explained through full worked examples Naive Bayes selects the sense with the largest prior × likelihood product. The context evidence denominator is identical across candidate senses, so it can be omitted for comparison. $$ \operatorname{score}(s)=P(s)\prod_j P(w_j\mid s). $$ For add-one multinomial smoothing: $$ P(w\mid s)=\frac{C(w,s)+1}{N_s+V}. $$ $$ C(w,s) $$ counts occurrences of word $$ w $$ in training contexts of sense $$ s $$ ; $$ N_s $$ is the total number of word occurrences in those contexts; $$ V $$ is the shared vocabulary size. It is not the number of training sentences. Repeated query words contribute repeated factors. ### Always keep these counts separate Class priors use **number of examples**. Word likelihoods in this multinomial model use **number of word occurrences**. The vocabulary counts **unique words across all classes**. 1. List the class examples and calculate priors. 2. Write one shared vocabulary and count its size. 3. Count total word occurrences in each class. 4. Count each query word in each class. 5. Apply the given smoothing rule. 6. Multiply each prior by all its query likelihoods. 7. Compare scores; divide each score by their sum only if a normalized posterior is requested. $$ P(s\mid C)=\frac{\operatorname{score}(s)}{\sum_{s'}\operatorname{score}(s')}. $$ **Why smoothing is needed:** an unseen clue gives $$ 0/N_s=0 $$ ; one zero makes the whole product zero. Add-one smoothing redistributes probability across every vocabulary word and prevents these zero factors. Use it only when requested, taught for the task, or state the assumption. **Different model warning:** a Bernoulli Naive Bayes question counts presence/absence per document and may include absence factors. Do not use its denominator in a multinomial word-count question. Pehle sentences se prior; phir words se likelihood; dono counts mix mat kar. --- ## TEE · Mam's bank numerical—every count and every multiplication Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/formulas/f-wsd--u5-bayes-mam/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/formulas/f-wsd/u5-bayes-mam Formulas explained through full worked examples **Question. Classify the sense of “bank” using the training table below. The sentence is: “As the stream flowed gently past the shore, we set up camp by the bank of the river, discussing the possibility of borrowing a loan to buy a boat for future trips.” Use only the highlighted feature words stream, shore and loan, and add-one multinomial smoothing.** #### Given | Training row | Context words | Sense | | --- | --- | --- | | 1 | money, deposit, loan | Finance | | 2 | river, water, shore | Riverbank | | 3 | account, savings, investment | Finance | | 4 | boat, river, stream | Riverbank | | 5 | branch, deposit, check | Finance | | 6 | fishing, shore, river | Riverbank | Query feature set: {stream, shore, loan}. Other sentence words are not included as features in this supplied example. #### To find The most probable bank sense and normalized posteriors. #### Formula and why it applies Naive Bayes selects the sense with the largest prior × likelihood product. The context evidence denominator is identical across candidate senses, so it can be omitted for comparison. $$ \operatorname{score}(s)=P(s)\prod_j P(w_j\mid s). $$ For add-one multinomial smoothing: $$ P(w\mid s)=\frac{C(w,s)+1}{N_s+V}. $$ $$ C(w,s) $$ counts occurrences of word $$ w $$ in training contexts of sense $$ s $$ ; $$ N_s $$ is the total number of word occurrences in those contexts; $$ V $$ is the shared vocabulary size. It is not the number of training sentences. Repeated query words contribute repeated factors. #### Step 1. Calculate priors Finance appears in 3 of the 6 rows; Riverbank appears in 3 of 6. $$ P(F)=\frac36=\frac12,\qquad P(R)=\frac36=\frac12. $$ #### Step 2. Construct the vocabulary {money, deposit, loan, river, water, shore, account, savings, investment, boat, stream, fishing, branch, check}. Thus V=14. Repeated deposit or river is counted once in vocabulary. #### Step 3. Count word occurrences per class There are 3 finance rows × 3 words = 9 word occurrences. There are 3 river rows × 3 words = 9. Thus $$ N_F=N_R=9 $$ , and each smoothed denominator is $$ 9+14=23 $$ . #### Step 4. Count the three query words | Word | Finance count | Riverbank count | | --- | --- | --- | | stream | 0 | 1 | | shore | 0 | 2 | | loan | 1 | 0 | #### Step 5. Calculate finance likelihoods $$ P(\text{stream}\mid F)=\frac{0+1}{23}=\frac1{23} $$ $$ P(\text{shore}\mid F)=\frac{0+1}{23}=\frac1{23} $$ $$ P(\text{loan}\mid F)=\frac{1+1}{23}=\frac2{23} $$ #### Step 6. Multiply the finance score $$ \operatorname{score}(F)=\frac12\times\frac1{23}\times\frac1{23}\times\frac2{23}=\frac1{23^3}=\frac1{12167}\approx0.00008219. $$ #### Step 7. Calculate river likelihoods $$ P(\text{stream}\mid R)=\frac{1+1}{23}=\frac2{23} $$ $$ P(\text{shore}\mid R)=\frac{2+1}{23}=\frac3{23} $$ $$ P(\text{loan}\mid R)=\frac{0+1}{23}=\frac1{23} $$ #### Step 8. Multiply the river score $$ \operatorname{score}(R)=\frac12\times\frac2{23}\times\frac3{23}\times\frac1{23}=\frac3{12167}\approx0.00024657. $$ #### Step 9. Compare scores 3/12167 is larger than 1/12167. Riverbank wins even though “loan” is financial evidence: the combined clues matter. #### Step 10. Normalize if a posterior is requested $$ P(F\mid C)=\frac{1/12167}{4/12167}=\frac14=0.25 $$ $$ P(R\mid C)=\frac{3/12167}{4/12167}=\frac34=0.75. $$ **Answer.** Bank is the riverbank sense. Scores are 1/12167 and 3/12167; normalized model posteriors are 0.25 and 0.75. **Without smoothing:** stream and shore are absent in Finance; loan is absent in Riverbank. Both unsmoothed scores become zero, so there is no valid winning class from those products. Also, unnormalized scores should not be labelled as final posterior probabilities. Zero word? Add-one: count+1 upar, total words+vocabulary neeche. Last mein score bada wala sense. --- ## TEE · More WSD numericals: given likelihoods, unequal priors and repeated clues Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/formulas/f-wsd--u5-bayes-practice/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/formulas/f-wsd/u5-bayes-practice Formulas explained through full worked examples **Question. For bass, let P(Fish)=0.6 and P(Music)=0.4. Given Fish likelihoods P(caught|Fish)=0.5, P(lake|Fish)=0.4 and Music likelihoods P(caught|Music)=0.05, P(lake|Music)=0.1, classify context {caught,lake} and normalize.** #### Given Priors and likelihoods are provided directly; no smoothing or training-count estimation is requested. #### To find The winning sense and its posterior. #### Formula and why it applies $$ \operatorname{score}(s)=P(s)P(\text{caught}\mid s)P(\text{lake}\mid s) $$ , because the two features are treated as conditionally independent. #### Step 1. Fish multiplication $$ 0.6\times0.5=0.3 $$ , then $$ 0.3\times0.4=0.12 $$ . #### Step 2. Music multiplication $$ 0.4\times0.05=0.02 $$ , then $$ 0.02\times0.1=0.002 $$ . #### Step 3. Total evidence $$ 0.12+0.002=0.122 $$ . #### Step 4. Normalize $$ P(Fish\mid C)=0.12/0.122\approx0.983607 $$ $$ P(Music\mid C)=0.002/0.122\approx0.016393. $$ **Answer.** Fish, with model posterior approximately 0.983607. **Question. Disambiguate bank from the repeated context tokens {loan, loan, river}. Priors are Finance=2/3 and River=1/3. Vocabulary V=4. Finance has N=8 total tokens, loan count 3 and river count 0; River has N=4, loan count 0 and river count 2. Apply add-one smoothing.** #### Given Multinomial counts are supplied. The two appearances of loan must both contribute. #### To find Scores and normalized Finance posterior. #### Formula and why it applies Naive Bayes selects the sense with the largest prior × likelihood product. The context evidence denominator is identical across candidate senses, so it can be omitted for comparison. $$ \operatorname{score}(s)=P(s)\prod_j P(w_j\mid s). $$ For add-one multinomial smoothing: $$ P(w\mid s)=\frac{C(w,s)+1}{N_s+V}. $$ $$ C(w,s) $$ counts occurrences of word $$ w $$ in training contexts of sense $$ s $$ ; $$ N_s $$ is the total number of word occurrences in those contexts; $$ V $$ is the shared vocabulary size. It is not the number of training sentences. Repeated query words contribute repeated factors. #### Step 1. Finance denominator $$ 8+4=12 $$ . $$ P(loan\mid F)=(3+1)/12=1/3 $$ ; $$ P(river\mid F)=(0+1)/12=1/12 $$ . #### Step 2. River denominator $$ 4+4=8 $$ . $$ P(loan\mid R)=(0+1)/8=1/8 $$ ; $$ P(river\mid R)=(2+1)/8=3/8 $$ . #### Step 3. Finance product $$ \operatorname{score}(F)=\frac23\times\frac13\times\frac13\times\frac1{12}=\frac1{162}\approx0.00617284. $$ #### Step 4. River product $$ \operatorname{score}(R)=\frac13\times\frac18\times\frac18\times\frac38=\frac1{512}\approx0.001953125. $$ #### Step 5. Compare and normalize $$ P(F\mid C)=\frac{1/162}{1/162+1/512}=\frac{512}{674}=\frac{256}{337}\approx0.759644. $$ **Answer.** Finance wins. Repeated loan contributes a squared likelihood; do not reduce the query to a unique-word set. ### Answer-writing order in an exam Write the class priors, vocabulary size, class token totals, a likelihood table, both score products and the final class. Keep fractions until the comparison where possible. If the question only asks classification, normalization is optional; if it asks posterior probabilities, normalization is required. Repeated word aaye toh factor bhi repeat. Sirf word-set bana diya toh multinomial answer badal jayega. --- ## TEE · Label and one-hot encoding: three complete examples Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/formulas/f-vectors--u2-label-onehot/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/formulas/f-vectors/u2-label-onehot Formulas explained through full worked examples **Label encoding** assigns each category or word a unique integer ID. **One-hot encoding** assigns a length- $$ V $$ vector with one 1 at the word's vocabulary position and zeros elsewhere. $$ \operatorname{onehot}(w)_j=\begin{cases}1,&j=\operatorname{index}(w)\\0,&\text{otherwise.}\end{cases} $$ The index is an ID, not a meaning score. One-hot represents one token/category; a multiword document generally needs multiple token vectors or an aggregation scheme. **Question. Encode “dog cat dog” using vocabulary [cat,dog,fish], IDs starting at 0. Give label IDs and a one-hot vector for every token.** #### Given cat→0, dog→1, fish→2. Vector columns have the same order. #### To find The ID sequence and three one-hot rows. #### Formula and why it applies Replace each word by its ID for labels; set its vocabulary position to 1 for one-hot. #### Step 1. Label IDs dog→1, cat→0, dog→1 → [1,0,1]. #### Step 2. First dog [0,1,0]. #### Step 3. Cat [1,0,0]. #### Step 4. Second dog [0,1,0]. Repeated tokens keep the same representation. **Answer.** IDs [1,0,1]; one-hot rows [0,1,0], [1,0,0], [0,1,0]. **Question. Encode the city sequence Mumbai, Delhi, Mumbai, Pune using vocabulary [Delhi,Mumbai,Pune] and label IDs starting at 1.** #### Given Delhi→1, Mumbai→2, Pune→3. #### To find IDs and one-hot rows. #### Formula and why it applies One-hot uses three coordinates regardless of whether label IDs begin at 0 or 1. #### Step 1. IDs [2,1,2,3]. #### Step 2. Mumbai rows Each Mumbai → [0,1,0]. #### Step 3. Delhi and Pune Delhi→[1,0,0]; Pune→[0,0,1]. **Answer.** Rows [0,1,0], [1,0,0], [0,1,0], [0,0,1]. **Question. One-hot encode categories [red,green,blue] for a linear regression with an intercept. Then give dummy coding after dropping blue.** #### Given Full column order [red,green,blue]. Blue is the baseline. #### To find Full and reduced codes. #### Formula and why it applies Full one-hot rows sum to 1. With an intercept this creates a linear dependency; dropping one category column gives k−1 independent dummy indicators. #### Step 1. Full encoding | Category | red | green | blue | | --- | --- | --- | --- | | red | 1 | 0 | 0 | | green | 0 | 1 | 0 | | blue | 0 | 0 | 1 | #### Step 2. Drop blue column | Category | red | green | | --- | --- | --- | | red | 1 | 0 | | green | 0 | 1 | | blue | 0 | 0 | #### Step 3. Interpret zero row [0,0] now identifies the baseline blue, not missing data. **Answer.** Full width 3; reduced dummy width 2. Dropping a column is model-dependent, not a requirement for every neural embedding input. **Limitations:** one-hot is large and sparse and does not express semantic similarity. Unseen words need a defined unknown-token or vocabulary-update policy. Label IDs are compact but should not be treated as an ordered scalar feature for nominal words. Label = ID number. One-hot = us ID ke box mein 1, baaki 0. --- ## TEE · Bag of words and bag of n-grams Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/formulas/f-vectors--u2-bow/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/formulas/f-vectors/u2-bow Formulas explained through full worked examples **Bag of words (BoW) represents a document by the counts of vocabulary terms, discarding their positions.** A binary BoW variant stores only presence/absence; a count BoW stores frequencies. State which is used. $$ x_{d,j}=C(v_j,d). $$ $$ x_{d,j} $$ is document $$ d $$ 's coordinate for vocabulary term $$ v_j $$ . $$ C(v_j,d) $$ is its occurrence count. A **bag of n-grams** uses short contiguous token sequences as features. Bigrams can distinguish “not good” from “good”, but increasing n enlarges the vocabulary and creates more rare features. **Question. Build count and binary BoW for D1=“cat cat dog” and D2=“dog fish”. Use vocabulary [cat,dog,fish] and no stopword removal.** #### Given Two tokenized documents, fixed vocabulary order. #### To find Count vectors and binary vectors. #### Formula and why it applies Count BoW uses $$ C(v_j,d) $$ . Binary BoW is 1 when that count is positive and 0 otherwise. #### Step 1. D1 counts cat=2, dog=1, fish=0 → [2,1,0]. #### Step 2. D2 counts cat=0, dog=1, fish=1 → [0,1,1]. #### Step 3. Convert to presence D1 [1,1,0]; D2 [0,1,1]. **Answer.** Count rows [2,1,0], [0,1,1]; binary rows [1,1,0], [0,1,1]. **Question. After lowercasing and removing {is,a,means}, obtain BoW vectors for D1=“Continuous learning is learning lifetime”, D2=“Continuous learning means lifetime learning”, D3=“Learning is a good practice”. Use [continuous,good,learning,lifetime,practice].** #### Given Three documents and exact stopword set from the supplied slides. #### To find The document–term count matrix. #### Formula and why it applies Remove only the listed stopwords, then count every remaining token in the fixed column order. #### Step 1. Clean D1 [continuous,learning,learning,lifetime] → [1,0,2,1,0]. #### Step 2. Clean D2 [continuous,learning,lifetime,learning] → [1,0,2,1,0]. #### Step 3. Clean D3 [learning,good,practice] → [0,1,1,0,1]. #### Step 4. Compare D1 and D2 They have identical BoW vectors despite different original wording/order. **Answer.** Rows [1,0,2,1,0], [1,0,2,1,0], [0,1,1,0,1]. **Question. Build a bag of bigrams for “not good but not bad” with vocabulary [not good,good but,but not,not bad]. Do not add sentence boundary tokens.** #### Given Five tokens; use adjacent pairs, stride 1. #### To find The bigram count vector. #### Formula and why it applies A sequence of $$ L $$ tokens has $$ L-1 $$ adjacent bigrams when no boundaries are added. #### Step 1. Enumerate pairs (not,good), (good,but), (but,not), (not,bad). #### Step 2. Count each vocabulary bigram Each appears once. #### Step 3. Write vector [1,1,1,1]. **Answer.** Four bigram features, each with count 1. BoW: order bhool, word-count rakh. Bag of bigrams: neighbouring word-pair count. --- ## TEE · TF–IDF from the exact slide convention Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/formulas/f-vectors--u2-tfidf/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/formulas/f-vectors/u2-tfidf Formulas explained through full worked examples **TF–IDF weights a term highly when it is frequent in a particular document but uncommon across the corpus.** The supplied Unit 2 slides use normalized term frequency and base-10 inverse document frequency: $$ \operatorname{TF}(t,d)=\frac{C(t,d)}{N_d} $$ $$ \operatorname{IDF}(t)=\log_{10}\frac{D}{\operatorname{df}(t)} $$ $$ \operatorname{TFIDF}(t,d)=\operatorname{TF}(t,d)\operatorname{IDF}(t). $$ $$ N_d $$ is the document's token count after the specified preprocessing; $$ D $$ is number of documents; $$ \operatorname{df}(t) $$ is the number of documents containing the term at least once. A word repeated ten times in one document contributes one to document frequency. **Why the log?** It compresses the ratio's scale. If a term occurs in all documents, $$ D/df=1 $$ , so IDF=0 under this unsmoothed convention. Other libraries use smoothed IDF, different TF or normalization. Use the question's formula; do not silently substitute a library default. **Question. A corpus contains 10 documents. The word model appears in 2 documents and occurs 3 times in a 12-token document. Compute TF, IDF and TF–IDF using the slide convention.** #### Given D=10, df(model)=2, C(model,d)=3, Nd=12; log base 10. #### To find The TF–IDF weight. #### Formula and why it applies $$ TF=C/N_d $$ , $$ IDF=\log_{10}(D/df) $$ , then multiply. #### Step 1. TF $$ 3/12=0.25 $$ . #### Step 2. Document ratio $$ 10/2=5 $$ . #### Step 3. IDF $$ \log_{10}5\approx0.698970 $$ . #### Step 4. Weight $$ 0.25\times0.698970\approx0.174743 $$ . **Answer.** TF=0.25, IDF≈0.698970, TF–IDF≈0.174743. **Question. A term occurs 4 times in a 20-token document and is present in all 8 corpus documents. Calculate its TF–IDF.** #### Given C=4, Nd=20, D=8, df=8. #### To find Its weight under unsmoothed base-10 IDF. #### Formula and why it applies $$ TF=C/N_d $$ , $$ IDF=\log_{10}(D/df) $$ . #### Step 1. TF $$ 4/20=0.2 $$ . #### Step 2. IDF $$ \log_{10}(8/8)=\log_{10}1=0 $$ . #### Step 3. Multiply $$ 0.2\times0=0 $$ . **Answer.** Weight 0: this term does not distinguish documents under this convention. **Question. Find TF–IDF vectors for D1=“cat cat dog”, D2=“dog fish”, D3=“cat fish fish”, using [cat,dog,fish], normalized TF and log10 IDF.** #### Given No further preprocessing. Each term occurs in exactly two of the three documents. #### To find All three vectors. #### Formula and why it applies First count df by documents, then compute each document's TF, then multiply by its term's IDF. #### Step 1. Common IDF $$ D=3 $$ , $$ df(cat)=df(dog)=df(fish)=2 $$ . Each IDF $$ =\log_{10}(3/2)\approx0.176091 $$ . #### Step 2. D1 term frequencies Length 3 → $$ [2/3,1/3,0] $$ . #### Step 3. D1 weights $$ [(2/3)0.176091,(1/3)0.176091,0]\approx[0.117394,0.058697,0] $$ . #### Step 4. D2 weights Length 2; TF $$ [0,1/2,1/2] $$ . Vector $$ \approx[0,0.088046,0.088046] $$ . #### Step 5. D3 weights Length 3; TF $$ [1/3,0,2/3] $$ . Vector $$ \approx[0.058697,0,0.117394] $$ . **Answer.** D1≈[0.117394,0.058697,0]; D2≈[0,0.088046,0.088046]; D3≈[0.058697,0,0.117394]. TF: iss document mein kitna. IDF: poore corpus mein kitna rare. Dono multiply. --- ## TEE · Cosine: products, lengths and division Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/formulas/f-vectors--u2-cosine/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/formulas/f-vectors/u2-cosine Formulas explained through full worked examples **Cosine similarity compares the direction of two nonzero vectors.** Multiply matching coordinates and add to obtain the dot product. To obtain each vector's norm, square its coordinates, add them and take a square root. $$ \cos(x,y)=\frac{\sum_i x_i y_i}{\sqrt{\sum_i x_i^2}\sqrt{\sum_i y_i^2}}. $$ The vectors must use the same coordinate/vocabulary order. For nonnegative BoW or TF–IDF vectors, the result lies from 0 to 1; for general real-valued vectors, it can range from −1 to 1. A zero vector makes this formula undefined. **Question. Find cosine similarity for x=[1,1] and y=[1,0].** #### Given Nonzero vectors, matching coordinate order. #### To find Find cosine similarity for x=[1,1] and y=[1,0]. #### Formula and why it applies Use $$ \cos(x,y)=(x\cdot y)/(\|x\|\|y\|) $$ because direction similarity is required. #### Step 1. Multiply matching coordinates 1 × 1 = 1; 1 × 0 = 0. #### Step 2. Add products 1 + 0 = 1. #### Step 3. Length of x $$ \sqrt{(1)^2+(1)^2}=\sqrt{2}. $$ #### Step 4. Length of y $$ \sqrt{(1)^2+(0)^2}=\sqrt{1}. $$ #### Step 5. Divide $$ \frac{1}{\sqrt{2}\sqrt{1}}\approx 0.707107. $$ **Answer.** Cosine≈0.707107. **Question. Find cosine similarity for x=[1,2] and y=[2,4].** #### Given Nonzero vectors, matching coordinate order. #### To find Find cosine similarity for x=[1,2] and y=[2,4]. #### Formula and why it applies Use $$ \cos(x,y)=(x\cdot y)/(\|x\|\|y\|) $$ because direction similarity is required. #### Step 1. Multiply matching coordinates 1 × 2 = 2; 2 × 4 = 8. #### Step 2. Add products 2 + 8 = 10. #### Step 3. Length of x $$ \sqrt{(1)^2+(2)^2}=\sqrt{5}. $$ #### Step 4. Length of y $$ \sqrt{(2)^2+(4)^2}=\sqrt{20}. $$ #### Step 5. Divide $$ \frac{10}{\sqrt{5}\sqrt{20}}\approx 1.000000. $$ **Answer.** Cosine≈1.000000. **Question. Find cosine similarity for x=[1,2,0] and y=[0,1,2].** #### Given Nonzero vectors, matching coordinate order. #### To find Find cosine similarity for x=[1,2,0] and y=[0,1,2]. #### Formula and why it applies Use $$ \cos(x,y)=(x\cdot y)/(\|x\|\|y\|) $$ because direction similarity is required. #### Step 1. Multiply matching coordinates 1 × 0 = 0; 2 × 1 = 2; 0 × 2 = 0. #### Step 2. Add products 0 + 2 + 0 = 2. #### Step 3. Length of x $$ \sqrt{(1)^2+(2)^2+(0)^2}=\sqrt{5}. $$ #### Step 4. Length of y $$ \sqrt{(0)^2+(1)^2+(2)^2}=\sqrt{5}. $$ #### Step 5. Divide $$ \frac{2}{\sqrt{5}\sqrt{5}}\approx 0.400000. $$ **Answer.** Cosine≈0.400000. Proportional vectors [1,2] and [2,4] have cosine 1 but nonzero Euclidean distance. Direction similarity does not require identical magnitudes. Cosine: matching products add upar; dono lengths multiply neeche. --- ## TEE · Jaccard: intersection, union and distance Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/formulas/f-vectors--u2-jaccard/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/formulas/f-vectors/u2-jaccard Formulas explained through full worked examples **Jaccard similarity is the fraction of shared distinct elements among all distinct elements in either set.** $$ J(A,B)=\frac{|A\cap B|}{|A\cup B|}. $$ Intersection means common elements; union means all distinct elements. Repetitions do not count again. Jaccard distance is $$ 1-J $$ , not the same quantity as similarity. Two empty sets require a declared convention because the fraction becomes 0/0. **Question. Find Jaccard similarity and distance for A={cat, dog} and B={dog, fish}.** #### Given Sets contain unique elements. #### To find Find Jaccard similarity and distance for A={cat, dog} and B={dog, fish}. #### Formula and why it applies $$ J=|A\cap B|/|A\cup B| $$ ; $$ d=1-J $$ . #### Step 1. Find intersection {dog}; count 1. #### Step 2. Find union {cat, dog, fish}; count 3. #### Step 3. Divide J=1/3≈0.333333. #### Step 4. Subtract from one Distance=1−0.333333≈0.666667. **Answer.** Similarity≈0.333333; distance≈0.666667. **Question. Find Jaccard similarity and distance for A={red, blue} and B={red, blue}.** #### Given Sets contain unique elements. #### To find Find Jaccard similarity and distance for A={red, blue} and B={red, blue}. #### Formula and why it applies $$ J=|A\cap B|/|A\cup B| $$ ; $$ d=1-J $$ . #### Step 1. Find intersection {red, blue}; count 2. #### Step 2. Find union {red, blue}; count 2. #### Step 3. Divide J=2/2≈1.000000. #### Step 4. Subtract from one Distance=1−1.000000≈0.000000. **Answer.** Similarity≈1.000000; distance≈0.000000. **Question. Find Jaccard similarity and distance for A={apple} and B={orange, pear}.** #### Given Sets contain unique elements. #### To find Find Jaccard similarity and distance for A={apple} and B={orange, pear}. #### Formula and why it applies $$ J=|A\cap B|/|A\cup B| $$ ; $$ d=1-J $$ . #### Step 1. Find intersection {}; count 0. #### Step 2. Find union {apple, orange, pear}; count 3. #### Step 3. Divide J=0/3≈0.000000. #### Step 4. Subtract from one Distance=1−0.000000≈1.000000. **Answer.** Similarity≈0.000000; distance≈1.000000. Jaccard = common unique ÷ total unique. Distance = 1 minus similarity. --- ## TEE · Manhattan, Euclidean and Minkowski: three complete comparisons Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/formulas/f-vectors--u2-distances/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/formulas/f-vectors/u2-distances Formulas explained through full worked examples **Distance measures separation; smaller means closer.** Absolute values prevent positive and negative differences from cancelling. $$ d_1=\sum_i|x_i-y_i|\quad\text{Manhattan} $$ $$ d_2=\sqrt{\sum_i(x_i-y_i)^2}\quad\text{Euclidean} $$ $$ d_p=\left(\sum_i|x_i-y_i|^p\right)^{1/p},\quad p\ge1\quad\text{Minkowski}. $$ Manhattan is p=1; Euclidean is p=2. For p=3, cube absolute differences, add, then take a cube root. **Question. Find Manhattan, Euclidean and Minkowski p=3 distances for x=[1,2], y=[4,6].** #### Given Vectors share coordinate order; negative inputs are allowed. #### To find Find Manhattan, Euclidean and Minkowski p=3 distances for x=[1,2], y=[4,6]. #### Formula and why it applies Use $$ d_1=\sum|\Delta_i| $$ , $$ d_2=\sqrt{\sum|\Delta_i|^2} $$ , $$ d_3=\sqrt[3]{\sum|\Delta_i|^3} $$ . #### Step 1. Absolute differences |1 − (4)| = 3; |2 − (6)| = 4. #### Step 2. Manhattan 3 + 4 = 7. #### Step 3. Square and add 3²=9; 4²=16. Sum=25. #### Step 4. Euclidean root $$ \sqrt{25}\approx 5.000000. $$ #### Step 5. Cube and add 3³=27; 4³=64. Sum=91. #### Step 6. Minkowski root $$ \sqrt[3]{91}\approx 4.497941. $$ **Answer.** Manhattan=7; Euclidean≈5.000000; p=3≈4.497941. **Question. Find Manhattan, Euclidean and Minkowski p=3 distances for x=[0,1,2], y=[2,1,0].** #### Given Vectors share coordinate order; negative inputs are allowed. #### To find Find Manhattan, Euclidean and Minkowski p=3 distances for x=[0,1,2], y=[2,1,0]. #### Formula and why it applies Use $$ d_1=\sum|\Delta_i| $$ , $$ d_2=\sqrt{\sum|\Delta_i|^2} $$ , $$ d_3=\sqrt[3]{\sum|\Delta_i|^3} $$ . #### Step 1. Absolute differences |0 − (2)| = 2; |1 − (1)| = 0; |2 − (0)| = 2. #### Step 2. Manhattan 2 + 0 + 2 = 4. #### Step 3. Square and add 2²=4; 0²=0; 2²=4. Sum=8. #### Step 4. Euclidean root $$ \sqrt{8}\approx 2.828427. $$ #### Step 5. Cube and add 2³=8; 0³=0; 2³=8. Sum=16. #### Step 6. Minkowski root $$ \sqrt[3]{16}\approx 2.519842. $$ **Answer.** Manhattan=4; Euclidean≈2.828427; p=3≈2.519842. **Question. Find Manhattan, Euclidean and Minkowski p=3 distances for x=[-1,3], y=[2,-1].** #### Given Vectors share coordinate order; negative inputs are allowed. #### To find Find Manhattan, Euclidean and Minkowski p=3 distances for x=[-1,3], y=[2,-1]. #### Formula and why it applies Use $$ d_1=\sum|\Delta_i| $$ , $$ d_2=\sqrt{\sum|\Delta_i|^2} $$ , $$ d_3=\sqrt[3]{\sum|\Delta_i|^3} $$ . #### Step 1. Absolute differences |-1 − (2)| = 3; |3 − (-1)| = 4. #### Step 2. Manhattan 3 + 4 = 7. #### Step 3. Square and add 3²=9; 4²=16. Sum=25. #### Step 4. Euclidean root $$ \sqrt{25}\approx 5.000000. $$ #### Step 5. Cube and add 3³=27; 4³=64. Sum=91. #### Step 6. Minkowski root $$ \sqrt[3]{91}\approx 4.497941. $$ **Answer.** Manhattan=7; Euclidean≈5.000000; p=3≈4.497941. Manhattan: absolute-add. Euclidean: square-add-root. Minkowski: p-power-add-p-root. --- ## TEE · Distributional hypothesis and a co-occurrence matrix Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/formulas/f-vectors--u5-distribution/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/formulas/f-vectors/u5-distribution Formulas explained through full worked examples **The distributional hypothesis states that words occurring in similar contexts tend to have similar meanings or semantic behaviour.** “Cat” and “dog” often occur near pet, feed and animal. Their context patterns can therefore be represented and compared numerically. A **vector** is an ordered list of numbers. Each position must have a fixed meaning. For context order [pet, food, road], cat=[4,3,0] means four co-occurrences with pet, three with food, none with road. A **matrix** stacks vectors as rows in a table. A **word–context co-occurrence matrix** records how often a target word occurs with each context word under a specified context definition. A window may be a sentence, document, or a fixed number of neighbouring tokens. State its width, direction and whether targets count themselves. Choose target words → Choose context/window → Count co-occurrences → Store each row as a vector → Compare vectors **Question. Build the symmetric co-occurrence matrix for “cats eat fish” using a window of one token on either side, no self-counts and vocabulary [cats,eat,fish].** #### Given One sentence, one occurrence of each token. Left and right adjacent tokens are contexts. #### To find A 3×3 target-by-context count matrix. #### Formula and why it applies $$ X_{ij} $$ counts the number of times context word $$ j $$ occurs within the chosen window of target $$ i $$ . #### Step 1. Target cats Only eat is adjacent. Row cats = [0,1,0]. #### Step 2. Target eat cats is left and fish is right. Row eat = [1,0,1]. #### Step 3. Target fish Only eat is adjacent. Row fish = [0,1,0]. #### Step 4. Assemble the matrix | Target / context | cats | eat | fish | | --- | --- | --- | --- | | cats | 0 | 1 | 0 | | eat | 1 | 0 | 1 | | fish | 0 | 1 | 0 | **Answer.** Rows are [0,1,0], [1,0,1], [0,1,0]. The symmetry follows from the symmetric window. **Question. Build a target–context matrix for sentences “cats eat fish” and “dogs eat fish”. Target words are cats and dogs; context columns [eat,fish]; use the entire sentence excluding the target.** #### Given Each sentence contains one target. Count context token occurrences. #### To find Two context vectors. #### Formula and why it applies Count each context word in every sentence containing the target. #### Step 1. Count cats contexts In its sentence: eat occurs once, fish once → [1,1]. #### Step 2. Count dogs contexts In its sentence: eat occurs once, fish once → [1,1]. #### Step 3. Compare Both targets have identical observed context patterns in this very small corpus. **Answer.** cats=[1,1], dogs=[1,1]. This corpus supplies no context evidence to distinguish them. **Question. For “a b a”, use a right-only window of one token. Build the matrix with rows/columns [a,b].** #### Given Count directed target→next-word events, not both directions. #### To find The co-occurrence matrix. #### Formula and why it applies $$ X_{ij}=C(\text{target }i\text{ immediately followed by }j) $$ . #### Step 1. First target a Next token is b → add 1 to X(a,b). #### Step 2. Target b Next token is a → add 1 to X(b,a). #### Step 3. Final a No following token → no addition. #### Step 4. Write matrix | Target / next | a | b | | --- | --- | --- | | a | 0 | 1 | | b | 1 | 0 | **Answer.** a=[0,1], b=[1,0]. A different context window would produce a different matrix. Word ka meaning: uske aas-paas kaunse words kitni baar aaye, uski number-list. --- ## TEE · Word2Vec: CBOW and skip-gram from training pairs Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/formulas/f-vectors--u5-word2vec/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/formulas/f-vectors/u5-word2vec Formulas explained through full worked examples **Word2Vec learns dense word embeddings by predicting words from nearby context.** Its two common architectures are CBOW and skip-gram. A vocabulary ID or one-hot vector is the input representation; the learned embedding table converts that identity into a short trainable vector. | Architecture | Input | Prediction | Memory cue | | --- | --- | --- | --- | | CBOW | Surrounding context words | Centre/target word | Context → centre | | Skip-gram | Centre/target word | Surrounding words | Centre → context | CBOW: the + sat → Average context embeddings → Predict: cat Skip-gram: cat → Embedding of cat → Predict: the and sat For vocabulary size $$ V $$ and embedding dimension $$ d $$ , an input one-hot row is $$ 1\times V $$ ; the input embedding matrix is $$ V\times d $$ ; the hidden row is $$ 1\times d $$ ; the output matrix is $$ d\times V $$ ; output scores are $$ 1\times V $$ . Multiplying a one-hot row by the embedding matrix simply selects one row. $$ P(w_o\mid w_i)=\frac{\exp(u_{w_o}^{T}v_{w_i})}{\sum_{w\in V}\exp(u_w^{T}v_{w_i})}. $$ $$ v_{w_i} $$ is the input word embedding, $$ u_w $$ an output-context embedding, and the dot product is a compatibility score. Softmax turns all scores into probabilities that sum to one. Training increases the probability of observed context words. Negative sampling or hierarchical softmax can reduce computation in larger vocabularies; use the algorithm specified by the question. **Question. For the sentence “the cat sat on mat”, generate CBOW and skip-gram training data for centre word sat with a window of one token on each side.** #### Given Tokens indexed 1 to 5: the, cat, sat, on, mat. Centre = sat. #### To find The CBOW example and the skip-gram pairs. #### Formula and why it applies CBOW predicts the centre from its neighbours; skip-gram predicts each neighbour from the centre. #### Step 1. Identify neighbours Left neighbour = cat; right neighbour = on. “the” and “mat” are outside this window. #### Step 2. CBOW example Input [cat,on] → target sat. #### Step 3. Skip-gram examples (sat,cat) and (sat,on), with first item as input and second as predicted context. **Answer.** CBOW: [cat,on]→sat. Skip-gram: sat→cat and sat→on. **Question. The embeddings of context words cat and on are [1,2] and [3,0]. Compute the CBOW hidden vector using mean aggregation. Output vectors for [sat,ran] are [1,0] and [0,1]. Compute softmax probabilities.** #### Given Context embeddings [1,2], [3,0]; mean aggregation; two candidate output vectors. #### To find The hidden vector, logits and probabilities. #### Formula and why it applies Mean context vector $$ h=(v_1+v_2)/2 $$ . Logit $$ z_w=u_w^Th $$ . Softmax $$ P(w)=e^{z_w}/\sum_j e^{z_j} $$ . #### Step 1. Add coordinates $$ [1+3,\,2+0]=[4,2] $$ . #### Step 2. Divide by number of words $$ h=[4/2,2/2]=[2,1] $$ . #### Step 3. Compute logits $$ z_{sat}=1(2)+0(1)=2 $$ . $$ z_{ran}=0(2)+1(1)=1 $$ . #### Step 4. Exponentials $$ e^2\approx7.389056 $$ , $$ e^1\approx2.718282 $$ ; sum $$ \approx10.107338 $$ . #### Step 5. Divide $$ P(sat)\approx0.731059 $$ , $$ P(ran)\approx0.268941 $$ . **Answer.** h=[2,1]; sat is more probable (0.731059). **Question. A skip-gram centre embedding is v=[1,−1]. Output embeddings for [eat,sleep,drive] are [1,1], [0,1], [1,0]. Calculate the predicted context probabilities.** #### Given One centre vector and three output vectors; full softmax over these three words. #### To find Three logits and three probabilities. #### Formula and why it applies $$ z_j=u_j^Tv $$ , then $$ P(j)=e^{z_j}/\sum_k e^{z_k} $$ . #### Step 1. Eat dot product $$ 1(1)+1(-1)=0 $$ . #### Step 2. Sleep and drive dot products $$ z_{sleep}=0(1)+1(-1)=-1 $$ . $$ z_{drive}=1(1)+0(-1)=1 $$ . #### Step 3. Exponentiate and sum $$ e^0=1 $$ , $$ e^{-1}\approx0.367879 $$ , $$ e^1\approx2.718282 $$ . Sum $$ \approx4.086161 $$ . #### Step 4. Normalize $$ P(eat)\approx0.244728 $$ ; $$ P(sleep)\approx0.090031 $$ ; $$ P(drive)\approx0.665241 $$ . **Answer.** drive has the highest probability in this toy learned model. **Limitations:** a standard static Word2Vec vector merges a word's senses; it has limited representation of subword morphology and out-of-vocabulary words; its local prediction objective does not directly fit a full global co-occurrence table. Similarity is learned from data and is not a guarantee of factual or causal understanding. CBOW = neighbours se missing centre. Skip-gram = centre se neighbours. --- ## TEE · Word analogies and sentence embeddings Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/formulas/f-vectors--u5-sentence/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/formulas/f-vectors/u5-sentence Formulas explained through full worked examples **A sentence embedding** represents an entire sentence by one fixed-length vector. A basic method averages word embeddings. More advanced encoders learn a sentence representation from context and task objectives. $$ v_{\text{sentence}}=\frac1n\sum_{i=1}^{n}v_{w_i}. $$ This means add corresponding coordinates and divide each coordinate by the number of words. Averaging ignores order: “dog bites man” and “man bites dog” have the same average if their words are identical. **Question. Average the word vectors [1,0], [0,2] and [2,1] to represent a three-word sentence.** #### Given Three two-dimensional vectors with equal weights. #### To find The two-dimensional sentence vector. #### Formula and why it applies Use coordinate-wise mean $$ v=(v_1+v_2+v_3)/3 $$ . #### Step 1. First coordinate $$ 1+0+2=3 $$ ; $$ 3/3=1 $$ . #### Step 2. Second coordinate $$ 0+2+1=3 $$ ; $$ 3/3=1 $$ . **Answer.** Sentence vector = [1,1]. **Question. Toy word vectors are king=[3,2], man=[1,0], woman=[1,1]. Find king−man+woman and compare to queen=[3,3].** #### Given All vectors use the same two-coordinate space. #### To find The analogy vector and its distance from queen. #### Formula and why it applies Vector arithmetic is coordinate by coordinate. Euclidean distance is the square root of the sum of squared coordinate differences. #### Step 1. Subtract man $$ [3-1,2-0]=[2,2] $$ . #### Step 2. Add woman $$ [2+1,2+1]=[3,3] $$ . #### Step 3. Compare with queen $$ \sqrt{(3-3)^2+(3-3)^2}=0 $$ . **Answer.** The toy analogy exactly equals queen. Real learned embeddings only approximate some analogies. **Contextual encoders** can assign “bank” different representations in different sentences. BERT and RoBERTa produce contextual token representations; obtaining strong sentence similarity vectors requires an appropriate pooling/training method, not simply assuming every raw token vector is a sentence embedding. Universal Sentence Encoder is designed to encode sentence-level meaning. Skip-thought predicts surrounding sentences; InferSent learns from natural-language-inference supervision. Sentence average: har column add karo, words ki count se divide. Word order ka loss yaad rakh. --- ## TEE · The GloVe objective and numerical interpretation Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/formulas/f-vectors--u5-glove-loss/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/formulas/f-vectors/u5-glove-loss Formulas explained through full worked examples $$ J=\sum_{i,j:X_{ij}>0}f(X_{ij})\left(w_i^T\widetilde w_j+b_i+\widetilde b_j-\ln X_{ij}\right)^2. $$ **What each piece means:** $$ w_i $$ is a target vector; $$ \widetilde w_j $$ is a separate context vector; the dot product estimates their compatibility; biases adjust word-level baselines. The desired score is the natural log of the observed count. Squaring penalizes both positive and negative errors. Weighting prevents unreliable rare pairs or extremely frequent pairs from dominating inappropriately. $$ f(x)=\begin{cases}(x/x_{\max})^\alpha,&0 0.0078125. These are unnormalised scores; the label is chosen by the larger score. Predicted label: spam. **Yaad rakho:** Class prior × har word ki class-likelihood; sabse bada score wins. --- ## TEE · Precision, recall, F1 and accuracy Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/formulas/f-applications--evaluation/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/formulas/f-applications/evaluation Formulas explained through full worked examples Evaluation compares predicted results with known answers. A **true positive** is a correctly predicted positive; a **false positive** is a negative item predicted positive; a **false negative** is a missed positive; a **true negative** is a correctly rejected negative. $$ \text{Precision}=\frac{TP}{TP+FP} $$ $$ \text{Recall}=\frac{TP}{TP+FN} $$ $$ F_1=\frac{2PR}{P+R}=\frac{2TP}{2TP+FP+FN} $$ $$ \text{Accuracy}=\frac{TP+TN}{TP+TN+FP+FN} $$ Precision asks how many predicted positives were correct. Recall asks how many actual positives were found. F1 balances them by a harmonic mean. If a denominator is zero, state the convention rather than dividing by zero. ### Example 1 · evaluate a classifier A binary text classifier gives TP=6, FP=2, FN=4, TN=8. Compute precision, recall, F1 and accuracy. **Given** TP=6; FP=2; FN=4; TN=8. **To find** Four evaluation measures. ### Formula and why it applies $$ P=\frac{TP}{TP+FP},\quad R=\frac{TP}{TP+FN} $$ $$ F_1=\frac{2TP}{2TP+FP+FN},\quad A=\frac{TP+TN}{TP+FP+FN+TN} $$ Use predicted positives in precision, actual positives in recall, and all examples in accuracy. ### Solution #### Step 1. Precision $$ P=\frac{6}{6+2}=0.750000 $$ #### Step 2. Recall $$ R=\frac{6}{6+4}=0.600000 $$ #### Step 3. F1 $$ F_1=\frac{12}{12+2+4}=0.666667 $$ #### Step 4. Accuracy $$ A=\frac{6+8}{6+2+4+8}=0.700000 $$ Precision 0.7500; recall 0.6000; F1 0.6667; accuracy 0.7000. ### Example 2 · evaluate a classifier A binary text classifier gives TP=9, FP=3, FN=1, TN=7. Compute precision, recall, F1 and accuracy. **Given** TP=9; FP=3; FN=1; TN=7. **To find** Four evaluation measures. ### Formula and why it applies $$ P=\frac{TP}{TP+FP},\quad R=\frac{TP}{TP+FN} $$ $$ F_1=\frac{2TP}{2TP+FP+FN},\quad A=\frac{TP+TN}{TP+FP+FN+TN} $$ Use predicted positives in precision, actual positives in recall, and all examples in accuracy. ### Solution #### Step 1. Precision $$ P=\frac{9}{9+3}=0.750000 $$ #### Step 2. Recall $$ R=\frac{9}{9+1}=0.900000 $$ #### Step 3. F1 $$ F_1=\frac{18}{18+3+1}=0.818182 $$ #### Step 4. Accuracy $$ A=\frac{9+7}{9+3+1+7}=0.800000 $$ Precision 0.7500; recall 0.9000; F1 0.8182; accuracy 0.8000. ### Example 3 · evaluate a classifier A binary text classifier gives TP=4, FP=0, FN=4, TN=12. Compute precision, recall, F1 and accuracy. **Given** TP=4; FP=0; FN=4; TN=12. **To find** Four evaluation measures. ### Formula and why it applies $$ P=\frac{TP}{TP+FP},\quad R=\frac{TP}{TP+FN} $$ $$ F_1=\frac{2TP}{2TP+FP+FN},\quad A=\frac{TP+TN}{TP+FP+FN+TN} $$ Use predicted positives in precision, actual positives in recall, and all examples in accuracy. ### Solution #### Step 1. Precision $$ P=\frac{4}{4+0}=1.000000 $$ #### Step 2. Recall $$ R=\frac{4}{4+4}=0.500000 $$ #### Step 3. F1 $$ F_1=\frac{8}{8+0+4}=0.666667 $$ #### Step 4. Accuracy $$ A=\frac{4+12}{4+0+4+12}=0.800000 $$ Precision 1.0000; recall 0.5000; F1 0.6667; accuracy 0.8000. --- ## TEE · Spelling correction and edit distance Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/formulas/f-applications--spelling/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/formulas/f-applications/spelling Formulas explained through full worked examples **Spelling correction** proposes the intended word or sentence when the observed text contains an error. A non-word error produces an invalid vocabulary item (“recieve”); a real-word error produces an existing word in the wrong context (“I will meat you”). Dictionary lookup can detect many non-words but cannot reliably detect real-word errors. Detect suspicious text → Generate candidates → Score edit + context fit → Choose correction ### Minimum edit distance Levenshtein distance is the minimum total cost of insertions, deletions and substitutions needed to transform one string into another. In the examples below, each edit costs 1; matching characters cost 0. Transposition is not a single Levenshtein operation. State different costs explicitly if the question gives them. $$ D(i,j)=\min\begin{cases}D(i-1,j)+1\\D(i,j-1)+1\\D(i-1,j-1)+[a_i\ne b_j]\end{cases} $$ Rows represent prefixes of the source; columns represent prefixes of the target. D(i,j) is the best cost for transforming the first i source characters to the first j target characters. Initialise D(i,0)=i and D(0,j)=j. The three choices are deletion, insertion and diagonal match/substitution. ### Example 1 · cat → cut Calculate the unit-cost Levenshtein distance from “cat” to “cut”. Show the dynamic-programming table. **Given** Source=cat; target=cut; insertion=deletion=substitution=1; match=0. **To find** Minimum edit cost. ### Formula and why it applies $$ D(i,j)=\min(D(i-1,j)+1,D(i,j-1)+1,D(i-1,j-1)+c) $$ c = 0 for matching characters, 1 otherwise. Compare all three possible ways to reach each cell. ### Solution #### Step 1. Initialise empty prefixes First row counts insertions; first column counts deletions. #### Step 2. Fill row 1: source character “c” $$ D(1,1)=\min(1+1,\ 1+1,\ 0+0)=0 $$ $$ D(1,2)=\min(2+1,\ 0+1,\ 1+1)=1 $$ $$ D(1,3)=\min(3+1,\ 1+1,\ 2+1)=2 $$ #### Step 3. Fill row 2: source character “a” $$ D(2,1)=\min(0+1,\ 2+1,\ 1+1)=1 $$ $$ D(2,2)=\min(1+1,\ 1+1,\ 0+1)=1 $$ $$ D(2,3)=\min(2+1,\ 1+1,\ 1+1)=2 $$ #### Step 4. Fill row 3: source character “t” $$ D(3,1)=\min(1+1,\ 3+1,\ 2+1)=2 $$ $$ D(3,2)=\min(1+1,\ 2+1,\ 1+1)=2 $$ $$ D(3,3)=\min(2+1,\ 2+1,\ 1+0)=1 $$ #### Step 5. Read the completed matrix | Source / target | ∅ | c | u | t | | --- | --- | --- | --- | --- | | ∅ | 0 | 1 | 2 | 3 | | c | 1 | 0 | 1 | 2 | | a | 2 | 1 | 1 | 2 | | t | 3 | 2 | 2 | 1 | Minimum edit distance = 1. ### Example 2 · at → cat Calculate the unit-cost Levenshtein distance from “at” to “cat”. Show the dynamic-programming table. **Given** Source=at; target=cat; insertion=deletion=substitution=1; match=0. **To find** Minimum edit cost. ### Formula and why it applies $$ D(i,j)=\min(D(i-1,j)+1,D(i,j-1)+1,D(i-1,j-1)+c) $$ c = 0 for matching characters, 1 otherwise. Compare all three possible ways to reach each cell. ### Solution #### Step 1. Initialise empty prefixes First row counts insertions; first column counts deletions. #### Step 2. Fill row 1: source character “a” $$ D(1,1)=\min(1+1,\ 1+1,\ 0+1)=1 $$ $$ D(1,2)=\min(2+1,\ 1+1,\ 1+0)=1 $$ $$ D(1,3)=\min(3+1,\ 1+1,\ 2+1)=2 $$ #### Step 3. Fill row 2: source character “t” $$ D(2,1)=\min(1+1,\ 2+1,\ 1+1)=2 $$ $$ D(2,2)=\min(1+1,\ 2+1,\ 1+1)=2 $$ $$ D(2,3)=\min(2+1,\ 2+1,\ 1+0)=1 $$ #### Step 4. Read the completed matrix | Source / target | ∅ | c | a | t | | --- | --- | --- | --- | --- | | ∅ | 0 | 1 | 2 | 3 | | a | 1 | 1 | 1 | 2 | | t | 2 | 2 | 2 | 1 | Minimum edit distance = 1. ### Example 3 · cats → cat Calculate the unit-cost Levenshtein distance from “cats” to “cat”. Show the dynamic-programming table. **Given** Source=cats; target=cat; insertion=deletion=substitution=1; match=0. **To find** Minimum edit cost. ### Formula and why it applies $$ D(i,j)=\min(D(i-1,j)+1,D(i,j-1)+1,D(i-1,j-1)+c) $$ c = 0 for matching characters, 1 otherwise. Compare all three possible ways to reach each cell. ### Solution #### Step 1. Initialise empty prefixes First row counts insertions; first column counts deletions. #### Step 2. Fill row 1: source character “c” $$ D(1,1)=\min(1+1,\ 1+1,\ 0+0)=0 $$ $$ D(1,2)=\min(2+1,\ 0+1,\ 1+1)=1 $$ $$ D(1,3)=\min(3+1,\ 1+1,\ 2+1)=2 $$ #### Step 3. Fill row 2: source character “a” $$ D(2,1)=\min(0+1,\ 2+1,\ 1+1)=1 $$ $$ D(2,2)=\min(1+1,\ 1+1,\ 0+0)=0 $$ $$ D(2,3)=\min(2+1,\ 0+1,\ 1+1)=1 $$ #### Step 4. Fill row 3: source character “t” $$ D(3,1)=\min(1+1,\ 3+1,\ 2+1)=2 $$ $$ D(3,2)=\min(0+1,\ 2+1,\ 1+1)=1 $$ $$ D(3,3)=\min(1+1,\ 1+1,\ 0+0)=0 $$ #### Step 5. Fill row 4: source character “s” $$ D(4,1)=\min(2+1,\ 4+1,\ 3+1)=3 $$ $$ D(4,2)=\min(1+1,\ 3+1,\ 2+1)=2 $$ $$ D(4,3)=\min(0+1,\ 2+1,\ 1+1)=1 $$ #### Step 6. Read the completed matrix | Source / target | ∅ | c | a | t | | --- | --- | --- | --- | --- | | ∅ | 0 | 1 | 2 | 3 | | c | 1 | 0 | 1 | 2 | | a | 2 | 1 | 0 | 1 | | t | 3 | 2 | 1 | 0 | | s | 4 | 3 | 2 | 1 | Minimum edit distance = 1. ### Noisy-channel correction $$ \hat w=\arg\max_w P(x\mid w)P(w) $$ x is the observed misspelling and w a candidate intended word. P(x | w) is the error-model likelihood; P(w) is the language-model prior. In context, replace the isolated-word prior with an appropriate contextual probability. A frequent word with a plausible typo may beat a rarer candidate, but context can reverse the ranking. ### Candidate ranking For observed “teh”, candidate “the” has prior 0.05 and error likelihood 0.02. Candidate “ten” has prior 0.01 and likelihood 0.01. Select the intended word. **Given** Two candidates and their prior/error-model likelihoods are supplied. **To find** Largest noisy-channel score. ### Formula and why it applies $$ s(w)=P(x\mid w)P(w) $$ Multiply likelihood by prior; the same evidence normaliser is irrelevant to ranking. ### Solution #### Step 1. Score the $$ s(\text{the})=0.02\times0.05=0.001 $$ #### Step 2. Score ten $$ s(\text{ten})=0.01\times0.01=0.0001 $$ #### Step 3. Compare 0.001 is ten times 0.0001. Choose “the”. --- ## TEE · Information retrieval Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/formulas/f-applications--retrieval/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/formulas/f-applications/retrieval Formulas explained through full worked examples **Information retrieval (IR)** locates and ranks items relevant to an information need, usually expressed as a query. The collection may contain documents, product descriptions or scientific papers. Retrieval estimates relevance; it does not itself establish that every retrieved claim is true. Documents → cleaning → Build index → Query → same representation → Match & rank → Return relevant results ### The inverted index An inverted index maps a term to the documents containing it, often with positions or counts. For D1 “cats drink milk” and D2 “dogs drink water”, the posting list for drink is {D1,D2}, while milk maps only to D1. This avoids rescanning every document for every query. | Term | Posting list | | --- | --- | | cats | D1 | | drink | D1, D2 | | milk | D1 | | dogs | D2 | | water | D2 | ### Matching and ranking Boolean retrieval combines term membership using AND, OR and NOT. Vector-space retrieval represents documents and queries as weighted vectors, for example with TF-IDF, and compares them with cosine similarity. Learned retrieval can use dense embeddings for meaning-based matching. The representation used at query time must be compatible with the document index. ### Search engines and recommendation A search engine receives an explicit query and ranks matching indexed pages. A recommender may derive an implicit query from a user’s liked items or profile and retrieve similar candidates. Both require candidate retrieval and ranking, but the source of the information need differs. ### Evaluate the results For retrieval, TP is the number of retrieved relevant documents, FP retrieved irrelevant documents, and FN relevant documents not retrieved. Precision and recall use the same formulas as classification. Ranking measures also consider the positions of relevant items; a relevant result at rank 1 is more useful than one far below. ### Retrieval evaluation A search system retrieves 8 documents; 6 are relevant. There are 10 relevant documents in the collection. Find precision, recall and F1. **Given** Retrieved=8; relevant retrieved=6; all relevant=10. **To find** P, R and F1. ### Formula and why it applies $$ P=\frac{\text{relevant retrieved}}{\text{retrieved}},\quad R=\frac{\text{relevant retrieved}}{\text{all relevant}} $$ $$ F_1=\frac{2PR}{P+R} $$ The denominator is 8 for precision and 10 for recall because they ask different questions. ### Solution #### Step 1. Identify the confusion counts TP=6; FP=8−6=2; FN=10−6=4. #### Step 2. Precision $$ P=\frac68=0.75 $$ #### Step 3. Recall $$ R=\frac6{10}=0.6 $$ #### Step 4. F1 $$ F_1=\frac{2(0.75)(0.6)}{0.75+0.6}=\frac{0.9}{1.35}=\frac23\approx0.666667 $$ Precision=0.75, recall=0.60, F1≈0.666667. **Yaad rakho:** Precision: jo laaye, unmein kitne sahi? Recall: jo sahi the, unmein kitne laaye? --- ## TEE · N-gram interpolation: weighted evidence from several histories Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/formulas/f-deep-u3-smoothing-extensions--interpolation/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/formulas/f-deep-u3-smoothing-extensions/interpolation Unit 3 PDF p147 names these methods. Neither supplied TEE paper asks their numerical calculation; the worked values below are authored examples. **Faculty topic; not an observed numerical PYQ.** Unit 3 PDF p147 names these methods. Neither supplied TEE paper asks their numerical calculation; the worked values below are authored examples. **Interpolation combines several context lengths for every prediction.** A trigram estimate is specific but may have little evidence. Bigram and unigram estimates use less context and can support unseen longer sequences. A weighted mixture combines their information. $$ P_{\mathrm{mix}}(w\mid u,v)=\lambda_3P(w\mid u,v)+\lambda_2P(w\mid v)+\lambda_1P(w) $$ $$ \lambda_1,\lambda_2,\lambda_3\ge0,\qquad\lambda_1+\lambda_2+\lambda_3=1. $$ Each component must be a normalized distribution over the same vocabulary. Then summing the mixture over all possible words gives λ₁+λ₂+λ₃=1. A zero higher-order count can be rescued by positive lower-order evidence. Unlike simple backoff, interpolation can combine all levels even when the trigram was seen. **Question. For predicting tea after I like, suppose trigram, bigram and unigram probabilities are 0, 0.2 and 0.1. Use weights λ₃=0.5, λ₂=0.3 and λ₁=0.2 to calculate the interpolated probability. Repeat for coffee with probabilities 0.4, 0.1 and 0.05. [Authored worked example]** ### Answer #### Given The three distributions use one vocabulary. Nonnegative weights sum to 1. These are supplied probabilities; no count estimation is needed. #### To find Pmix(tea | I,like) and Pmix(coffee | I,like). #### Formula and why it applies $$ P_{mix}=\lambda_3P_3+\lambda_2P_2+\lambda_1P_1 $$ We add weighted alternatives from models, rather than multiply sequential word events. #### Solution ##### Step 1. Check the weights $$ 0.5+0.3+0.2=1. $$ ##### Step 2. Calculate the three tea contributions $$ 0.5(0)=0,\qquad0.3(0.2)=0.06,\qquad0.2(0.1)=0.02. $$ ##### Step 3. Add the tea contributions $$ P_{mix}(tea)=0+0.06+0.02=0.08. $$ ##### Step 4. Calculate the coffee contributions $$ 0.5(0.4)=0.20,\qquad0.3(0.1)=0.03,\qquad0.2(0.05)=0.01. $$ ##### Step 5. Add and compare $$ P_{mix}(coffee)=0.20+0.03+0.01=0.24. $$ Coffee gets three times tea’s probability in this model. Other vocabulary outcomes carry the remaining mass; these two displayed entries are not the whole distribution. **Final answer.** Tea=0.08 and coffee=0.24. A zero trigram probability does not force a zero mixture. **Yaad rakho:** Interpolation = sab context-lengths ka weighted mix. Word-sequence product aur model-mixture sum alag operations hain. --- ## TEE · Backoff and the important score-versus-probability distinction Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/formulas/f-deep-u3-smoothing-extensions--stupid-backoff/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/formulas/f-deep-u3-smoothing-extensions/stupid-backoff Unit 3 PDF p147 names these methods. Neither supplied TEE paper asks their numerical calculation; the worked values below are authored examples. **Faculty topic; not an observed numerical PYQ.** Unit 3 PDF p147 names these methods. Neither supplied TEE paper asks their numerical calculation; the worked values below are authored examples. **Backoff uses a shorter history when the longer-history estimate lacks evidence.** A properly normalized backoff model discounts some seen-event probability and allocates the remaining mass across unseen continuations using lower-order probabilities and normalization factors. **Stupid Backoff** is a specifically named efficient scoring method in the slides’ list. It uses an observed n-gram relative frequency when available; otherwise it multiplies the shorter-history score by a fixed penalty, commonly 0.4. Its output is generally an **unnormalized score**, not a conditional probability distribution. Do not insert raw Stupid Backoff scores into standard probability-based perplexity and call the result a normalized-model perplexity. $$ S(w\mid h)=\begin{cases}C(h,w)/C(h),&C(h,w)>0\\ \alpha S(w\mid h'),&C(h,w)=0.\end{cases} $$ Here h′ drops the oldest history word; α is the backoff penalty. At the unigram base, use C(w)/N. The letter α here is a penalty, not the forward-HMM variable introduced later. **Question. A bigram scorer has complete vocabulary {a,b}. After h, C(h,a)=2 and C(h,b)=0. The unigram probabilities are P(a)=P(b)=0.5. Apply Stupid Backoff with penalty 0.4 and show whether the scores form a normalized row. [Authored worked example]** ### Answer #### Given C(h)=2. The a bigram is observed; the b bigram is unseen. Unigram probabilities are supplied. #### To find S(a|h), S(b|h), their sum, and whether they are probabilities. #### Formula and why it applies Use relative frequency for an observed pair and α times the unigram base for an unseen pair. The method does not discount a’s observed score in this example. #### Solution ##### Step 1. Score the observed pair $$ S(a\mid h)=2/2=1. $$ ##### Step 2. Back off for the unseen pair $$ S(b\mid h)=0.4(0.5)=0.2. $$ ##### Step 3. Add the complete row $$ 1+0.2=1.2\ne1. $$ ##### Step 4. Interpret the failure of normalization The scores can rank a above b, but the raw row cannot be called a probability distribution. Merely labelling the method “smoothing” does not change this. **Final answer.** Scores are 1 and 0.2; their sum is 1.2. They are useful ranking scores under this rule, not normalized probabilities. | Method | Operation | Normalization | | --- | --- | --- | | Add-one/add-k | Adjust all allowed outcome counts and the total | Normalized with a fixed vocabulary | | Interpolation | Mix normalized distributions using nonnegative weights summing to 1 | Normalized | | Normalized backoff | Discount seen events and distribute remaining mass over unseen events | Normalized when its factors are correctly computed | | Stupid Backoff | Keep seen relative frequencies; penalize shorter-history scores for unseen n-grams | Generally not normalized | Scope: Unit 3 slide 147 lists Stupid Backoff and n-gram interpolation; the class calculations emphasize Laplace. These short worked extensions explain the listed methods without claiming extra official exam weight. --- ## TEE · Prepare a grammar for CYK / CKY Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/formulas/f-deep-u4-forward-cky--cnf/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/formulas/f-deep-u4-forward-cky/cnf Unit 4 PDF p105 names CYK. The supplied papers ask top-down / bottom-up CFG trees, not CKY charts. These traces explain the taught topic. **Faculty topic; not an observed numerical PYQ.** Unit 4 PDF p105 names CYK. The supplied papers ask top-down / bottom-up CFG trees, not CKY charts. These traces explain the taught topic. **CYK and CKY are two names for the same dynamic-programming parsing family.** A chart stores which grammatical categories can generate each contiguous span of the sentence. Small spans are solved first and reused to build larger spans. Basic CKY uses a context-free grammar in Chomsky normal form (CNF): a production has either two nonterminals on its right, A→B C, or one terminal, A→word. Empty-string handling is a separate special case; the examples here have no empty production. | Rule | Basic CKY status | Reason | | --- | --- | --- | | S → NP VP | Allowed | Two nonterminals | | N → cat | Allowed | One terminal | | NP → Det Adj N | Convert first | Three nonterminals | | NP → N | Eliminate or handle unary closure | One nonterminal, not a lexical terminal | | VP → eats NP | Convert first | Mixed terminal and nonterminal | **Question. Convert NP→Det Adj N, NP→N, N→cat and VP→eats NP into basic CNF-compatible productions. Preserve their possible word yields. [Authored worked example]** ### Answer #### Given The symbols NP,Det,Adj,N,VP are nonterminals; cat and eats are terminals. Other needed lexical rules remain as supplied. #### To find Equivalent binary or terminal productions for these cases. #### Formula and why it applies Introduce helper categories for a long right-hand side or a terminal mixed with nonterminals. Eliminate a unary nonterminal rule by inheriting the reachable non-unit productions. #### Solution ##### Step 1. Binarize the three-part NP Introduce X: NP→Det X and X→Adj N. X is a helper, not a word; the terminal order remains Det, Adj, N. ##### Step 2. Remove the unit production NP→N and N→cat allow NP to derive cat. Add NP→cat, retain N→cat where needed, and remove NP→N. With more rules reachable through unit chains, inherit every corresponding non-unit rule, not only the first one. ##### Step 3. Replace the mixed terminal Introduce Vlex→eats and replace VP→eats NP by VP→Vlex NP. ##### Step 4. Read the converted forms NP→Det X | cat; X→Adj N; N→cat; VP→Vlex NP; Vlex→eats. All displayed productions are binary nonterminal or single-terminal rules. **Final answer.** The converted rules preserve the original word sequences while satisfying the displayed basic CKY forms. Helper nodes change tree shape; they can be removed when presenting the original grammar’s tree. General CNF conversion may also require eliminating nullable productions and useless symbols while preserving the intended language. Do not claim that simply splitting long rules handles every arbitrary CFG. An Earley parser instead supports general CFG rules using prediction, scanning and completion; it combines top-down expectations with bottom-up evidence. LR and operator-precedence parsers are other methods named in the faculty list, with grammar restrictions of their own. --- ## TEE · CYK / CKY: complete charts, backpointers and ambiguity Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/formulas/f-deep-u4-forward-cky--cky/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/formulas/f-deep-u4-forward-cky/cky Unit 4 PDF p105 names CYK. The supplied papers ask top-down / bottom-up CFG trees, not CKY charts. These traces explain the taught topic. **Faculty topic; not an observed numerical PYQ.** Unit 4 PDF p105 names CYK. The supplied papers ask top-down / bottom-up CFG trees, not CKY charts. These traces explain the taught topic. Number the words 1 to n. Let **T[i,j]** contain all categories that derive the span from word i through word j, inclusive. Start with one-word cells from lexical rules. For every longer span try **every split k** between i and j: if B belongs to T[i,k], C belongs to T[k+1,j], and A→B C is a rule, put A in T[i,j]. Store the rule and split as backpointers when a tree is required. $$ A\in T[i,j]\quad\text{if}\quad A\to BC,\ B\in T[i,k],\ C\in T[k+1,j],\ i\le kj); ∅ is an evaluated but empty valid span. Read the chart by increasing span length, not left-to-right across a row. ##### Step 7. Follow the stored backpointers S[1,5]→NP[1,2] VP[3,5]; NP[1,2]→Det[1,1] N[2,2]; VP[3,5]→V[3,3] NP[4,5]; NP[4,5]→Det[4,4] N[5,5]. ``` S ├── NP │ ├── Det: the │ └── N: cat └── VP ├── V: eats └── NP ├── Det: a └── N: fish ``` **Final answer.** Accepted because S∈T[1,5]. The reconstructed tree’s leaves are exactly the five input words in order. **Question. A grammar has S→A A, A→A A and A→a. Use CKY for “a a a” and explain how two parse trees are preserved. [Authored worked example]** ### Answer #### Given Three identical terminal tokens. S is the start symbol. No unit or empty rules are present. #### To find Chart categories and both successful split choices for the root. #### Formula and why it applies A chart cell is a set of possible categories. For each category keep all successful backpointers when the task requires every parse. #### Solution ##### Step 1. Initialize the diagonal Each of T[1,1], T[2,2], T[3,3] is {A}, from A→a. ##### Step 2. Fill two-word spans T[1,2] and T[2,3] each contain {A,S}: A+A can use A→A A or S→A A. ##### Step 3. Try both root splits k=1: A[1,1] + A[2,3] permits S. k=2: A[1,2] + A[3,3] also permits S. The complete cell is {A,S}, but S has two distinct backpointer alternatives. ##### Step 4. Draw both bracketed trees [S [A a] [A [A a] [A a]]] and [S [A [A a] [A a]] [A a]]. Both yield a a a; the constituent grouping differs. **Final answer.** Accepted, with two S-rooted parses. A category appearing once in a set does not mean there is only one derivation. **Question. With S→NP VP, NP→Det N, VP→V NP; Det→the|a; N→cat|fish; V→eats, test “the eats cat” using CKY. [Authored worked example]** ### Answer #### Given Tokens are the/eats/cat, with no punctuation. Use exactly the supplied grammar. #### To find Whether the grammar accepts the whole input. #### Formula and why it applies A parse exists only if the complete chart cell contains the start symbol. #### Solution ##### Step 1. Initialize the diagonal T[1,1]={Det}; T[2,2]={V}; T[3,3]={N}. ##### Step 2. Try length-two combinations Det+V matches no binary rule; V+N also matches no rule (VP requires V+NP). Both two-word cells are empty. ##### Step 3. Try the two full-span splits Each split has an empty child, so T[1,3]=∅ and S is absent. **Final answer.** Rejected by this grammar. Do not invent NP→N to force acceptance. Grammar rejection and a universal judgment about every possible English grammar are different claims. **Yaad rakho:** Diagonal par words ke tags; phir chhote spans jodo. Har split dekho. Full sentence cell mein S aaya toh parse possible. --- ## TEE · TEE formula sheet: vectors and applications Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/formulas/tee-formula-index--all-tee/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/formulas/tee-formula-index/all-tee TEE · used together with the M2 formula sheet These formulas extend the M2 sheet to the complete TEE syllabus. Each coordinate, token-count convention and log base must be defined before calculation. Worked-example chapters follow this sheet. ### Type–token ratio $$ TTR=\frac{\#\text{types}}{\#\text{tokens}} $$ Distinct vocabulary types divided by all token occurrences. Often expressed as a percentage. ### Normalized term frequency $$ TF(t,d)=\frac{C(t,d)}{\sum_w C(w,d)} $$ The slides use count of the term divided by the document token total after the specified preprocessing. ### Inverse document frequency $$ IDF(t)=\log_{10}\frac{D}{df(t)} $$ D is the number of documents; df counts documents containing the term at least once. This is the course slide convention. ### TF–IDF $$ TF\!\operatorname{-}\!IDF(t,d)=TF(t,d)IDF(t) $$ Multiply the local frequency weight by the global rarity weight. ### Dot product $$ a\cdot b=\sum_i a_ib_i $$ Multiply corresponding coordinates, then add. Vectors need the same coordinate order. ### Vector length $$ \|a\|_2=\sqrt{\sum_i a_i^2} $$ Square each coordinate, add, then take the square root. ### Cosine similarity $$ \cos(a,b)=\frac{\sum_i a_ib_i}{\sqrt{\sum_i a_i^2}\sqrt{\sum_i b_i^2}} $$ Compare direction; undefined for a zero-length vector. ### Jaccard similarity $$ J(A,B)=\frac{|A\cap B|}{|A\cup B|} $$ Set membership, not repeated word counts. Jaccard distance = 1 − J. ### Manhattan distance $$ d_1(a,b)=\sum_i|a_i-b_i| $$ Add coordinate-wise absolute differences. ### Euclidean distance $$ d_2(a,b)=\sqrt{\sum_i(a_i-b_i)^2} $$ Square differences, add, square root. ### Minkowski distance $$ d_p(a,b)=\left(\sum_i|a_i-b_i|^p\right)^{1/p} $$ p≥1; Manhattan p=1, Euclidean p=2. ### Mean sentence embedding $$ v_{sentence}=\frac1n\sum_{i=1}^n v_{w_i} $$ Add word vectors coordinate by coordinate; divide each coordinate by n. This baseline loses word order. ### GloVe objective $$ J=\sum_{i,j}f(X_{ij})(w_i^T\widetilde w_j+b_i+\widetilde b_j-\log X_{ij})^2 $$ X is a co-occurrence count. Compute the residual, square it and apply the weighting function. Use observed positive counts. ### Precision and recall $$ P=\frac{TP}{TP+FP},\qquad R=\frac{TP}{TP+FN} $$ Precision checks predicted positives; recall checks actual positives. ### F1 score $$ F_1=\frac{2PR}{P+R}=\frac{2TP}{2TP+FP+FN} $$ Harmonic mean of precision and recall. ### Accuracy $$ A=\frac{TP+TN}{TP+TN+FP+FN} $$ All correct predictions divided by all predictions. ### Noisy-channel correction $$ \hat w=\arg\max_w P(x\mid w)P(w) $$ Rank intended word w using observed typo x; likelihood × prior. ### Unit-cost edit distance $$ D(i,j)=\min\{D(i-1,j)+1,D(i,j-1)+1,D(i-1,j-1)+[a_i\ne b_j]\} $$ Initialise first row/column by insertion/deletion counts; fill all cells using min. --- ## TEE · Q1A · NLU and NLG Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/final-2025--final-2025-q1a/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/final-2025/final-2025-q1a Actual final examination · 2 December 2025 · 100 marks · 3 hours Question. Differentiate between Natural Language Understanding (NLU) and Natural Language Generation (NLG) in NLP with suitable examples. [5 marks] ### Answer **Natural Language Understanding (NLU)** interprets language to identify its meaning, intent and relevant information. **Natural Language Generation (NLG)** produces language that expresses information in a useful form. | Basis | NLU | NLG | | --- | --- | --- | | Direction | Language → meaning or structured information | Meaning or structured information → language | | Typical input | A user utterance, document or sentence | Facts, a database record, a prompt or a representation | | Typical output | Intent, entities, sentiment or a semantic representation | A sentence, explanation, summary or dialogue response | | Main difficulty | Resolving ambiguity and context | Choosing accurate content and producing coherent, fluent wording | ### Example: a weather assistant User: “Will it rain in Mumbai tomorrow?” NLU recognises a weather-query intent, the location Mumbai and the date tomorrow. After a weather service supplies the forecast, NLG produces “Rain is likely in Mumbai tomorrow; carry an umbrella.” User language → NLU: identify request → Retrieve forecast → NLG: express answer The two functions often work together in a chatbot. Understanding the request and wording the reply are separate tasks; fluent generation alone does not prove correct understanding. **Yaad rakho:** NLU mein meaning nikalo; NLG mein meaning ko sentence banao. --- ## TEE · Q1B · Chunking and chinking Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/final-2025--final-2025-q1b/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/final-2025/final-2025-q1b Actual final examination · 2 December 2025 · 100 marks · 3 hours Question. Explain the concepts of chunking and chinking in Natural Language Processing (NLP). Discuss how Part-of-Speech (POS) tagging supports these processes, and illustrate your explanation with a suitable example. [5 marks] ### Answer **Chunking** groups adjacent tagged words into shallow phrases, such as noun phrases, without building a complete sentence parse. **Chinking** removes a specified sequence of tags or words from a chunk, splitting a larger candidate chunk when required. ### POS tags provide the pattern A POS tag describes a grammatical category: DT = determiner, JJ = adjective, NN = singular noun, VBD = past-tense verb and IN = preposition. A noun-phrase rule can be written as `NP: {
?*+}`: an optional determiner, zero or more adjectives, followed by one or more nouns. ### Worked example | Word | The | small | cat | sat | on | the | mat | | --- | --- | --- | --- | --- | --- | --- | --- | | Tag | DT | JJ | NN | VBD | IN | DT | NN | Applying the NP rule gives **[The small cat] sat on [the mat]**. The first chunk contains DT–JJ–NN; the second contains DT–NN. “sat on” is not included because it does not match the noun-phrase rule. For chinking, begin with one broad chunk: **[The small cat sat on the mat]**. Remove the VBD and IN sequence “sat on”, for example with `}+{`. The result is **[The small cat] sat on [the mat]**. POS tagging therefore supplies grammatical labels; chunking uses labels to select phrases, while chinking uses labels to exclude unwanted material. Incorrect tags can lead to incorrect chunks. **Yaad rakho:** Chunking = phrase ko jodo; chinking = beech ka unwanted part nikalo. --- ## TEE · Q1C · Hyponymy, hypernymy and WSD Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/final-2025--final-2025-q1c/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/final-2025/final-2025-q1c Actual final examination · 2 December 2025 · 100 marks · 3 hours Question. Define hyponymy and hypernymy with suitable linguistic examples. How can hypernymy be applied in Word Sense Disambiguation (WSD) to resolve the correct meaning of polysemous words? Provide an example? [5 marks] ### Answer **Hyponymy** is the “is-a-kind-of” relation from a specific concept to a broader concept. **Hypernymy** is the inverse relation from the broader concept to the specific one. “Rose” is a hyponym of “flower”; “flower” is a hypernym of “rose”. rose / tulip → flower → plant The arrow points towards a broader category. Hyponymy is not a part–whole relation: a wheel is part of a car, but it is not a kind of car. ### Using the hierarchy for WSD WSD selects the intended sense of a word from its context. Obtain candidate senses from a lexical resource such as WordNet, inspect their hypernym chains, and favour the chain most compatible with nearby words. 1. Sentence: “The mouse clicked the icon.” 2. Candidate 1: mouse → rodent → mammal → animal. Candidate 2: mouse → pointing device → computer peripheral. 3. The context words “clicked” and “icon” belong to the computer-use domain. They support the device hierarchy, rather than the animal hierarchy. 4. Select the computer-pointing-device sense. In “The mouse ate cheese,” the animal hierarchy is more compatible. Hypernym evidence supplies semantic generalisation; it is a useful signal rather than a guarantee, because short or unusual contexts may still be ambiguous. **Yaad rakho:** Hyponym specific hai; hypernym uska bada category hai. --- ## TEE · Q1D · Pragmatics and discourse Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/final-2025--final-2025-q1d/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/final-2025/final-2025-q1d Actual final examination · 2 December 2025 · 100 marks · 3 hours Question. Compare pragmatic and discourse analysis with the help of appropriate examples. [5 marks] ### Answer **Pragmatic analysis** interprets an utterance using the situation, speaker intention, shared knowledge and conversational expectations. **Discourse analysis** studies how multiple utterances form a connected text or conversation. | Basis | Pragmatic analysis | Discourse analysis | | --- | --- | --- | | Focus | Intended meaning in context | Relationships and coherence across utterances | | Useful information | Speaker, listener, place, time and common knowledge | Previous sentences, entities, topics and rhetorical relations | | Typical tasks | Interpret requests, implicatures and deixis | Resolve references and analyse cohesion, coherence and discourse structure | | Example | “It is cold here” may be a request to close a window | “Riya bought a book. She read it.” links She to Riya and it to book | In “Can you pass the salt?”, literal syntax asks about ability; pragmatic interpretation usually identifies a polite request. In “The road flooded. Therefore, the school closed,” discourse analysis identifies a cause–result link between the sentences. The levels interact: resolving “she” across a conversation may require both earlier text and real-world context. They are complementary rather than mutually exclusive. **Yaad rakho:** Pragmatics: bolne ka intention. Discourse: sentences ka connection. --- ## TEE · Q2A · Morpheme, lexeme and NER Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/final-2025--final-2025-q2a/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/final-2025/final-2025-q2a Actual final examination · 2 December 2025 · 100 marks · 3 hours Question. i. Differentiate between Morpheme and Lexeme with examples. ii. Discuss the significance of NER in three different NLP applications. [10 marks] ### Answer ### i. Morpheme and lexeme A **morpheme** is the smallest linguistic unit that contributes meaning or grammatical function. A **lexeme** is an abstract vocabulary item representing a family of inflected word forms. | Basis | Morpheme | Lexeme | | --- | --- | --- | | Unit | A meaningful component of a word | An abstract dictionary vocabulary item | | Examples | un-, happy, -ness; cat and plural -s | RUN covers run, runs, ran and running | | Independence | Free morphemes can stand alone; bound morphemes cannot | A lexeme is realised through one or more word forms | | NLP relevance | Morphological segmentation, stemming and analysis | Lemmatization and lexical meaning | “Cats” contains two morphemes: cat + plural -s. “cat” and “cats” are forms of the lexeme CAT. “Unhappiness” can be segmented into un + happy + ness, although its spelling changes at the boundary. A morpheme and a syllable are different: syllables concern pronunciation, while morphemes concern meaning or grammar. ### ii. Named Entity Recognition in applications **NER** locates spans that name entities and assigns types such as PERSON, ORGANIZATION, LOCATION, DATE or domain-specific categories. | Application | Entities and example | Why NER matters | | --- | --- | --- | | News information extraction | “Asha joined Infosys in Pune.” → Asha/PERSON, Infosys/ORG, Pune/LOCATION | Creates searchable people–organisation–location records and supports later relation extraction | | Healthcare document processing | Find disease, drug and dosage spans in a clinical note | Supports record indexing and medication extraction; specialist labels and validation are required | | Customer support and commerce | Extract product names, dates and order identifiers from “My Pixel arrived damaged on Monday.” | Routes complaints, links the correct product and fills structured support fields | NER answers “Which entity spans occur here, and what type are they?” It does not by itself establish every relation between those entities or resolve all names to unique real-world identities. **Yaad rakho:** Morpheme = word ka meaningful piece; lexeme = word family. NER = naam dhoondho aur type do. --- ## TEE · Q2B · Laplace-smoothed bigram probability Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/final-2025--final-2025-q2b/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/final-2025/final-2025-q2b Actual final examination · 2 December 2025 · 100 marks · 3 hours Question. Consider the following training corpus: 1. artists paint murals in city 2. musicians play songs in city 3. painters paint murals on walls 4. signs are near city Using a bigram language model with Laplace smoothing, evaluate the probability of the test sentence: “artists paint murals near city”. Assume the and for each sentence. [10 marks] ### Answer ### Given - artists paint murals in city - musicians play songs in city - painters paint murals on walls - signs are near city Test: artists paint murals near city. Add one start token and one end token to every training and test sentence. ### To find The Laplace-smoothed bigram probability of the complete test sentence, including its end token. ### Formula: which one and why $$ P_{+1}(w_i\mid w_{i-1})=\frac{C(w_{i-1},w_i)+1}{C(w_{i-1})+V} $$ C(previous,current) counts that adjacent pair in the training corpus. C(previous) counts its occurrences as a history. V is the vocabulary size. Add-one smoothing is necessary because an unseen pair must not force the whole sentence probability to zero. $$ P(w_1,\ldots,w_n,\langle/s\rangle\mid\langle s\rangle)=\prod_{i=1}^{n+1}P(w_i\mid w_{i-1}) $$ ### Solution ### Step 1. Fix the token and vocabulary convention Lowercase words and remove the final punctuation. The supplied course key counts both boundary tokens in V. Following that convention: are, artists, city, in, murals, musicians, near, on, paint, painters, play, signs, songs, walls, , gives **V = 16**. The start marker supplies history; it is not an extra probability factor before the first word. ### Step 2. List every required transition The complete test chain is → artists → paint → murals → near → city → . ### Step 3. Count first, then add one | Required bigram | History count | Pair count | Smoothed probability | | --- | --- | --- | --- | | → artists | 4 | 1 | (1+1)/(4+16) = 1/10 | | artists → paint | 1 | 1 | (1+1)/(1+16) = 2/17 | | paint → murals | 2 | 2 | (2+1)/(2+16) = 1/6 | | murals → near | 2 | 0 | (0+1)/(2+16) = 1/18 | | near → city | 1 | 1 | (1+1)/(1+16) = 2/17 | | city → | 3 | 3 | (3+1)/(3+16) = 4/19 | Count only adjacent words, never words that merely appear in the same sentence. The unseen pairs have count zero, so their numerator becomes one. ### Step 4. Multiply all six probabilities $$ P=\frac{2}{20}\times\frac{2}{17}\times\frac{3}{18}\times\frac{1}{18}\times\frac{2}{17}\times\frac{4}{19} $$ Start with subtotal 1. At each row, multiply the previous subtotal by exactly one next factor. Simplify the fraction after multiplication, but do not round it. | Factor being included | Previous subtotal × next factor = new subtotal | | --- | --- | | 1. → artists | $$ 1\times\frac{1}{10}=\frac{1}{10} $$ | | 2. artists → paint | $$ \frac{1}{10}\times\frac{2}{17}=\frac{1}{85} $$ | | 3. paint → murals | $$ \frac{1}{85}\times\frac{1}{6}=\frac{1}{510} $$ | | 4. murals → near | $$ \frac{1}{510}\times\frac{1}{18}=\frac{1}{9180} $$ | | 5. near → city | $$ \frac{1}{9180}\times\frac{2}{17}=\frac{1}{78030} $$ | | 6. city → | $$ \frac{1}{78030}\times\frac{4}{19}=\frac{2}{741285} $$ | The last subtotal is the complete product. Only now convert the fraction to a decimal. $$ P=\frac{2}{741285}\approx 2.69801763\times10^{-6} $$ ### Step 5. Interpret the result This is a joint sentence probability, so a small value is normal. Do not add transition probabilities; each word must occur in sequence, so the chain multiplies them. ### Final answer **P ≈ 2.69801763 × 10-6** with the stated course vocabulary convention. **Yaad rakho:** Pair count mein 1 jodo, previous-word count mein V; phir saare fractions multiply karo. **Convention note:** Some implementations exclude from the predicted vocabulary. Their V is one smaller and their numerical probability differs. In an exam, state V explicitly and follow the convention specified in the question or class solution. --- ## TEE · Q3A · Ambiguity and bottom-up parsing Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/final-2025--final-2025-q3a/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/final-2025/final-2025-q3a Actual final examination · 2 December 2025 · 100 marks · 3 hours Question. i. Explain the various types of ambiguities encountered in Natural Language Processing (NLP). Illustrate each type with suitable examples. ii. Compare top-down and bottom-up parsing and apply bottom-up syntax parsing approach to the sentence “A cat chased the mouse” and draw a parse tree using the grammar rules: S → NP VP; NP → Pronoun | Proper-Noun | Det Nominal; Nominal → Noun | Nominal Noun | Nominal PP; VP → Verb | Verb NP | VP PP; PP → Prep NP. Lexicon: Det → {A, the}; Noun → {cat, mouse}; Verb → {chased}. [10 marks] ### Answer ### i. Types of ambiguity Ambiguity exists when the same language expression permits more than one interpretation. The correct interpretation usually requires grammatical, semantic or contextual evidence. | Type | Meaning | Example and readings | | --- | --- | --- | | Lexical | One word has multiple meanings | “She visited the bank”: financial institution or riverbank | | Syntactic | The sentence permits different structural attachments | “I saw the man with a telescope”: I used the telescope, or the man had it | | Semantic / scope | A structured sentence permits different interpretations | “Every student read a book”: one shared book, or possibly a different book per student | | Referential | A referring expression has more than one candidate referent | “Asha told Riya that she won”: she could refer to Asha or Riya | | Pragmatic | Intention depends on the situation | “Can you open the door?” can ask about ability or request an action | ### ii. Parsing approaches | Aspect | Top-down | Bottom-up | | --- | --- | --- | | Start | The start symbol S | The input words | | Operation | Expand a nonterminal using grammar productions | Combine words and constituents using productions in reverse | | Success | Leaves match the complete input | All input is reduced to S | | Possible difficulty | Naive recursive descent can loop on left recursion or need backtracking | A naive shift-reduce parser may face shift/reduce conflicts | ### Given The five tokens are A / cat / chased / the / mouse. The supplied grammar permits determiner + nominal noun phrases and a verb + noun phrase verb phrase. ### To find A bottom-up sequence of reductions and a parse tree whose root is S and whose leaves preserve the input order. ### Rule: which one and why A production A → B C permits B C to be reduced to A. Bottom-up parsing begins with the words; lexical rules assign their categories, and phrase rules combine those categories. ### Step 1. Assign lexical categories A → Det; cat → Noun; chased → Verb; the → Det; mouse → Noun. ### Step 2. Build the first noun phrase cat is Noun. Reduce Noun to Nominal using Nominal → Noun. Then reduce Det Nominal, representing “A cat”, to NP. ### Step 3. Build the object noun phrase mouse is Noun. Reduce it to Nominal. Reduce “the” + “mouse”, now Det Nominal, to NP. ### Step 4. Build the verb phrase “chased” is Verb and “the mouse” is NP. Reduce Verb NP to VP. ### Step 5. Build the sentence The remaining constituents are NP (“A cat”) and VP (“chased the mouse”). Apply S → NP VP in reverse to obtain S. Every input token is used exactly once. SNPVPDetNominalVerbNPNounDetNominalNounAcatchasedthemouse Bottom-up parse tree for A cat chased the mouse. **Final answer:** [S [NP [Det A] [Nominal [Noun cat]]] [VP [Verb chased] [NP [Det the] [Nominal [Noun mouse]]]]]. **Yaad rakho:** Bottom-up: words se phrases, phrases se S. Top-down: S se words. --- ## TEE · Q3B · Co-occurrence matrix and cosine Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/final-2025--final-2025-q3b/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/final-2025/final-2025-q3b Actual final examination · 2 December 2025 · 100 marks · 3 hours Question. Consider the given paragraph: The teacher wrote on the board. The students listened to the teacher. The board was full of notes. The student asked a question. The teacher answered from the book. Target words: teacher, student, board, book. Context words: wrote, listened, notes, question. i. Construct a co-occurrence matrix for the target and context words. Take the window size as one whole sentence. ii. Evaluate the similarity between the vectors of the target words “teacher-student” and “teacher-board” using the cosine similarity. iii. Plot the target word vectors by considering only two dimensions: “wrote” and “notes”. [10 marks] ### Answer ### Given Five sentences, four target rows and four context columns. Treat students and student as the same target lemma, and count each target–context pair once when both occur in a sentence. ### To find The 4 × 4 sentence-window co-occurrence matrix, two cosine similarities and the requested 2-D plot. ### Formula: which one and why $$ M_{ij}=\sum_{s=1}^{5}\mathbf{1}[\text{target}_i\text{ and context}_j\text{ occur in sentence }s] $$ The indicator is 1 when both words appear in that sentence and 0 otherwise. Each matrix row is the target word’s vector: its list of four context counts. $$ \cos(\mathbf a,\mathbf b)=\frac{\mathbf a\cdot\mathbf b}{\|\mathbf a\|\|\mathbf b\|} $$ The dot product multiplies corresponding coordinates and adds them. The length is the square root of the sum of squared coordinates. Cosine compares directions while allowing different vector lengths. ### Solution ### Step 1. Record the contribution of each sentence | Sentence | Targets present | Relevant context | Counts added | | --- | --- | --- | --- | | 1 | teacher, board | wrote | teacher–wrote +1; board–wrote +1 | | 2 | student, teacher | listened | student–listened +1; teacher–listened +1 | | 3 | board | notes | board–notes +1 | | 4 | student | question | student–question +1 | | 5 | teacher, book | none | No requested context count | ### Step 2. Construct every requested column | Target | wrote | listened | notes | question | | --- | --- | --- | --- | --- | | teacher | 1 | 1 | 0 | 0 | | student | 0 | 1 | 0 | 1 | | board | 1 | 0 | 1 | 0 | | book | 0 | 0 | 0 | 0 | ### Step 3. Calculate teacher–student $$ \mathbf t\cdot\mathbf s=(1)(0)+(1)(1)+(0)(0)+(0)(1)=1 $$ $$ \|\mathbf t\|=\sqrt{1^2+1^2+0^2+0^2}=\sqrt2 $$ $$ \|\mathbf s\|=\sqrt{0^2+1^2+0^2+1^2}=\sqrt2 $$ $$ \cos(\mathbf t,\mathbf s)=\frac1{\sqrt2\sqrt2}=\frac12=0.5 $$ ### Step 4. Calculate teacher–board $$ \mathbf t\cdot\mathbf b=(1)(1)+(1)(0)+(0)(1)+(0)(0)=1 $$ $$ \|\mathbf b\|=\sqrt{1^2+0^2+1^2+0^2}=\sqrt2 $$ $$ \cos(\mathbf t,\mathbf b)=\frac1{\sqrt2\sqrt2}=0.5 $$ ### Step 5. Select only the two requested plot coordinates | Word | x = wrote | y = notes | | --- | --- | --- | | teacher | 1 | 0 | | student | 0 | 0 | | board | 1 | 1 | | book | 0 | 0 | 001122teacherstudent, bookboardwrotenotes The plot uses only the requested two coordinates; cosine uses all context columns. ### Final answer Both requested cosine similarities are **0.5**. Student and book overlap at the origin of the 2-D projection, but their complete four-dimensional vectors differ. The book row is zero, so a cosine involving book would be undefined without an explicit convention. **Yaad rakho:** Har sentence mein target aur context saath? Ek count. Cosine ke liye poori row lo, plot ke liye sirf do columns. --- ## TEE · Q4A · Three preprocessing techniques Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/final-2025--final-2025-q4a/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/final-2025/final-2025-q4a Actual final examination · 2 December 2025 · 100 marks · 3 hours Question. Explain any three basic text preprocessing techniques used in Natural Language Processing (NLP). Discuss the purpose of each technique and illustrate them with suitable examples. [10 marks] ### Answer Text preprocessing transforms raw text into a representation suitable for a chosen NLP task. The best steps depend on the task; removing information indiscriminately can reduce performance. ### 1. Tokenization Tokenization divides text into units called tokens, such as words, subwords or sentences. These units are required for counting words and building many model inputs. “The cats are sleeping.” can become [The, cats, are, sleeping, .] under a punctuation-preserving word tokenizer. A different tokenizer may split contractions or punctuation differently, so the convention must be consistent during training and prediction. ### 2. Lowercasing Lowercasing maps uppercase letters to lowercase so that variants such as “Book”, “BOOK” and “book” can share one feature. This reduces vocabulary duplication in a case-insensitive search system. However, “US” and “us” are different, and capitals help NER recognise names. Therefore lowercasing should be chosen according to the application. ### 3. Lemmatization Lemmatization maps an inflected word to its dictionary base form, often using its POS and context. Examples: cats → cat; running/VBG → run; better/JJR → good. It allows a search for “run” to match relevant inflections. A stemmer instead uses simpler truncation rules and can produce a non-word stem; lemmatization is linguistically informed and can need more processing. ### Combined example | Stage | Representation | | --- | --- | | Raw | The cats are running. | | Tokenized | The | cats | are | running | . | | Lowercased | the | cats | are | running | . | | Lemmatized with correct POS | the | cat | be | run | . | These steps do not automatically remove stop words or punctuation. Those are separate choices. For sentiment analysis, preserve negation such as “not”; for syntax parsing, function words and word order are often essential. **Yaad rakho:** Preprocess task dekh ke karo: har word ko blindly delete mat karo. --- ## TEE · Q4B · HMM emission, transition and sequence Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/final-2025--final-2025-q4b/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/final-2025/final-2025-q4b Actual final examination · 2 December 2025 · 100 marks · 3 hours Question. Consider the following POS tagged corpus. Apply Hidden Markov Model (HMM) approach to: 1. Cat/Noun chase/Verb mice/Noun 2. The/Det dog/Noun runs/Verb 3. A/Det cat/Noun eats/Verb cheese/Noun i. Construct the emission probability matrix. ii. Construct the transmission probability matrix. iii. Calculate the probability of sequence of events: The/Det cat/Noun eats/Verb mice/Noun. Note: Include start and end characters for all the sentences. [10 marks] ### Answer ### Given | Sentence | Tagged sequence | | --- | --- | | 1 | cat/Noun chase/Verb mice/Noun | | 2 | the/Det dog/Noun runs/Verb | | 3 | a/Det cat/Noun eats/Verb cheese/Noun | Test words: the cat eats mice. Given tag sequence: Det → Noun → Verb → Noun. Include Start and End for all sentences. Lowercase the corpus so sentence-initial Cat and cat are one word. ### To find Emission probabilities, transition probabilities and the joint probability of the specified tag-and-word sequence. ### Formula: which one and why $$ P(w\mid t)=\frac{C(t,w)}{C(t)} $$ An emission probability asks: among tokens carrying this tag t, what fraction are the word w? $$ P(t_j\mid t_i)=\frac{C(t_i,t_j)}{\sum_k C(t_i,t_k)} $$ A transition probability asks: among transitions leaving tag tᵢ, what fraction go to tⱼ? Include End in the outgoing counts. The paper says “transmission”; the standard HMM term is transition. $$ P(\mathbf w,\mathbf t)=P(t_1\mid\mathrm{Start})\prod_{i=1}^{n}P(w_i\mid t_i)\prod_{i=2}^{n}P(t_i\mid t_{i-1})P(\mathrm{End}\mid t_n) $$ ### Solution ### Step 1. Count tokens for each tag Det has 2 tokens; Noun has 5 tokens; Verb has 3 tokens. The repeated noun cat occurs twice with Noun; every other displayed noun occurs once. The three verb types each occur once. ### Step 2. Build the emission matrix | Tag / word | a | cat | chase | cheese | dog | eats | mice | runs | the | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | Det | 1/2 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1/2 | | Noun | 0 | 2/5 | 0 | 1/5 | 1/5 | 0 | 1/5 | 0 | 0 | | Verb | 0 | 0 | 1/3 | 0 | 0 | 1/3 | 0 | 1/3 | 0 | Each row sums to one. A zero means no occurrence under that tag in this unsmoothed training corpus; no add-one smoothing was requested. ### Step 3. Write the three tag chains Start → Noun → Verb → Noun → End Start → Det → Noun → Verb → End Start → Det → Noun → Verb → Noun → End ### Step 4. Count outgoing transitions | From | To Det | To Noun | To Verb | To End | Outgoing total | | --- | --- | --- | --- | --- | --- | | Start | 2 | 1 | 0 | 0 | 3 | | Det | 0 | 2 | 0 | 0 | 2 | | Noun | 0 | 0 | 3 | 2 | 5 | | Verb | 0 | 2 | 0 | 1 | 3 | ### Step 5. Divide each row by its outgoing total | From / To | Det | Noun | Verb | End | | --- | --- | --- | --- | --- | | Start | 2/3 | 1/3 | 0 | 0 | | Det | 0 | 1 | 0 | 0 | | Noun | 0 | 0 | 3/5 | 2/5 | | Verb | 0 | 2/3 | 0 | 1/3 | End is terminal, so it has no estimated outgoing row. The four nonterminal rows each sum to one. ### Step 6. Choose the factors for the given sequence | Event in order | Probability | | --- | --- | | Start → Det | 2/3 | | Det emits the | 1/2 | | Det → Noun | 1 | | Noun emits cat | 2/5 | | Noun → Verb | 3/5 | | Verb emits eats | 1/3 | | Verb → Noun | 2/3 | | Noun emits mice | 1/5 | | Noun → End | 2/5 | ### Step 7. Multiply, including the end transition $$ P=\frac23\times\frac12\times1\times\frac25\times\frac35\times\frac13\times\frac23\times\frac15\times\frac25 $$ Start with subtotal 1. At each row, multiply the previous subtotal by exactly one next factor. Simplify the fraction after multiplication, but do not round it. | Factor being included | Previous subtotal × next factor = new subtotal | | --- | --- | | 1. Start → Det | $$ 1\times\frac{2}{3}=\frac{2}{3} $$ | | 2. Det emits the | $$ \frac{2}{3}\times\frac{1}{2}=\frac{1}{3} $$ | | 3. Det → Noun | $$ \frac{1}{3}\times1=\frac{1}{3} $$ | | 4. Noun emits cat | $$ \frac{1}{3}\times\frac{2}{5}=\frac{2}{15} $$ | | 5. Noun → Verb | $$ \frac{2}{15}\times\frac{3}{5}=\frac{2}{25} $$ | | 6. Verb emits eats | $$ \frac{2}{25}\times\frac{1}{3}=\frac{2}{75} $$ | | 7. Verb → Noun | $$ \frac{2}{75}\times\frac{2}{3}=\frac{4}{225} $$ | | 8. Noun emits mice | $$ \frac{4}{225}\times\frac{1}{5}=\frac{4}{1125} $$ | | 9. Noun → End | $$ \frac{4}{1125}\times\frac{2}{5}=\frac{8}{5625} $$ | The last subtotal is the complete product. Only now convert the fraction to a decimal. $$ P=\frac8{5625}\approx0.0014222222 $$ ### Final answer **Joint sequence probability = 8/5625 ≈ 0.0014222222.** This is the probability of the specified words and tags together. It is not the probability of the word sequence summed over every possible tag path. **Yaad rakho:** Tag se tag = transition; tag se word = emission. Dono alternate multiply, end ko mat bhoolo. **Boundary reminder:** The initial factor is P(Det | Start) = 2/3, based on three sentence starts, and the final factor is P(End | Noun) = 2/5. The overall fraction of Det tokens, 2/10, is not the start probability. Using it and omitting End, as in the handwritten key’s last line, computes a different quantity. --- ## TEE · Q5A · TF–IDF matrix Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/final-2025--final-2025-q5a/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/final-2025/final-2025-q5a Actual final examination · 2 December 2025 · 100 marks · 3 hours Question. For the following two sentences: Sentence A: “The student read a science book in the library” Sentence B: “The teacher read a history book in the classroom” 1. Construct the TF-IDF matrix for these sentences after removing the stop words {The, a, in}. 2. Identify and comment on the important terms in each sentence based on their TF-IDF scores. [10 marks] ### Answer ### Given Two documents, N = 2. Remove the listed stop words case-insensitively, so both “The” and “the” are removed. Use raw count TF and natural-log IDF, the convention used in the course key. ### To find The term-by-document TF–IDF matrix and the terms that distinguish each sentence. ### Formula: which one and why $$ \mathrm{TF}(t,d)=C(t,d),\qquad \mathrm{IDF}(t)=\ln\frac{N}{\mathrm{df}(t)} $$ $$ \mathrm{TFIDF}(t,d)=\mathrm{TF}(t,d)\,\mathrm{IDF}(t) $$ C(t,d) is the number of occurrences of term t in document d. df(t) is the number of documents containing t, not its total token count. N is the number of documents. IDF downweights terms shared by many documents. ### Solution ### Step 1. Clean and list the tokens A: student, read, science, book, library. B: teacher, read, history, book, classroom. Each retained token appears once in its sentence. ### Step 2. Fix one common vocabulary order student, read, science, book, library, teacher, history, classroom. Each row must refer to the same term in both documents. ### Step 3. Count document frequency and calculate IDF | Term type | df | IDF calculation | | --- | --- | --- | | read, book | 2 | ln(2/2) = ln(1) = 0 | | Every other term | 1 | ln(2/1) = ln(2) ≈ 0.693147 | A term absent from a document has TF = 0. A present term has TF = 1 here. ### Step 4. Multiply TF by IDF | Term | TF in A | TF in B | IDF | TF–IDF A | TF–IDF B | | --- | --- | --- | --- | --- | --- | | student | 1 | 0 | 0.693147 | 0.693147 | 0 | | read | 1 | 1 | 0 | 0 | 0 | | science | 1 | 0 | 0.693147 | 0.693147 | 0 | | book | 1 | 1 | 0 | 0 | 0 | | library | 1 | 0 | 0.693147 | 0.693147 | 0 | | teacher | 0 | 1 | 0.693147 | 0 | 0.693147 | | history | 0 | 1 | 0.693147 | 0 | 0.693147 | | classroom | 0 | 1 | 0.693147 | 0 | 0.693147 | ### Step 5. Interpret the weights A is distinguished by student, science and library. B is distinguished by teacher, history and classroom. “read” and “book” have zero IDF in this tiny unsmoothed two-document collection because they appear in every document; that does not mean those words are meaningless in English. ### Final answer All six document-specific terms receive **0.693147** in their own document and 0 in the other. Shared read and book receive 0 in both. **Yaad rakho:** TF: is document mein kitni baar? IDF: kitne documents mein common? Rare-across-documents term ko zyada weight. If normalized TF is explicitly requested, divide each present-token count by five; each nonzero weight becomes 0.693147/5 = 0.1386294. Do not silently switch conventions. --- ## TEE · Q5B · Naïve Bayes word-sense classification Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/final-2025--final-2025-q5b/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/final-2025/final-2025-q5b Actual final examination · 2 December 2025 · 100 marks · 3 hours Question. Consider the sentence: “The bat hit the ball hard on the cricket field.” Two possible senses of “bat”: Sense 1: Bat (Sports equipment – cricket/baseball); Sense 2: Bat (Nocturnal flying mammal). | Sentence ID | Context words | Sense | | --- | --- | --- | | 1 | ball, cricket, score, stadium | Sports | | 2 | night, cave, wings, fly | Animal | | 3 | player, run, match, bat | Sports | | 4 | nocturnal, vampire, insect | Animal | | 5 | team, field, bowl, bat | Sports | | 6 | dark, flying, cave, night | Animal | a) Using the Naïve Bayes algorithm, classify the sense of “bat” in the given sentence. Apply Laplace Smoothing wherever applicable. b) Discuss the significance of Laplace Smoothing in Naïve Bayes Classification. [10 marks] ### Answer ### Given There are three Sports examples and three Animal examples. Treat training context lists as bag-of-words token counts. Exclude the ambiguous target “bat” from the test context. Use the fixed training vocabulary; ignore out-of-vocabulary test words hit and hard. The usable test features are ball, cricket and field. ### To find The posterior-score comparison for Sports and Animal, and the role of smoothing. ### Formula: which one and why $$ \mathrm{score}(s)=P(s)\prod_{w\in\mathrm{context}}P(w\mid s) $$ $$ P(w\mid s)=\frac{C(w,s)+1}{N_s+V} $$ P(s) is the class prior, C(w,s) counts occurrences of context word w in class s, Nₛ is the total number of context tokens in that class and V is the shared vocabulary size. Conditional independence lets us multiply word probabilities. The common denominator P(context) is unnecessary when only comparing classes. ### Solution ### Step 1. Calculate the priors $$ P(S)=\frac36=0.5,\qquad P(A)=\frac36=0.5 $$ ### Step 2. Count class tokens and vocabulary Sports has 4 + 4 + 4 = 12 tokens. Animal has 4 + 3 + 4 = 11 tokens. The shared training vocabulary has 20 distinct words: ball, cricket, score, stadium, player, run, match, bat, team, field, bowl, night, cave, wings, fly, nocturnal, vampire, insect, dark, flying. “bat”, “night” and “cave” repeat but count only once each towards V. ### Step 3. Look up each usable context word | Feature | Sports count | Animal count | P(word | Sports) | P(word | Animal) | | --- | --- | --- | --- | --- | | ball | 1 | 0 | (1+1)/(12+20) = 2/32 | (0+1)/(11+20) = 1/31 | | cricket | 1 | 0 | 2/32 | 1/31 | | field | 1 | 0 | 2/32 | 1/31 | ### Step 4. Multiply the Sports score $$ \mathrm{score}(S)=0.5\times\frac2{32}\times\frac2{32}\times\frac2{32}=0.0001220703125 $$ ### Step 5. Multiply the Animal score $$ \mathrm{score}(A)=0.5\times\frac1{31}\times\frac1{31}\times\frac1{31}\approx0.00001678359236 $$ ### Step 6. Compare the scores 0.0001220703125 is greater than 0.00001678359236. Therefore select Sports. Optional normalization gives P(Sports | retained features) ≈ 0.87913; normalization is not needed for the decision. ### Final answer The intended sense is **bat as sports equipment**. Ball, cricket and field favour that sense. **Yaad rakho:** Pehle class prior, phir har context-word likelihood multiply; bigger score wali sense choose karo. ### Why Laplace smoothing matters Without smoothing, all three test features have zero count in Animal. One zero factor makes the entire class score zero, even if other evidence could support that class. Add-one smoothing assigns nonzero probability to every word in the defined vocabulary and redistributes probability mass. It does not mean every unseen word outside the vocabulary can be inserted without changing the vocabulary model. ### If following the supplied paper key exactly The key also multiplies probabilities for hit and hard using the same denominators 32 and 31. That convention produces Sports = 1.19209290 × 10⁻⁷ and Animal ≈ 1.74647163 × 10⁻⁸, again selecting Sports. Its treatment of out-of-vocabulary words is not a normalized fixed-vocabulary model. State your feature convention clearly; the complete calculation above uses a consistent fixed training vocabulary. --- ## TEE · Q6A · Coreference, pronouns and Word2Vec Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/final-2025--final-2025-q6a/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/final-2025/final-2025-q6a Actual final examination · 2 December 2025 · 100 marks · 3 hours Question. i. Differentiate between co-reference resolution and pronoun resolution with suitable examples. ii. Explain the working principle of the Continuous Bag of Words (CBOW) model and Skipgram model in Word2Vec. Illustrate how it predicts the target word from its surrounding context. [10 marks] ### Answer ### i. Coreference and pronoun resolution **Coreference resolution** groups expressions that refer to the same entity, including names, descriptions and pronouns. **Pronoun resolution** identifies the referent of a pronoun; it is narrower in scope. | Expression / task | Example | | --- | --- | | Coreference cluster | “Dr Rao arrived. The surgeon opened her bag.” → {Dr Rao, the surgeon, her} | | Pronoun resolution | For the same text, resolve “her” to Dr Rao. | | Difference | Resolving the description “the surgeon” is coreference work even when no pronoun is involved. | ### ii. Word2Vec Word2Vec learns dense word embeddings by predicting words from local context. A dense embedding is a short learned list of real numbers for a word; words used in similar contexts can obtain related vectors. CBOW and Skip-gram use opposite prediction directions. | Feature | CBOW | Skip-gram | | --- | --- | --- | | Input | Surrounding context words | One centre word | | Prediction target | The centre word | Surrounding context words | | Training example | [the, cat, on, the] → sat | sat → the; sat → cat; sat → on; sat → the | | Representation | Combine context embeddings, often by averaging | Use the centre-word embedding for each context prediction | Example sentence: “The cat sat on the mat.” With a radius-two window around sat, the context is [The, cat, on, the]. In CBOW, obtain the four context embeddings, average or sum them, calculate scores for vocabulary words, and train the model to assign high probability to sat. In Skip-gram, use sat as input and create four centre–context training pairs. CBOW: nearby words → combine embeddings → predict centre word Skip-gram: centre word → its embedding → predict nearby words Training adjusts embedding parameters to improve prediction; the final word vectors are the learned representation. Thus the question’s phrase “predicts the target word from surrounding context” describes CBOW specifically; Skip-gram reverses that direction. **Yaad rakho:** CBOW: neighbours se centre. Skip-gram: centre se neighbours. --- ## TEE · Q6B · Hobbs: Ravi and Suresh Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/final-2025--final-2025-q6b/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/final-2025/final-2025-q6b Actual final examination · 2 December 2025 · 100 marks · 3 hours Question. i. Explain Hobb’s Algorithm. Discuss the steps with help of example and state its limitation. ii. Sentence: Ravi met Suresh at the station after he arrived from Delhi. Using Hobbs’ algorithm, determine the most likely antecedent of the pronoun “he”. Show the steps of traversal according to the algorithm. [10 marks] ### Answer ### i. Method **Hobbs’ algorithm** is a syntax-based pronoun-resolution procedure that searches constituency parse trees in a specified order. It proposes noun phrases as antecedents and filters incompatible candidates, for example by number and gender agreement. **Read the tree labels first:** NP means noun phrase; S means sentence/clause; VP means verb phrase. X is the current NP or S search node, and p is the upward path used to reach it. Breadth-first means shallower nodes before deeper nodes; left-to-right fixes the order at the same depth. An N-bar (N′) node is the noun-centred part of an NP in a parse using that label. ### Complete ordered procedure 1. **Locate the pronoun.** Start at the NP immediately dominating the pronoun: the smallest NP containing it. 2. **Make the first climb.** Move upward to the first NP or S above the starting NP. Call it X and record the path p. 3. **Search initially to the left.** Traverse branches below X to the left of p, breadth-first and left-to-right. Propose an NP only if another NP or S lies between that candidate and X. Check agreement and other applicable constraints for each proposed candidate; accept the first eligible compatible one. 4. **Check the highest sentence.** If X is the highest S and no candidate has been accepted, search previous sentence trees, most recent first. In each, traverse breadth-first and left-to-right and propose NPs. Accept the first eligible compatible candidate; if the available context contains none, report no resolved antecedent. 5. **Otherwise climb again.** Move from X to the next NP or S above it. Make that node the new X and record the new upward path p. 6. **Test a higher NP.** If the new X is an NP, propose X only when p did not pass through its nominal N-bar child. Apply the same compatibility checks. Use the labels in the supplied parse; do not invent a missing N-bar branch. 7. **Search left again.** Traverse the new X’s branches to the left of p, breadth-first and left-to-right, proposing NPs. The extra intervening-NP/S requirement in Step 3 belongs to the initial search. 8. **Search permitted right branches.** If X is an S, traverse branches to the right of p, breadth-first and left-to-right, but do not descend below any NP or S encountered. Propose the encountered eligible NPs and test their compatibility. 9. **Repeat or finish.** Return to Step 4 until a compatible candidate is accepted or the available sentence trees are exhausted. Record the candidate order and why each candidate was accepted or rejected. **Why the initial restriction matters:** in “John saw him”, the subject NP John is directly under the clause S, with no additional NP or S between them. Step 3 therefore does not propose John as the ordinary local antecedent of him. A reflexive such as himself is a different grammatical case. The worked question below shows the specific parse and traversal; selecting an antecedent is not simply choosing the nearest name. ### ii. Apply the method **Given:** the pronoun he occurs as the subject of the subordinate clause “after he arrived from Delhi”. Assume that this clause is attached to the main verb phrase. The relevant main-clause NPs are Ravi, Suresh and the station. **To find:** the first eligible singular-male antecedent in Hobbs’ syntactic search order. ``` S (main) ├── NP: Ravi ← first higher-left candidate └── VP ├── VP: met [NP Suresh] at [NP the station] └── SBAR: after └── S (embedded) ← first X ├── NP: he ← start └── VP: arrived from Delhi ``` ### Step 1. Start at the pronoun NP Locate NP “he” inside the after-clause. Move to that clause’s S node. No earlier noun phrase in this small clause provides a candidate. ### Step 2. Move to the higher sentence The embedded S is not the highest S, so Step 5 climbs through SBAR and VP to the main S. Step 7 now searches to the left of that new path. This is the later left search, so the extra initial-search restriction from Step 3 is not imposed again. ### Step 3. Examine candidates in structural order The main subject NP Ravi is encountered before the deeper object NP Suresh in the relevant breadth-first search. Ravi is singular and compatible with “he”. “the station” would fail the ordinary person constraint. ### Step 4. Choose the first compatible candidate Select Ravi under the assumed parse and agreement settings. Suresh remains a possible interpretation in ordinary language, but the syntactic preference chooses Ravi first. ### Limitations - The result depends on the accuracy and attachment choices of the parse tree. - Syntactic ordering alone does not model all semantic plausibility, discourse focus or world knowledge. - Agreement can eliminate candidates but cannot distinguish two compatible male singular people. - Long-distance references and sentences with multiple possible parses can remain ambiguous. **Final answer:** Hobbs’ syntactic preference is **Ravi** for this analysis. This is an algorithmic selection, not proof of the speaker’s intended referent. **Yaad rakho:** Nearest naam blindly nahi: tree ki search order aur agreement follow karo. --- ## TEE · Q7A · Text classification pipeline Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/final-2025--final-2025-q7a/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/final-2025/final-2025-q7a Actual final examination · 2 December 2025 · 100 marks · 3 hours Question. Explain the working of text classification in Natural Language Processing (NLP) and discuss the key steps involved in building a text classification system in context of real-world applications such as spam detection or news categorization. [10 marks] ### Answer **Text classification** assigns one or more predefined labels to a document or message based on its content. In spam detection the labels may be spam and not-spam; in news categorization they may be sports, business and politics. Labelled text → Prepare and split data → Represent text → Train classifier → Evaluate → Classify new text 1. **Define the labels and task.** Decide whether each text gets one label or multiple labels. Establish what counts as spam so annotators use a consistent rule. 2. **Collect and label representative examples.** Gather messages from the intended application and assign reliable labels. Include enough examples of each class and protect private information. 3. **Split training, validation and test data.** Use training data to learn, validation data to choose settings and an untouched test set for final evaluation. Keep duplicate or closely related texts from leaking between splits. 4. **Preprocess consistently.** Tokenize and normalize only as appropriate. In spam detection, URLs, currency symbols and capitalisation may contain useful information. 5. **Create features.** A classical baseline uses BoW or TF–IDF. A neural system may use embeddings or a pretrained contextual model. Fit the vocabulary and IDF on training data only. 6. **Train a classifier.** For example, multinomial Naïve Bayes estimates class priors and word likelihoods; logistic regression learns a decision boundary. Training uses the labelled examples. 7. **Evaluate.** Inspect a confusion matrix, precision, recall and F1 as well as accuracy. False spam flags may hide legitimate mail, so the costs of errors matter. 8. **Deploy and monitor.** Apply the same preprocessing and feature mapping to new texts. Monitor changing vocabulary and error patterns, and retrain when suitable new labelled data becomes available. ### Example prediction A message such as “Claim your free prize now” is transformed into its feature representation. The classifier compares the evidence for the two labels and outputs spam if its decision score crosses the chosen threshold. The label comes from learned patterns; the mere presence of one word should not be treated as a universally correct rule. **Yaad rakho:** Labelled examples → features → model → evaluation → new text ka label. --- ## TEE · Q7B · Corpora and Jaccard similarity Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/final-2025--final-2025-q7b/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/final-2025/final-2025-q7b Actual final examination · 2 December 2025 · 100 marks · 3 hours Question. i. Differentiate between a flat text corpus and an annotated corpus, and discuss how each can be applied in various NLP tasks with suitable examples. ii. Explain a real-world scenario where the Jaccard Similarity measure is more appropriate than Euclidean or Manhattan distance, and justify why Jaccard is the best choice in that case. [10 marks] ### Answer ### i. Flat and annotated corpora A **corpus** is a collection of language data. A flat text corpus stores text without the linguistic labels needed by a particular task. An annotated corpus adds labels or structure such as POS tags, entity spans, parse trees or sentence sentiment. | Basis | Flat text corpus | Annotated corpus | | --- | --- | --- | | Example | A collection of plain news articles | The same articles with PERSON/ORG spans marked | | Information | The original text and possibly simple metadata | Text plus explicit task-related linguistic labels | | Applications | Word frequency, collocations and unsupervised language-model training | Supervised POS tagging, NER or parser training and evaluation | | Preparation | Relatively inexpensive to gather, but still needs cleaning | Requires annotation guidelines, labour and quality checks | “Flat” describes the lack of the annotations being discussed, not an absence of all file organization. Both corpus types may still have document boundaries and metadata. ### ii. Jaccard in a set-based application Consider recommending articles using their topic-tag sets. Article A has {AI, NLP, ethics}; article B has {AI, NLP, healthcare}. We care about overlap among present tags, not how many irrelevant tags are absent from both. ### Given A = {AI, NLP, ethics}; B = {AI, NLP, healthcare}. ### To find The fraction of distinct present tags shared by the articles. ### Formula and why $$ J(A,B)=\frac{|A\cap B|}{|A\cup B|} $$ The intersection counts common items; the union counts all distinct items present in either set. ### Step 1. Find common tags A ∩ B = {AI, NLP}, so the intersection size is 2. ### Step 2. Find all distinct present tags A ∪ B = {AI, NLP, ethics, healthcare}, so the union size is 4. ### Step 3. Divide $$ J(A,B)=\frac24=0.5 $$ Jaccard directly measures proportional overlap and ignores joint absences. Euclidean or Manhattan distance on binary tag vectors counts mismatches but does not by itself normalize by the union size. If frequency or magnitude were meaningful, those distance measures could be more appropriate. Here, membership-only tags and sparse sets make Jaccard a suitable choice. **Yaad rakho:** Jaccard: common present items / total distinct present items. --- ## TEE · Q1A · Generic NLP pipeline Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/reexam-2026--reexam-2026-q1a/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/reexam-2026/reexam-2026-q1a Actual re-examination · 9 February 2026 · 100 marks · 3 hours Question. Draw block diagram of generic NLP pipeline and explain the task associated with each block. [5 marks] ### Answer An NLP pipeline is an ordered workflow that transforms language data into a task-specific result. The exact components depend on the application; a generic supervised pipeline is shown below. Collect text → Clean and preprocess → Represent text → Train / apply model → Evaluate and deliver output | Block | Task | Example for sentiment analysis | | --- | --- | --- | | Data acquisition | Collect representative text and any required labels | Reviews labelled positive or negative | | Cleaning and preprocessing | Handle noise, tokenize and normalize appropriately | Remove HTML while preserving “not” | | Text representation | Convert tokens into usable model features | TF–IDF values or contextual embeddings | | Modeling | Learn patterns from training data and predict new labels | Train a classifier on labelled review features | | Evaluation | Measure performance on unseen held-out data and inspect errors | Precision, recall, F1 and confusion matrix | | Deployment and feedback | Deliver predictions and monitor changing data | Show review sentiment and monitor new product vocabulary | Training/validation/test separation must precede fitting the learned preprocessing and feature statistics. New inputs must use the same feature mapping as training. Not every task uses every linguistic level explicitly: a modern end-to-end model may combine several operations internally. **Yaad rakho:** Text lo → prepare karo → numbers banao → model lagao → result check aur use karo. --- ## TEE · Q1B · Content and function words Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/reexam-2026--reexam-2026-q1b/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/reexam-2026/reexam-2026-q1b Actual re-examination · 9 February 2026 · 100 marks · 3 hours Question. i. Define function words and content words. ii. Which category (function words and content words) belong to open class and closed class? Justify your answer. iii. For the sentence “The small cat is sleeping under the tree”, identify the function words and content words. [5 marks] ### Answer **Content words** carry much of a sentence’s lexical meaning, usually including nouns, main verbs, adjectives and many adverbs. **Function words** primarily express grammatical relationships, including determiners, prepositions, conjunctions, pronouns and auxiliary verbs. | Category | Typical class | Reason | | --- | --- | --- | | Content words | Open class | New nouns, verbs and adjectives enter the language readily, such as new technology names | | Function words | Closed class | The grammatical inventory is relatively stable; new determiners or pronouns are much less frequently added | | Word | Role in this sentence | Category | | --- | --- | --- | | The | Determiner | Function | | small | Adjective | Content | | cat | Noun | Content | | is | Auxiliary supporting progressive tense | Function | | sleeping | Main verb | Content | | under | Preposition | Function | | the | Determiner | Function | | tree | Noun | Content | **Final answer:** content words = small, cat, sleeping, tree; function words = The, is, under, the. “is” is auxiliary here; a word’s role must be judged in context. “Closed” means relatively resistant to new members, not that change is logically impossible. **Yaad rakho:** Content words scene batate hain; function words grammar ka connection banate hain. --- ## TEE · Q1C · Word2Vec and GloVe Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/reexam-2026--reexam-2026-q1c/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/reexam-2026/reexam-2026-q1c Actual re-examination · 9 February 2026 · 100 marks · 3 hours Question. Differentiate between word2vec and GloVe in terms of their learning strategies. Which type of corpus would be more suitable for each method, and why? [5 marks] ### Answer Both Word2Vec and GloVe learn dense word vectors from distributional information: words occurring in related contexts tend to receive related representations. They differ mainly in the training objective and how corpus co-occurrence evidence is used. | Basis | Word2Vec | GloVe | | --- | --- | --- | | Learning strategy | Predictive: CBOW predicts a centre word; Skip-gram predicts nearby words | Count-based objective fitted to global word–context co-occurrence statistics | | Training evidence | Local context windows converted into prediction examples | A corpus-wide co-occurrence matrix and relationships among counts | | Parameters learned | Word/context embeddings that make context predictions accurate | Word/context embeddings and biases that fit weighted log co-occurrence counts | | Corpus considerations | Large token sequences with useful local context, including domain-specific text | A sufficiently large corpus from which reliable global co-occurrence statistics can be collected | For a domain such as medicine, either method can benefit from a representative medical corpus. Word2Vec is convenient when generating local training windows directly from a stream of tokens. GloVe is natural when a global sparse co-occurrence matrix can be constructed and stored. Corpus size, domain match, preprocessing and available compute matter; there is no rule that Word2Vec requires informal text while GloVe requires formal text. Both standard forms produce one static vector per vocabulary word, so multiple senses are blended. They do not automatically provide a different vector for the same word in each sentence. **Yaad rakho:** Word2Vec predicts local neighbours; GloVe fits global co-occurrence counts. --- ## TEE · Q1D · Anaphora, cataphora and coreference Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/reexam-2026--reexam-2026-q1d/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/reexam-2026/reexam-2026-q1d Actual re-examination · 9 February 2026 · 100 marks · 3 hours Question. Differentiate between cataphora and anaphora in terms of direction of reference. How coreference resolution is different from anaphora reference? [5 marks] ### Answer **Anaphora** points backward to an expression or information introduced earlier. **Cataphora** points forward to an expression introduced later. | Relation | Example | Direction | | --- | --- | --- | | Anaphora | “Riya arrived. She smiled.” She refers to the earlier Riya. | Backward | | Cataphora | “Before she spoke, Riya checked the microphone.” She refers to the later Riya. | Forward | **Coreference** means two or more expressions refer to the same entity. Coreference resolution groups such expressions, regardless of whether they are names, descriptions or pronouns and regardless of which is mentioned first. For “Dr Sen arrived. The surgeon smiled. She waved,” {Dr Sen, the surgeon, She} is a coreference cluster. The latter two mentions are also backward-looking references. Coreference is an identity relation; anaphora describes a dependency on prior discourse and is not always identity. For example, “I bought a house. The roof leaks” contains a bridging reference: the roof depends on the house for interpretation, but roof and house are not the same entity. **Yaad rakho:** Ana = pehle ki taraf; cata = aage ki taraf; coreference = same entity. --- ## TEE · Q2A · NLP tasks and syntax versus semantics Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/reexam-2026--reexam-2026-q2a/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/reexam-2026/reexam-2026-q2a Actual re-examination · 9 February 2026 · 100 marks · 3 hours Question. i. Discuss five different tasks associated with Natural Language Processing. ii. Compare and contrast syntactic analysis and semantic analysis in NLP. Support your explanation with examples. [10 marks] ### Answer ### i. Five NLP tasks | Task | What it does | Example | | --- | --- | --- | | Machine translation | Expresses source-language content in a target language | English “Good morning” → Hindi greeting | | Sentiment analysis | Identifies the opinion or attitude expressed | “The service was excellent” → positive sentiment | | Named Entity Recognition | Finds and types entity spans | “Asha joined Google” → PERSON and ORGANIZATION | | Question answering | Returns an answer supported by a question’s relevant information | From a passage, identify who discovered a described result | | Text classification | Assigns predefined category labels | Classify an email as spam or legitimate | ### ii. Syntactic and semantic analysis | Basis | Syntactic analysis | Semantic analysis | | --- | --- | --- | | Main question | How are words grammatically organised? | What do the words and sentence mean? | | Typical output | POS tags, constituency or dependency structure | Word senses, semantic relations or a meaning representation | | Knowledge used | Grammar rules and structural patterns | Lexical meaning, context and semantic constraints | | Example | Identify “The boy” as subject NP and “ate an apple” as VP | Interpret the boy as eater and apple as the thing eaten | “Colourless green ideas sleep furiously” can have a plausible English grammatical structure while remaining semantically unusual. Syntax alone therefore does not guarantee ordinary meaning. Conversely, “boy eat apple yesterday” conveys a possible event despite grammatical errors. The analyses support each other. In “I saw the man with a telescope,” syntactic PP attachment determines which meaning is available, and semantic knowledge can help choose a plausible attachment. A full NLP system often needs both. **Yaad rakho:** Syntax = sentence ka structure; semantics = us structure ka meaning. --- ## TEE · Q2B · Laplace smoothing on a second corpus Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/reexam-2026--reexam-2026-q2b/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/reexam-2026/reexam-2026-q2b Actual re-examination · 9 February 2026 · 100 marks · 3 hours Question. Consider the following training corpus: 1. cats chase mice in the garden 2. dogs bark loudly at night 3. birds sing in the morning 4. children play games in the park Using a bigram language model with Laplace smoothing, evaluate the probability of the test sentence: “cats bark in the park.” Assume and as start and end of the sentence. [10 marks] ### Answer ### Given - cats chase mice in the garden - dogs bark loudly at night - birds sing in the morning - children play games in the park Test: cats bark in the park. Add one start token and one end token to every training and test sentence. ### To find The Laplace-smoothed bigram probability of the complete test sentence, including its end token. ### Formula: which one and why $$ P_{+1}(w_i\mid w_{i-1})=\frac{C(w_{i-1},w_i)+1}{C(w_{i-1})+V} $$ C(previous,current) counts that adjacent pair in the training corpus. C(previous) counts its occurrences as a history. V is the vocabulary size. Add-one smoothing is necessary because an unseen pair must not force the whole sentence probability to zero. $$ P(w_1,\ldots,w_n,\langle/s\rangle\mid\langle s\rangle)=\prod_{i=1}^{n+1}P(w_i\mid w_{i-1}) $$ ### Solution ### Step 1. Fix the token and vocabulary convention Lowercase words and remove the final punctuation. The supplied course key counts both boundary tokens in V. Following that convention: at, bark, birds, cats, chase, children, dogs, games, garden, in, loudly, mice, morning, night, park, play, sing, the, , gives **V = 20**. The start marker supplies history; it is not an extra probability factor before the first word. ### Step 2. List every required transition The complete test chain is → cats → bark → in → the → park → . ### Step 3. Count first, then add one | Required bigram | History count | Pair count | Smoothed probability | | --- | --- | --- | --- | | → cats | 4 | 1 | (1+1)/(4+20) = 1/12 | | cats → bark | 1 | 0 | (0+1)/(1+20) = 1/21 | | bark → in | 1 | 0 | (0+1)/(1+20) = 1/21 | | in → the | 3 | 3 | (3+1)/(3+20) = 4/23 | | the → park | 3 | 1 | (1+1)/(3+20) = 2/23 | | park → | 1 | 1 | (1+1)/(1+20) = 2/21 | Count only adjacent words, never words that merely appear in the same sentence. The unseen pairs have count zero, so their numerator becomes one. ### Step 4. Multiply all six probabilities $$ P=\frac{2}{24}\times\frac{1}{21}\times\frac{1}{21}\times\frac{4}{23}\times\frac{2}{23}\times\frac{2}{21} $$ Start with subtotal 1. At each row, multiply the previous subtotal by exactly one next factor. Simplify the fraction after multiplication, but do not round it. | Factor being included | Previous subtotal × next factor = new subtotal | | --- | --- | | 1. → cats | $$ 1\times\frac{1}{12}=\frac{1}{12} $$ | | 2. cats → bark | $$ \frac{1}{12}\times\frac{1}{21}=\frac{1}{252} $$ | | 3. bark → in | $$ \frac{1}{252}\times\frac{1}{21}=\frac{1}{5292} $$ | | 4. in → the | $$ \frac{1}{5292}\times\frac{4}{23}=\frac{1}{30429} $$ | | 5. the → park | $$ \frac{1}{30429}\times\frac{2}{23}=\frac{2}{699867} $$ | | 6. park → | $$ \frac{2}{699867}\times\frac{2}{21}=\frac{4}{14697207} $$ | The last subtotal is the complete product. Only now convert the fraction to a decimal. $$ P=\frac{4}{14697207}\approx 2.72160554\times10^{-7} $$ ### Step 5. Interpret the result This is a joint sentence probability, so a small value is normal. Do not add transition probabilities; each word must occur in sequence, so the chain multiplies them. ### Final answer **P ≈ 2.72160554 × 10-7** with the stated course vocabulary convention. **Yaad rakho:** Pair count mein 1 jodo, previous-word count mein V; phir saare fractions multiply karo. **Convention note:** Some implementations exclude from the predicted vocabulary. Their V is one smaller and their numerical probability differs. In an exam, state V explicitly and follow the convention specified in the question or class solution. --- ## TEE · Q3A · Top-down parsing Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/reexam-2026--reexam-2026-q3a/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/reexam-2026/reexam-2026-q3a Actual re-examination · 9 February 2026 · 100 marks · 3 hours Question. In the context of natural language processing, explain the concept of top-down and bottom-up parsing approaches. Apply top-down syntax parsing approach to the sentence “The tall boy saw a girl in the park.” and draw a parse tree using the CFG grammar rules: S → NP VP; NP → Det Adj N | Det N; VP → V NP | VP PP; PP → P NP. Lexicon: Det → {the | a}; Adj → {tall}; N → {boy | girl | park}; V → {saw}; P → {in}. [10 marks] ### Answer | Aspect | Top-down | Bottom-up | | --- | --- | --- | | Start | The start symbol S | The input words | | Operation | Expand a nonterminal using grammar productions | Combine words and constituents using productions in reverse | | Success | Leaves match the complete input | All input is reduced to S | | Possible difficulty | Naive recursive descent can loop on left recursion or need backtracking | A naive shift-reduce parser may face shift/reduce conflicts | ### Given The sentence contains the tokens the / tall / boy / saw / a / girl / in / the / park. Lowercase the initial “The” to match the lexicon. The full stop is punctuation outside the listed grammar. ### To find A derivation beginning at S and a parse tree matching the whole sentence. ### Rule: which one and why A context-free production replaces one nonterminal with its right-hand sequence. A top-down derivation begins at S and expands the desired structure. The supplied grammar attaches the PP through VP → VP PP; it does not provide NP → NP PP. ### Step 1. Expand the sentence S ⇒ NP VP. Choose NP → Det Adj N for “the tall boy”. ### Step 2. Match the subject words Det ⇒ the, Adj ⇒ tall, N ⇒ boy. The remaining material must be generated by VP. ### Step 3. Allow a verb phrase followed by a PP Expand the outer VP using VP → VP PP. Then expand the inner VP using VP → V NP. This finite derivation selects the recursive rule once. ### Step 4. Match the verb and object V ⇒ saw. Choose NP → Det N, then Det ⇒ a and N ⇒ girl. ### Step 5. Expand the prepositional phrase PP ⇒ P NP. P ⇒ in. NP ⇒ Det N, with Det ⇒ the and N ⇒ park. ### Step 6. Check the complete leaf order The leaves read “the tall boy saw a girl in the park”, exactly the normalized input. The PP “in the park” attaches to the VP under this supplied grammar. SNPVPDetAdjNVPPPVNPPNPDetNDetNThetallboysawagirlinthepark Top-down parse tree showing the PP attached to VP. ### Complete derivation S ⇒ NP VP ⇒ Det Adj N VP ⇒ the tall boy VP ⇒ the tall boy VP PP ⇒ the tall boy V NP PP ⇒ the tall boy saw Det N PP ⇒ the tall boy saw a girl P NP ⇒ the tall boy saw a girl in Det N ⇒ the tall boy saw a girl in the park. **Implementation caution:** VP → VP PP is left-recursive. A naive recursive-descent implementation may loop; it must transform or otherwise handle the grammar. A valid finite top-down derivation, as shown above, still exists. **Yaad rakho:** Rule jo diya hai wahi use karo: yahan “in the park” VP ka part hai. --- ## TEE · Q3B · Lesk word-sense disambiguation Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/reexam-2026--reexam-2026-q3b/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/reexam-2026/reexam-2026-q3b Actual re-examination · 9 February 2026 · 100 marks · 3 hours Question. Consider the sentence: “The bank can ensure your deposits are safe.” The word “bank” has the following two possible senses (WordNet-style glosses): Bank₁: Sloping land especially beside a body of water. Bank₂: A financial institution that accepts deposits and channels the money into lending activities. i. Apply the LESK algorithm to determine the correct sense of the word “bank” in the given sentence. Show the steps clearly. ii. Briefly discuss limitations of the LESK algorithm in handling ambiguous words in real-world contexts. [10 marks] ### Answer ### Given Context sentence and the two supplied dictionary glosses. Use a simplified Lesk method: lowercase, remove function words, exclude the target bank and compare distinct content-word overlap. Do not add synonyms that the question does not supply. ### To find Which candidate gloss has the greatest overlap with the local context. ### Formula: which one and why $$ \mathrm{score}(s)=|C\cap G_s|,\qquad \hat s=\arg\max_s\mathrm{score}(s) $$ C is the set of context content words. Gₛ is the content-word set of the gloss for sense s. The intersection contains words appearing in both; its size is the score. This is an overlap count, not a probability. ### Solution ### Step 1. Prepare the context Remove The, can, your, are and the target bank. A reasonable content set is C = {ensure, deposits, safe}. If “can” is retained under another fixed stop-list it matches neither gloss and does not change the result. ### Step 2. Prepare Bank₁’s gloss G₁ = {sloping, land, especially, beside, body, water}. There is no shared word with {ensure, deposits, safe}. ### Step 3. Calculate Bank₁’s score $$ C\cap G_1=\varnothing,\qquad \mathrm{score}(1)=0 $$ ### Step 4. Prepare Bank₂’s gloss and count overlap G₂ = {financial, institution, accepts, deposits, channels, money, lending, activities}. The shared word is deposits. ### Step 5. Calculate Bank₂’s score and select $$ C\cap G_2=\{\mathrm{deposits}\},\qquad \mathrm{score}(2)=1 $$ 1 > 0, so choose the financial-institution sense. ### Final answer **Bank₂: financial institution.** The explicit overlap “deposits” supports this choice. **Yaad rakho:** Lesk: context aur har meaning ki definition mein common words count karo; highest overlap choose karo. ### ii. Limitations - Short contexts or short glosses may produce no overlaps or ties. - Exact word matching misses related words and synonyms, such as deposit versus savings, unless an extension is defined. - Irrelevant frequent words can create misleading overlaps if preprocessing is poor. - Dictionary glosses may omit a domain-specific sense or fail to express the context’s wording. - Simplified Lesk uses limited local information and may miss discourse or real-world knowledge. Lemmatization, richer glosses and semantic similarity can improve variants, but the exact variant and tie-breaking rule should be stated. --- ## TEE · Q4A · Bag of Words and three cosine similarities Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/reexam-2026--reexam-2026-q4a/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/reexam-2026/reexam-2026-q4a Actual re-examination · 9 February 2026 · 100 marks · 3 hours Question. Consider the following three documents: Document A: “The dog chased the cat.” Document B: “The cat chased the mouse.” Document C: “The dog ran after the cat.” Using the Bag of Words (BoW) model, represent each document as a vector after removing the stop words {The, by, was}. i. Construct the term-document matrix. ii. Compute the cosine similarity between each pair of documents (A–B, A–C, and B–C). iii. Interpret and comment on the results obtained in terms of document similarity. [10 marks] ### Answer ### Given Three documents. Lowercase and remove punctuation. Remove the specified stop words case-insensitively; “the” is removed everywhere. Keep “after” because it is not in the supplied stop list. Use raw word counts. ### To find A common-vocabulary count matrix, three cosine values and their interpretation. ### Formula: which one and why $$ \mathrm{BoW}_j(d)=C(v_j,d) $$ vⱼ is vocabulary term j. The coordinate stores its count in document d; order in the sentence is not retained. $$ \cos(\mathbf a,\mathbf b)=\frac{\sum_j a_jb_j}{\sqrt{\sum_j a_j^2}\sqrt{\sum_j b_j^2}} $$ ### Solution ### Step 1. Clean each document A: dog chased cat. B: cat chased mouse. C: dog ran after cat. ### Step 2. Fix vocabulary and matrix Vocabulary: dog, chased, cat, mouse, ran, after. | Term | A | B | C | | --- | --- | --- | --- | | dog | 1 | 0 | 1 | | chased | 1 | 1 | 0 | | cat | 1 | 1 | 1 | | mouse | 0 | 1 | 0 | | ran | 0 | 0 | 1 | | after | 0 | 0 | 1 | A = [1,1,1,0,0,0]; B = [0,1,1,1,0,0]; C = [1,0,1,0,1,1]. ### Step 3. Calculate vector lengths $$ \|A\|=\sqrt{1^2+1^2+1^2}=\sqrt3 $$ $$ \|B\|=\sqrt{1^2+1^2+1^2}=\sqrt3 $$ $$ \|C\|=\sqrt{1^2+1^2+1^2+1^2}=2 $$ ### Step 4. Calculate A–B $$ A\cdot B=(1)(0)+(1)(1)+(1)(1)+(0)(1)+(0)(0)+(0)(0)=2 $$ $$ \cos(A,B)=\frac2{\sqrt3\sqrt3}=\frac23\approx0.666667 $$ ### Step 5. Calculate A–C $$ A\cdot C=(1)(1)+(1)(0)+(1)(1)+(0)(0)+(0)(1)+(0)(1)=2 $$ $$ \cos(A,C)=\frac2{\sqrt3\cdot2}=\frac1{\sqrt3}\approx0.577350 $$ ### Step 6. Calculate B–C $$ B\cdot C=(0)(1)+(1)(0)+(1)(1)+(1)(0)+(0)(1)+(0)(1)=1 $$ $$ \cos(B,C)=\frac1{\sqrt3\cdot2}\approx0.288675 $$ ### Final answer **A–B = 0.666667; A–C = 0.577350; B–C = 0.288675.** A and B have the highest lexical overlap under this representation. B and C share only cat. BoW similarity reflects token overlap, not full semantic equivalence or word order. **Yaad rakho:** Same vocabulary order rakho. Corresponding counts multiply-add; phir dono lengths se divide. --- ## TEE · Q4B · A second HMM sequence Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/reexam-2026--reexam-2026-q4b/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/reexam-2026/reexam-2026-q4b Actual re-examination · 9 February 2026 · 100 marks · 3 hours Question. Consider the following POS tagged corpus. Apply Hidden Markov Model (HMM) approach to: 1. Dog/Noun eats/Verb bread/Noun 2. The/Det cat/Noun sleeps/Verb 3. A/Det dog/Noun eats/Verb fish/Noun i. Construct the emission probability matrix. ii. Construct the transmission probability matrix. iii. Calculate the probability of the given sequence of events: The/Det dog/Noun eats/Verb bread/Noun. Note: Include start and end characters for all the sentences. [10 marks] ### Answer The repeated verb eats has a different emission count from the final paper; recalculate it rather than copying the previous numerical result. ### Given | Sentence | Tagged sequence | | --- | --- | | 1 | dog/Noun eats/Verb bread/Noun | | 2 | the/Det cat/Noun sleeps/Verb | | 3 | a/Det dog/Noun eats/Verb fish/Noun | Lowercase the corpus. Test sequence: the/Det dog/Noun eats/Verb bread/Noun. Add Start and End markers. ### To find Emission and transition matrices, then the probability of the supplied tagged sequence. ### Formula: which one and why $$ P(w\mid t)=\frac{C(t,w)}{C(t)} $$ $$ P(t_j\mid t_i)=\frac{C(t_i,t_j)}{\sum_kC(t_i,t_k)} $$ Emission divides the word’s tag-specific count by the total tokens with that tag. Transition divides the tag-pair count by all transitions leaving the previous tag, including End. $$ P(\mathbf w,\mathbf t)=P(t_1\mid\mathrm{Start})\prod_iP(w_i\mid t_i)\prod_{i=2}^{n}P(t_i\mid t_{i-1})P(\mathrm{End}\mid t_n) $$ ### Solution ### Step 1. Count tag totals and word frequencies Det: the=1, a=1, total2. Noun: dog=2, bread=1, cat=1, fish=1, total5. Verb: eats=2, sleeps=1, total3. ### Step 2. Construct emission matrix | Tag / word | a | bread | cat | dog | eats | fish | sleeps | the | | --- | --- | --- | --- | --- | --- | --- | --- | --- | | Det | 1/2 | 0 | 0 | 0 | 0 | 0 | 0 | 1/2 | | Noun | 0 | 1/5 | 1/5 | 2/5 | 0 | 1/5 | 0 | 0 | | Verb | 0 | 0 | 0 | 0 | 2/3 | 0 | 1/3 | 0 | ### Step 3. Count transitions from the three chains Start → Noun → Verb → Noun → End Start → Det → Noun → Verb → End Start → Det → Noun → Verb → Noun → End | From | Det count | Noun count | Verb count | End count | Total | | --- | --- | --- | --- | --- | --- | | Start | 2 | 1 | 0 | 0 | 3 | | Det | 0 | 2 | 0 | 0 | 2 | | Noun | 0 | 0 | 3 | 2 | 5 | | Verb | 0 | 2 | 0 | 1 | 3 | ### Step 4. Divide outgoing counts | From / to | Det | Noun | Verb | End | | --- | --- | --- | --- | --- | | Start | 2/3 | 1/3 | 0 | 0 | | Det | 0 | 1 | 0 | 0 | | Noun | 0 | 0 | 3/5 | 2/5 | | Verb | 0 | 2/3 | 0 | 1/3 | All nonterminal rows sum to one. No smoothing is requested. ### Step 5. Write every sequence factor | Event | Probability | | --- | --- | | Start → Det | 2/3 | | Det emits the | 1/2 | | Det → Noun | 1 | | Noun emits dog | 2/5 | | Noun → Verb | 3/5 | | Verb emits eats | 2/3 | | Verb → Noun | 2/3 | | Noun emits bread | 1/5 | | Noun → End | 2/5 | ### Step 6. Multiply all factors $$ P=\frac23\times\frac12\times1\times\frac25\times\frac35\times\frac23\times\frac23\times\frac15\times\frac25 $$ Start with subtotal 1. At each row, multiply the previous subtotal by exactly one next factor. Simplify the fraction after multiplication, but do not round it. | Factor being included | Previous subtotal × next factor = new subtotal | | --- | --- | | 1. Start → Det | $$ 1\times\frac{2}{3}=\frac{2}{3} $$ | | 2. Det emits the | $$ \frac{2}{3}\times\frac{1}{2}=\frac{1}{3} $$ | | 3. Det → Noun | $$ \frac{1}{3}\times1=\frac{1}{3} $$ | | 4. Noun emits dog | $$ \frac{1}{3}\times\frac{2}{5}=\frac{2}{15} $$ | | 5. Noun → Verb | $$ \frac{2}{15}\times\frac{3}{5}=\frac{2}{25} $$ | | 6. Verb emits eats | $$ \frac{2}{25}\times\frac{2}{3}=\frac{4}{75} $$ | | 7. Verb → Noun | $$ \frac{4}{75}\times\frac{2}{3}=\frac{8}{225} $$ | | 8. Noun emits bread | $$ \frac{8}{225}\times\frac{1}{5}=\frac{8}{1125} $$ | | 9. Noun → End | $$ \frac{8}{1125}\times\frac{2}{5}=\frac{16}{5625} $$ | The last subtotal is the complete product. Only now convert the fraction to a decimal. $$ P=\frac{16}{5625}\approx0.0028444444 $$ ### Final answer **P = 16/5625 ≈ 0.0028444444** for these words and specified tags jointly. The two occurrences of eats give P(eats | Verb) = 2/3. **Yaad rakho:** Same tag structure ho sakta hai, lekin word emission counts dobara count karna. **Boundary reminder:** Include P(Det | Start) = 2/3 and P(End | Noun) = 2/5. The handwritten key’s final line instead inserts the overall Det fraction 2/10 and omits End; the complete boundary-aware probability requested here is 0.0028444444. --- ## TEE · Q5A · Why TF–IDF is useful Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/reexam-2026--reexam-2026-q5a/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/reexam-2026/reexam-2026-q5a Actual re-examination · 9 February 2026 · 100 marks · 3 hours Question. Justify with an appropriate example the importance of TF-IDF technique in the document representation and information retrieval task. [10 marks] ### Answer **TF–IDF** represents each document using weights that combine within-document term frequency with how unusual a term is across the collection. It supports information retrieval by giving larger weight to terms that are frequent in a document but not present in every document. ### Given Use three short documents: D₁ = “solar energy solar”; D₂ = “wind energy”; D₃ = “energy policy”. Use raw-count TF and natural-log unsmoothed IDF. N = 3. ### To find A TF–IDF representation illustrating why distinctive terms help retrieve relevant documents. ### Formula: which one and why $$ \mathrm{TFIDF}(t,d)=C(t,d)\ln\frac{N}{\mathrm{df}(t)} $$ C counts occurrences within a document; df counts documents containing the term. The logarithm reduces the scale of very large rarity ratios. ### Solution ### Step 1. Count term occurrences | Term | D₁ | D₂ | D₃ | df | | --- | --- | --- | --- | --- | | solar | 2 | 0 | 0 | 1 | | energy | 1 | 1 | 1 | 3 | | wind | 0 | 1 | 0 | 1 | | policy | 0 | 0 | 1 | 1 | ### Step 2. Calculate IDF $$ \mathrm{IDF}(solar)=\mathrm{IDF}(wind)=\mathrm{IDF}(policy)=\ln3\approx1.098612 $$ $$ \mathrm{IDF}(energy)=\ln(3/3)=0 $$ ### Step 3. Multiply each count by its IDF | Term | D₁ weight | D₂ weight | D₃ weight | | --- | --- | --- | --- | | solar | 2 × 1.098612 = 2.197224 | 0 | 0 | | energy | 0 | 0 | 0 | | wind | 0 | 1.098612 | 0 | | policy | 0 | 0 | 1.098612 | ### Step 4. Connect the representation to retrieval A query containing “solar” matches D₁’s high solar coordinate. A query containing “wind” matches D₂. “energy” alone cannot distinguish these documents under this unsmoothed collection because it occurs in all three. A retrieval system can compare query and document vectors, commonly by cosine similarity. ### Final answer TF–IDF preserves distinctive lexical evidence and reduces dominance by collection-wide frequent terms. D₁ is most relevant to a solar query under these features. **Yaad rakho:** Document mein frequent + collection mein rare = high TF–IDF. ### Strengths and limits The representation is sparse, interpretable and useful as a fast baseline. However, basic TF–IDF ignores word order and does not recognise synonyms or multiple word senses automatically. Its weights depend on the collection and on the chosen TF/IDF convention. Fit IDF on training or indexed documents, not on a hidden test set. --- ## TEE · Q5B · Sentence-window vectors and cosine Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/reexam-2026--reexam-2026-q5b/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/reexam-2026/reexam-2026-q5b Actual re-examination · 9 February 2026 · 100 marks · 3 hours Question. Consider the paragraph: The farmer planted seeds in the soil. The rain watered the soil. The sun helped the plants grow. The farmer harvested the plants. The soil became rich and fertile. Target words: farmer, soil, sun, plant. Context words: plant, water, grow, harvest. i. Construct a co-occurrence matrix for the target and context words. Take the window size as one whole sentence. ii. Evaluate the similarity between the vectors of the target words “farmer-soil”, “farmer-sun”, and “farmer-plant” using the cosine similarity. iii. Plot the target word vectors by considering only two dimensions: “planted” and “harvested”. [10 marks] ### Answer ### Given Five sentence windows. For alignment with the supplied matrix, target plant means the noun plants, occurring in sentences 3 and 4. Context plant covers planted and plants; watered → water; harvested → harvest. Count a target–context pair once per sentence in which both are present, including the plant–plant sentence-presence diagonal. ### To find The target–context matrix, three cosine similarities and a two-coordinate plot. ### Formula: which one and why $$ M_{ij}=\sum_s\mathbf1[\text{target}_i\text{ and context}_j\text{ present in }s] $$ $$ \cos(\mathbf a,\mathbf b)=\frac{\sum_j a_jb_j}{\sqrt{\sum_j a_j^2}\sqrt{\sum_j b_j^2}} $$ This is a sentence-presence matrix, not a position-based sliding window that excludes a token from its own context. Stating the convention is necessary because those alternatives give a different diagonal. ### Solution ### Step 1. Normalize and identify each sentence’s contribution | Sentence | Targets present | Context lemmas present | | --- | --- | --- | | 1 | farmer, soil | plant | | 2 | soil | water | | 3 | sun, plant | plant, grow | | 4 | farmer, plant | plant, harvest | | 5 | soil | none | ### Step 2. Fill all matrix rows | Target | plant | water | grow | harvest | | --- | --- | --- | --- | --- | | farmer | 2 | 0 | 0 | 1 | | soil | 1 | 1 | 0 | 0 | | sun | 1 | 0 | 1 | 0 | | plant | 2 | 0 | 1 | 1 | Farmer meets context plant in sentences 1 and 4. Soil meets plant in sentence 1 and water in sentence 2. Sun meets plant and grow in sentence 3. The noun plant occurs in sentences 3 and 4, producing its last row. ### Step 3. Compute farmer–soil $$ \mathbf f\cdot\mathbf s=(2)(1)+(0)(1)+(0)(0)+(1)(0)=2 $$ $$ \|\mathbf f\|=\sqrt{2^2+0^2+0^2+1^2}=\sqrt5,\quad\|\mathbf s\|=\sqrt2 $$ $$ \cos(\mathbf f,\mathbf s)=\frac2{\sqrt{10}}\approx0.632456 $$ ### Step 4. Compute farmer–sun $$ \mathbf f\cdot\mathbf u=(2)(1)+(0)(0)+(0)(1)+(1)(0)=2 $$ $$ \|\mathbf u\|=\sqrt2,\quad\cos(\mathbf f,\mathbf u)=\frac2{\sqrt{10}}\approx0.632456 $$ ### Step 5. Compute farmer–plant $$ \mathbf f\cdot\mathbf p=(2)(2)+(0)(0)+(0)(1)+(1)(1)=5 $$ $$ \|\mathbf p\|=\sqrt{2^2+0^2+1^2+1^2}=\sqrt6 $$ $$ \cos(\mathbf f,\mathbf p)=\frac5{\sqrt{30}}\approx0.912871 $$ ### Step 6. Plot the requested two coordinates Interpret the axis names planted and harvested as their normalized context features plant and harvest. Farmer=(2,1), soil=(1,0), sun=(1,0), plant=(2,1). Farmer and plant overlap; soil and sun overlap. 001122farmer, plantsoil, sunplant (planted)harvest (harvested) The plot uses only the requested two coordinates; cosine uses all context columns. ### Final answer Using the stated noun-target convention: **farmer–soil = 0.632456, farmer–sun = 0.632456, farmer–plant = 0.912871.** **Yaad rakho:** Plant noun aur plant verb ko same maana ya separate? Pehle convention likho, phir count karo. ### Two details that prevent an incorrect answer **Arithmetic:** 5/√30 = 0.912871; the source key’s printed approximation 0.9444 does not equal its own fraction. Keep the exact fraction until the last step. **Normalization:** If the question instead requires all noun and verb forms of plant to be merged as targets, sentence 1 also contains the plant target. Its row becomes [3,0,1,1], and farmer–plant becomes 7/√55 ≈ 0.943880. The paper leaves this choice unstated; the main worked solution explicitly uses the noun-target interpretation that reproduces its supplied matrix. --- ## TEE · Q6A · Pronoun approaches and lexical relations Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/reexam-2026--reexam-2026-q6a/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/reexam-2026/reexam-2026-q6a Actual re-examination · 9 February 2026 · 100 marks · 3 hours Question. i. Briefly describe any two approaches to pronoun resolution and mention one advantage of each. ii. Describe the difference between Homonymy and Synonymy with suitable examples. Discuss how these two relations differently affect natural language processing applications? [10 marks] ### Answer ### i. Two approaches to pronoun resolution | Approach | How it works | Advantage | | --- | --- | --- | | Rule-based / syntactic | Uses parse structure, agreement, recency and linguistic constraints. Hobbs’ algorithm searches parse trees for eligible antecedents. | Interpretable: a proposed antecedent can be traced to explicit rules. | | Statistical or neural | Learns candidate–pronoun compatibility or whole-document coreference patterns from labelled examples, using context and linguistic features or contextual embeddings. | Can learn interactions and contextual cues that are difficult to enumerate as hand-written rules. | For “Asha dropped the glass because it was slippery,” a system should evaluate candidate referents and context rather than merely choose the most recent noun. Rules can constrain number and syntax; learned models can use compatibility patterns, though neither approach is guaranteed to resolve every ambiguity. ### ii. Homonymy and synonymy | Relation | Definition | Example | | --- | --- | --- | | Homonymy | Same spelling or sound, unrelated meanings | bank as financial institution versus bank as river edge | | Synonymy | Different words with the same or closely related meanings in a context | car and automobile; big and large | Homonymy creates an ambiguity problem: a search for bank may retrieve finance or rivers. WSD uses context to choose the intended sense. Synonymy creates a vocabulary-matching problem: a document saying automobile may be relevant to a query saying car. Query expansion or semantic representations can improve recall by connecting related terms. The remedies differ. Do not merge unrelated homonym senses simply because their spelling matches. Do not insist on exact surface-form matching when different words express the same concept. Synonyms are also context-sensitive: “big mistake” and “large mistake” are not always equally natural. **Yaad rakho:** Homonym: same word-form, different meaning. Synonym: different words, similar meaning. --- ## TEE · Q6B · Hobbs: Ramesh and Shyam Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/reexam-2026--reexam-2026-q6b/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/reexam-2026/reexam-2026-q6b Actual re-examination · 9 February 2026 · 100 marks · 3 hours Question. i. Explain Hobbs’ Algorithm for pronoun resolution. State its main steps and limitations. ii. Apply Hobbs’ Algorithm to the sentence: “When Ramesh gave Shyam a book, he smiled.” Identify the antecedent of “he” by showing traversal steps. [10 marks] ### Answer **Hobbs’ algorithm** is a syntax-based pronoun-resolution procedure that searches constituency parse trees in a specified order. It proposes noun phrases as antecedents and filters incompatible candidates, for example by number and gender agreement. **Read the tree labels first:** NP means noun phrase; S means sentence/clause; VP means verb phrase. X is the current NP or S search node, and p is the upward path used to reach it. Breadth-first means shallower nodes before deeper nodes; left-to-right fixes the order at the same depth. An N-bar (N′) node is the noun-centred part of an NP in a parse using that label. ### Complete ordered procedure 1. **Locate the pronoun.** Start at the NP immediately dominating the pronoun: the smallest NP containing it. 2. **Make the first climb.** Move upward to the first NP or S above the starting NP. Call it X and record the path p. 3. **Search initially to the left.** Traverse branches below X to the left of p, breadth-first and left-to-right. Propose an NP only if another NP or S lies between that candidate and X. Check agreement and other applicable constraints for each proposed candidate; accept the first eligible compatible one. 4. **Check the highest sentence.** If X is the highest S and no candidate has been accepted, search previous sentence trees, most recent first. In each, traverse breadth-first and left-to-right and propose NPs. Accept the first eligible compatible candidate; if the available context contains none, report no resolved antecedent. 5. **Otherwise climb again.** Move from X to the next NP or S above it. Make that node the new X and record the new upward path p. 6. **Test a higher NP.** If the new X is an NP, propose X only when p did not pass through its nominal N-bar child. Apply the same compatibility checks. Use the labels in the supplied parse; do not invent a missing N-bar branch. 7. **Search left again.** Traverse the new X’s branches to the left of p, breadth-first and left-to-right, proposing NPs. The extra intervening-NP/S requirement in Step 3 belongs to the initial search. 8. **Search permitted right branches.** If X is an S, traverse branches to the right of p, breadth-first and left-to-right, but do not descend below any NP or S encountered. Propose the encountered eligible NPs and test their compatibility. 9. **Repeat or finish.** Return to Step 4 until a compatible candidate is accepted or the available sentence trees are exhausted. Record the candidate order and why each candidate was accepted or rejected. **Why the initial restriction matters:** in “John saw him”, the subject NP John is directly under the clause S, with no additional NP or S between them. Step 3 therefore does not propose John as the ordinary local antecedent of him. A reflexive such as himself is a different grammatical case. The worked question below shows the specific parse and traversal; selecting an antecedent is not simply choosing the nearest name. ### Given The pronoun he is the subject of the main clause. The preceding subordinate clause is “When Ramesh gave Shyam a book”. Assume the temporal clause attaches to the whole main sentence. ### To find The first suitable antecedent in the left-to-right breadth-first search of the preceding clause. ``` S (main) ← first X ├── SBAR: When │ └── S (embedded) ← intervening S │ ├── NP: Ramesh ← first candidate │ └── VP │ ├── V: gave │ ├── NP: Shyam │ └── NP: a book ├── NP: he ← start └── VP: smiled ``` ### Step 1. Locate the pronoun NP Start at NP “he” and move upward to its S ancestor. Record the path from he. ### Step 2. Search branches before the pronoun The initial when-clause lies to the left of the path. Apply Step 3: its embedded S lies between each of its noun phrases and the main S, so these NPs satisfy the initial intervening-NP/S requirement. Traverse left-to-right and breadth-first. ### Step 3. Propose noun phrases The subject NP Ramesh is encountered before the deeper object NP Shyam and object NP a book. Ramesh agrees with the assumed singular-male pronoun. ### Step 4. Accept the first compatible antecedent Select Ramesh. Shyam is also a semantically possible referent in ordinary discourse, but the algorithm’s stated syntactic preference selects Ramesh first. ### Limitations - Requires a suitable and accurate constituency parse. - Agreement information may be unknown or insufficient when several people match. - Ignores much of the event semantics, discourse focus and world knowledge. - An ambiguous parse or different attachment can change candidate order. **Final answer:** **Ramesh**, under the shown parse and ordinary agreement assumptions. A syntactic preference is not a guarantee about the author’s intended meaning. **Yaad rakho:** Left branches ko breadth-first dekho; pehla eligible matching NP choose karo. --- ## TEE · Q7A · Information retrieval Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/reexam-2026--reexam-2026-q7a/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/reexam-2026/reexam-2026-q7a Actual re-examination · 9 February 2026 · 100 marks · 3 hours Question. i. Explain the concept of Information Retrieval, highlighting its role in Recommender Systems and Search Engines. ii. Discuss the various steps involved in the Information Retrieval process. [10 marks] ### Answer ### i. Concept and roles **Information Retrieval (IR)** finds and ranks documents or other information objects relevant to a user’s information need. A query is the user’s expression of that need. IR typically returns a ranked set of items; information extraction instead identifies structured facts or fields inside those items. | Application | Role of IR | Example | | --- | --- | --- | | Search engine | Matches a submitted query against a large indexed collection and ranks candidate pages | A search for “symptoms of dehydration” returns relevant health pages | | Recommender system | Retrieves candidate items using a user profile, recent activity or a content representation; a later model may personalize ranking | Retrieve articles similar to topics the user reads | Retrieval is a component of many recommenders; recommendation also involves preferences, feedback, novelty and business constraints. A recommender need not wait for an explicit textual query. ### ii. Retrieval process Collect → Preprocess → Index → Process query → Retrieve candidates → Rank → Evaluate 1. **Collect and define documents.** Crawl or ingest documents, record metadata and remove unwanted duplicates. 2. **Preprocess.** Tokenize and normalize using rules suitable for both documents and queries. Preserve task-relevant information. 3. **Build representations and an index.** An inverted index maps each term to the documents containing it; a dense retrieval system stores vector representations. 4. **Process the query.** Apply compatible tokenization, spelling handling or expansion, then obtain query features. 5. **Retrieve candidates.** Use the index to identify potentially relevant documents efficiently. 6. **Score and rank.** A lexical method may use TF–IDF/BM25-style evidence; a semantic method compares representations. Rank scores estimate relevance rather than proving correctness. 7. **Present results.** Return titles, snippets and links, possibly with filtering or diversification. 8. **Evaluate and update.** Use relevance judgments and measures such as precision, recall or ranking metrics. Refresh the index as documents change. Example: for “solar energy storage”, an inverted index retrieves documents containing those terms; a ranking function rewards useful matches. A semantic system may also retrieve relevant material using related terms such as batteries. **Yaad rakho:** IR mein relevant document dhoondho aur rank karo; extraction mein document ke andar facts nikalo. --- ## TEE · Q7B · Gutenberg corpus and normalization Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/reexam-2026--reexam-2026-q7b/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/reexam-2026/reexam-2026-q7b Actual re-examination · 9 February 2026 · 100 marks · 3 hours Question. i. Discuss how the Gutenberg Corpus can be utilized for various text analysis tasks in Natural Language Processing? ii. Explain the importance of text normalization in Natural Language Processing (NLP). Illustrate your answer with a suitable example of an application where text normalization plays a crucial role. [10 marks] ### Answer ### i. Using the Gutenberg corpus The Gutenberg corpus provides literary texts, commonly used through tools such as NLTK’s Gutenberg collection. It supplies naturally occurring sentences and longer documents for exploring language patterns. The available subset and licensing should be checked before reuse in a new distribution. | Task | Use of the corpus | | --- | --- | | Frequency and vocabulary study | Count tokens and word types, inspect common words and compare lexical diversity across texts | | Concordances and collocations | View a word in context and discover words that occur near each other | | N-gram language modeling | Estimate unigram or bigram counts and compare sequence probabilities | | Stylistic or authorship analysis | Compare sentence lengths, vocabulary and function-word patterns across authors | | Text preprocessing experiments | Study the effect of tokenization, case normalization and lemmatization on counts | A sound workflow selects files, inspects encoding and boundaries, tokenizes consistently, computes the relevant statistics and interprets them against corpus composition. Literary text is not automatically representative of chat, medical records or present-day search queries; domain mismatch matters. ### ii. Text normalization **Normalization** converts selected surface variants into a consistent form so that equivalent expressions do not unnecessarily produce different features. Possible steps include case folding, whitespace cleanup, Unicode normalization, contraction expansion and lemmatization. Choose these according to the task. ### Example: product search 1. Raw query: “ RUNNING shoes ”. Product description: “Comfortable run shoes”. 2. Trim extra whitespace and apply consistent case: “running shoes”. 3. If the application chooses lemmatization, running → run. The query and the indexed description now share the intended run and shoes features. 4. Apply the same normalization to stored documents and future queries, otherwise matching remains inconsistent. Normalization reduces avoidable vocabulary variation and can improve recall. It must preserve distinctions important to the task: lowercasing US to us may lose meaning, and removing “not” can reverse sentiment. Aggressive stemming or synonym replacement can create incorrect matches. **Yaad rakho:** Same meaning ke surface variants ko consistent banao; useful meaning delete mat karo. --- ## TEE · Q1a · BoW and one-hot encoding Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r25--q1a/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-r25/q1a Previous examination · complete worked answers **Question. Explain Bag of Words and one-hot encoding with appropriate examples.** [individual allocation not printed; Q1 totals 20] ### Answer **Bag of words (BoW) represents a document by the counts of vocabulary terms, discarding their positions.** A binary BoW variant stores only presence/absence; a count BoW stores frequencies. State which is used. $$ x_{d,j}=C(v_j,d). $$ $$ x_{d,j} $$ is document $$ d $$ 's coordinate for vocabulary term $$ v_j $$ . $$ C(v_j,d) $$ is its occurrence count. A **bag of n-grams** uses short contiguous token sequences as features. Bigrams can distinguish “not good” from “good”, but increasing n enlarges the vocabulary and creates more rare features. **Question. Build count and binary BoW for D1=“cat cat dog” and D2=“dog fish”. Use vocabulary [cat,dog,fish] and no stopword removal.** #### Given Two tokenized documents, fixed vocabulary order. #### To find Count vectors and binary vectors. #### Formula and why it applies Count BoW uses $$ C(v_j,d) $$ . Binary BoW is 1 when that count is positive and 0 otherwise. #### Step 1. D1 counts cat=2, dog=1, fish=0 → [2,1,0]. #### Step 2. D2 counts cat=0, dog=1, fish=1 → [0,1,1]. #### Step 3. Convert to presence D1 [1,1,0]; D2 [0,1,1]. **Answer.** Count rows [2,1,0], [0,1,1]; binary rows [1,1,0], [0,1,1]. **Question. After lowercasing and removing {is,a,means}, obtain BoW vectors for D1=“Continuous learning is learning lifetime”, D2=“Continuous learning means lifetime learning”, D3=“Learning is a good practice”. Use [continuous,good,learning,lifetime,practice].** #### Given Three documents and exact stopword set from the supplied slides. #### To find The document–term count matrix. #### Formula and why it applies Remove only the listed stopwords, then count every remaining token in the fixed column order. #### Step 1. Clean D1 [continuous,learning,learning,lifetime] → [1,0,2,1,0]. #### Step 2. Clean D2 [continuous,learning,lifetime,learning] → [1,0,2,1,0]. #### Step 3. Clean D3 [learning,good,practice] → [0,1,1,0,1]. #### Step 4. Compare D1 and D2 They have identical BoW vectors despite different original wording/order. **Answer.** Rows [1,0,2,1,0], [1,0,2,1,0], [0,1,1,0,1]. **Question. Build a bag of bigrams for “not good but not bad” with vocabulary [not good,good but,but not,not bad]. Do not add sentence boundary tokens.** #### Given Five tokens; use adjacent pairs, stride 1. #### To find The bigram count vector. #### Formula and why it applies A sequence of $$ L $$ tokens has $$ L-1 $$ adjacent bigrams when no boundaries are added. #### Step 1. Enumerate pairs (not,good), (good,but), (but,not), (not,bad). #### Step 2. Count each vocabulary bigram Each appears once. #### Step 3. Write vector [1,1,1,1]. **Answer.** Four bigram features, each with count 1. BoW: order bhool, word-count rakh. Bag of bigrams: neighbouring word-pair count. **Label encoding** assigns each category or word a unique integer ID. **One-hot encoding** assigns a length- $$ V $$ vector with one 1 at the word's vocabulary position and zeros elsewhere. $$ \operatorname{onehot}(w)_j=\begin{cases}1,&j=\operatorname{index}(w)\\0,&\text{otherwise.}\end{cases} $$ The index is an ID, not a meaning score. One-hot represents one token/category; a multiword document generally needs multiple token vectors or an aggregation scheme. **Question. Encode “dog cat dog” using vocabulary [cat,dog,fish], IDs starting at 0. Give label IDs and a one-hot vector for every token.** #### Given cat→0, dog→1, fish→2. Vector columns have the same order. #### To find The ID sequence and three one-hot rows. #### Formula and why it applies Replace each word by its ID for labels; set its vocabulary position to 1 for one-hot. #### Step 1. Label IDs dog→1, cat→0, dog→1 → [1,0,1]. #### Step 2. First dog [0,1,0]. #### Step 3. Cat [1,0,0]. #### Step 4. Second dog [0,1,0]. Repeated tokens keep the same representation. **Answer.** IDs [1,0,1]; one-hot rows [0,1,0], [1,0,0], [0,1,0]. **Question. Encode the city sequence Mumbai, Delhi, Mumbai, Pune using vocabulary [Delhi,Mumbai,Pune] and label IDs starting at 1.** #### Given Delhi→1, Mumbai→2, Pune→3. #### To find IDs and one-hot rows. #### Formula and why it applies One-hot uses three coordinates regardless of whether label IDs begin at 0 or 1. #### Step 1. IDs [2,1,2,3]. #### Step 2. Mumbai rows Each Mumbai → [0,1,0]. #### Step 3. Delhi and Pune Delhi→[1,0,0]; Pune→[0,0,1]. **Answer.** Rows [0,1,0], [1,0,0], [0,1,0], [0,0,1]. **Question. One-hot encode categories [red,green,blue] for a linear regression with an intercept. Then give dummy coding after dropping blue.** #### Given Full column order [red,green,blue]. Blue is the baseline. #### To find Full and reduced codes. #### Formula and why it applies Full one-hot rows sum to 1. With an intercept this creates a linear dependency; dropping one category column gives k−1 independent dummy indicators. #### Step 1. Full encoding | Category | red | green | blue | | --- | --- | --- | --- | | red | 1 | 0 | 0 | | green | 0 | 1 | 0 | | blue | 0 | 0 | 1 | #### Step 2. Drop blue column | Category | red | green | | --- | --- | --- | | red | 1 | 0 | | green | 0 | 1 | | blue | 0 | 0 | #### Step 3. Interpret zero row [0,0] now identifies the baseline blue, not missing data. **Answer.** Full width 3; reduced dummy width 2. Dropping a column is model-dependent, not a requirement for every neural embedding input. **Limitations:** one-hot is large and sparse and does not express semantic similarity. Unseen words need a defined unknown-token or vocabulary-update policy. Label IDs are compact but should not be treated as an ordered scalar feature for nominal words. Label = ID number. One-hot = us ID ke box mein 1, baaki 0. --- ## TEE · Q1b · Lesk WSD Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r25--q1b/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-r25/q1b Previous examination · complete worked answers **Question. Discuss Lesk word-sense disambiguation with a suitable example.** [individual allocation not printed; Q1 totals 20] ### Answer **Lesk is a knowledge-based WSD method that chooses senses whose dictionary descriptions have the greatest overlap with contextual evidence.** Original Lesk compares definitions of neighbouring words' senses. A common simplified version compares each target-sense gloss directly with the sentence context. State which version you use. ### Simplified Lesk procedure 1. Remove the target word from the context; tokenize and normalize the remaining content words. 2. Obtain the gloss for each candidate sense and apply the same normalization. 3. Count shared words between the context and each gloss. 4. Select the sense with maximum overlap. If scores tie, report the tie or use a specified tie-break rule. $$ \operatorname{score}(s)=|C\cap G_s|,\qquad s^*=\arg\max_s\operatorname{score}(s) $$ $$ C $$ is the set of context words, $$ G_s $$ the set of words in sense $$ s $$ 's gloss, $$ \cap $$ means common elements, $$ |\cdot| $$ means count, and $$ \arg\max $$ means “the sense giving the largest score”. This version counts unique matching words once. **Question. Use simplified Lesk to disambiguate “bank” in “The river water rose near the bank.” Use C = {river, water, rose, near}; financial gloss = {institution, money, deposit}; river gloss = {land, river, water, edge}.** #### Given The normalized context and two gloss sets are given. Count exact shared words. #### To find The score of each sense and the selected meaning. #### Formula and why it applies Use $$ \operatorname{score}(s)=|C\cap G_s| $$ because the task specifies context–gloss overlap. #### Step 1. Compare the financial sense No word occurs in both {river, water, rose, near} and {institution, money, deposit}. Overlap = {}. #### Step 2. Count the financial score $$ \operatorname{score}(F)=0 $$ . #### Step 3. Compare the river sense Shared words are {river, water}. “Rose” and “near” are not in the gloss. #### Step 4. Count and select $$ \operatorname{score}(R)=2 $$ . Since $$ 2>0 $$ , choose the river-edge sense. **Answer.** Bank means land beside a river. **Question. Use simplified Lesk for “The dog barked loudly.” Treat barked as bark. Context C = {dog, loudly}; sound gloss = {sharp, cry, dog}; tree gloss = {outer, covering, tree}; ship gloss = {sailing, ship}.** #### Given Three glosses and normalized context {dog, loudly}. #### To find Select the sense of bark. #### Formula and why it applies $$ \operatorname{score}(s)=|C\cap G_s| $$ . Lemmatization lets barked match the target lemma bark; it does not create additional overlap words. #### Step 1. Sound sense Intersection = {dog}; score = 1. #### Step 2. Tree sense Intersection = {}; score = 0. #### Step 3. Ship sense Intersection = {}; score = 0. #### Step 4. Compare The sound sense has the unique maximum: 1 > 0. **Answer.** Bark is the dog's cry. **Question. Apply definition–definition Lesk to “pine cone”. Pine senses: P1 = {evergreen, tree, needle, leaf}; P2 = {waste, sorrow, illness}. Cone senses: C1 = {solid, shape, circular}; C2 = {fruit, evergreen, tree}.** #### Given Two candidate senses for each word. Tokens are already normalized; tree and trees would be reduced to the same form if necessary. #### To find The most compatible pair of senses. #### Formula and why it applies Use $$ |G_{P_i}\cap G_{C_j}| $$ for every pair because this is original-style gloss–gloss comparison. #### Step 1. P1 with C1 No shared word → 0. #### Step 2. P1 with C2 Shared {evergreen, tree} → 2. #### Step 3. P2 with each cone sense P2–C1 has no overlap → 0; P2–C2 has no overlap → 0. #### Step 4. Choose the best pair The four scores are 0, 2, 0, 0. Maximum 2 belongs to P1–C2. **Answer.** Pine is the evergreen tree; cone is its fruit-like reproductive structure. **Limitations:** exact overlap misses synonyms, short glosses may produce zeros, and different preprocessing choices change the score. Extended Lesk may include related synset definitions or longer phrase overlaps; do not silently substitute it for a simple question. Lesk: context aur dictionary mein common words gino; sabse bada overlap jeeta. --- ## TEE · Q1c · Syntax parsing approaches Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r25--q1c/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-r25/q1c Previous examination · complete worked answers **Question. Explain syntax parsing and compare top-down and bottom-up approaches.** [individual allocation not printed; Q1 totals 20] ### Answer **A context-free grammar (CFG)** specifies a language using production rules whose left side is a single nonterminal. Formally, G=(N,Σ,R,S): N is the set of nonterminals, Σ the terminal vocabulary, R the production rules, and S the start symbol. A rule can expand its nonterminal regardless of the neighboring symbols; that is the meaning of “context-free”. | Concept | Plain explanation | Example | | --- | --- | --- | | Terminal | A word at a leaf; it is not expanded further | the, girl, read | | Nonterminal | A grammatical category that can be expanded | S, NP, VP, Det, N, V | | Production | A permitted replacement | S → NP VP | | Start symbol | Category for a complete sentence | S | | Lexicon | Word-to-category rules | N → girl | paper | | Derivation | Sequence of rule applications | S ⇒ NP VP ⇒ Det N VP ⇒ … | | Parse tree | Hierarchical picture of those expansions | Root S, internal phrases, word leaves | $$ S\to NP\ VP,\quad NP\to Det\ N,\quad VP\to V\ NP $$ Read S→NP VP as “a sentence can contain a noun phrase followed by a verb phrase”. It does not mean subtraction or multiplication. A vertical bar in a grammar rule means alternatives: Det→the | a allows either the or a. This is different from the probability bar meaning “given”. S: the whole sentence → NP: the girl + VP: read the paper → NP→Det N; VP→V NP → Leaves in order: the / girl / read / the / paper **A constituent** is a group of words that acts as a grammatical unit, such as “the girl”. **Syntactic parsing** assigns a grammatical structure to an input sentence using a grammar. It can support grammar checking, information extraction, translation and later semantic analysis. Failure under one restricted grammar means that grammar could not derive the sentence; it does not necessarily prove the sentence is invalid English. **A treebank** is a corpus annotated with parse trees. It supplies training/evaluation structures for parsers and lets linguists study recurring syntactic patterns. A tagged corpus gives word tags; a treebank additionally records relationships among phrases. | Representation | Example information retained | | --- | --- | | Bag of words | Words occur, but word order/grammar is lost | | POS tags | Each word's local grammatical category | | Chunks | Short phrase boundaries | | Parse tree | Nested constituent structure of the whole sentence | **Yaad rakho:** Grammar rule ek expansion instruction hai: S ko NP aur VP mein kholo. **Top-down parsing** starts at the grammar's start symbol S and expands nonterminals until the leaves match the input words in order. **Recursive descent** commonly implements each nonterminal as a procedure that tries its productions. It builds from the root toward the leaves. Start with S → Choose an allowed S production → Expand its next nonterminal → Match the resulting terminal to the input → Continue, or backtrack if the choice fails **Algorithm.** Start with S as the current goal. Choose a production for the leftmost unexpanded nonterminal. Replace that nonterminal by the production's right side. If a terminal appears, compare it with the next input token. If it matches, consume the token; if it does not, reject that branch and try another production. Accept only when the complete input is consumed and all grammar goals are satisfied. A **leftmost derivation** always expands the leftmost remaining nonterminal first. Top-down parsing can make guesses that the actual words cannot support, so it may need backtracking. A naive recursive-descent parser can loop on left-recursive rules such as NP→NP PP unless the grammar or algorithm is changed. Predictive LL(1) parsing chooses productions using one lookahead token when the grammar permits; not every English CFG is LL(1). | Approach | Begins with | Direction | Common difficulty | | --- | --- | --- | --- | | Top-down | Start symbol S | Root to word leaves | Predictions that do not match input; left recursion | | Bottom-up | Actual input tokens | Leaves to root S | Reductions that cannot contribute to a complete sentence | **Yaad rakho:** Top-down: sentence ka structure pehle, words tak neeche aao. **Bottom-up parsing** starts from the input tokens and combines them into larger constituents until the start symbol is reached. A **shift-reduce parser** maintains a stack of recognized material and a list of remaining input tokens. | Action | Exact operation | Example | | --- | --- | --- | | Shift | Move the next input token onto the stack | Stack: the; remaining input: dog runs | | Reduce | Replace a suffix of the stack matching a rule's RHS with its LHS | Det N becomes NP using NP→Det N | | Accept | Stack is S and input is exhausted | Stack: S; input: empty | | Fail/backtrack | Chosen reductions cannot complete a parse | A noun reading of book blocks an imperative parse | Reduction uses a production in reverse: the grammar NP→Det N expands an NP top-down, but reduces Det N into NP bottom-up. The stack's **rightmost item is its top** in the traces below. Each row shows the state **after** the listed action, avoiding an off-by-one mismatch between actions and stack contents. A lexical reduction such as N→dog still counts as a reduction. Do not skip the sequence from word to category to phrase when the question asks for a complete trace. Backtracking means returning to an earlier choice and trying another applicable rule, not changing the original input. **Yaad rakho:** Shift = input se stack. Reduce = chhote pieces se bada grammatical unit. --- ## TEE · Q1d · NLU versus NLG and applications Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r25--q1d/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-r25/q1d Previous examination · complete worked answers **Question. Compare Natural Language Understanding and Natural Language Generation. List real-world NLP applications that use these concepts.** [individual allocation not printed; Q1 totals 20] ### Answer **Natural language understanding (NLU)** maps language into an interpretation useful for a task: intents, entities, semantic relations or another meaning representation. **Natural language generation (NLG)** produces human-language output from data, a representation or another source. “Book a ticket to Pune tomorrow.” → NLU: intent=book; destination=Pune; date=tomorrow → Application: search available journeys → NLG: “Here are the available journeys to Pune.” | Feature | NLU | NLG | | --- | --- | --- | | Direction | Language → interpretation | Data/representation → language | | Main concern | Meaning, intent, ambiguity | Content choice, organization, fluent wording | | Example | Find destination in a request | Write the booking confirmation | | Common failure | Wrong destination or sense | Unclear, unsupported or ungrammatical output | ### NLG stages in a classical pipeline 1. **Content determination:** choose facts to communicate. 2. **Data interpretation:** identify relationships or trends. 3. **Document planning:** organize facts in a useful order. 4. **Sentence aggregation:** combine related facts without repetition. 5. **Lexical and grammatical realization:** choose words and construct grammatical sentences. 6. **Presentation:** format the generated message for its audience. For a weather report, facts {city=Pune, temperature=28°C, rain=yes} can become “Pune will have rain with a temperature of 28°C.” NLU would instead extract facts or intent from an incoming language message. NLU = language se meaning andar. NLG = data se language bahar. A question-answering assistant interprets intent/entities through NLU and composes an answer through NLG. Translation interprets source meaning and generates a target-language sequence. A customer-support system classifies requests and generates a contextual reply. --- ## TEE · Q2a · TF–IDF cosine after a fixed stoplist Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r25--q2a/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-r25/q2a Previous examination · complete worked answers **Question. For “The cat sat on the mat.”, “The dog barked at the cat.” and “I love my dog and my cat.” remove stopwords {I,the,on,my,at,and}; construct TF–IDF and calculate cosine similarity between the first two documents.** [10 marks] ### Answer ### Given | Document | Original text | | --- | --- | | 1 | The cat sat on the mat. | | 2 | The dog barked at the cat. | | 3 | I love my dog and my cat. | Stopwords: at, my, i, on, the, and ### To find BoW, IDF and TF–IDF plus requested cosine ### Formula: which one and why Use lowercase alphabetic tokens and exactly the stated stoplist. If no stoplist is supplied, retain function words. Adopt raw-count TF and natural-log IDF; the paper does not prescribe another TF/log convention. $$ \mathrm{TFIDF}(w,d)=c(w,d)\ln\frac{N}{\mathrm{df}(w)} $$ A term shared by all N documents has IDF ln(1)=0. Counts and TF–IDF weights are different representations. ### Step 1. Tokenize each document | Document | Tokens | | --- | --- | | 1 | cat, sat, mat | | 2 | dog, barked, cat | | 3 | love, dog, cat | ### Step 2. Fix the vocabulary order barked, cat, dog, love, mat, sat ### Step 3. Count occurrences (Bag of Words) | Document | barked | cat | dog | love | mat | sat | | --- | --- | --- | --- | --- | --- | --- | | 1 | 0 | 1 | 0 | 0 | 1 | 1 | | 2 | 1 | 1 | 1 | 0 | 0 | 0 | | 3 | 0 | 1 | 1 | 1 | 0 | 0 | ### Step 4. Calculate document frequency and IDF | Term | df | N / df | ln(N/df) | | --- | --- | --- | --- | | barked | 1 | 3 | 1.098612289 | | cat | 3 | 1 | 0.000000000 | | dog | 2 | 3/2 | 0.405465108 | | love | 1 | 3 | 1.098612289 | | mat | 1 | 3 | 1.098612289 | | sat | 1 | 3 | 1.098612289 | ### Step 5. Multiply each TF by its IDF | Document | barked | cat | dog | love | mat | sat | | --- | --- | --- | --- | --- | --- | --- | | 1 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 1.098612 | 1.098612 | | 2 | 1.098612 | 0.000000 | 0.405465 | 0.000000 | 0.000000 | 0.000000 | | 3 | 0.000000 | 0.000000 | 0.405465 | 1.098612 | 0.000000 | 0.000000 | ### Step 6. Calculate cosine from these same TF–IDF rows $$ \cos(u,v)=\frac{\sum_i u_i v_i}{\sqrt{\sum_i u_i^2}\sqrt{\sum_i v_i^2}} $$ Coordinate products: 0.000000000, 0.000000000, 0.000000000, 0.000000000, 0.000000000, 0.000000000 Dot product=0.000000000; squared norms=2.413897922, 1.371350915. Cosine=0.000000000 ### Final answer The final table is the TF–IDF matrix in the declared vocabulary order. A different explicitly stated log base rescales all IDFs; length-normalized TF changes each document scale. Cosine is 0.0. Shared words whose IDF is zero contribute nothing to the TF–IDF dot product. ### Which terms are prominent? D1: sat and mat have IDF ln3; D2: barked has ln3 and dog has ln(3/2); D3: love has ln3 and dog has ln(3/2). Cat occurs in all three documents and has zero unsmoothed IDF. Thus shared cat does not make D1/D2 TF–IDF cosine positive. --- ## TEE · Q2b · Weather Naive Bayes Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r25--q2b/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-r25/q2b Previous examination · complete worked answers **Question. Use the supplied class/conditional probabilities to predict Play Tennis for Outlook=Sunny, Temperature=Cool, Humidity=High, Windy=True.** [10 marks] ### Answer ### Given | Class | Prior | Sunny | Cool | High | True | | --- | --- | --- | --- | --- | --- | | Yes | 9/14 | 2/9 | 3/9 | 3/9 | 3/9 | | No | 5/14 | 3/5 | 1/5 | 4/5 | 3/5 | ### To find Compare the Yes and No class scores. ### Formula: which one and why $$ \mathrm{score}(c)=P(c)P(Sunny\mid c)P(Cool\mid c)P(High\mid c)P(True\mid c) $$ ### Step 1. Multiply the Yes factors 9/14 × 2/9 × 3/9 × 3/9 × 3/9 = 1/189 = 0.005291005 ### Step 2. Multiply the No factors 5/14 × 3/5 × 1/5 × 4/5 × 3/5 = 18/875 = 0.020571429 ### Step 3. Compare and normalize Yes=0.204582651, No=0.795417349 ### Final answer **Predict No.** Apply the supplied categorical probabilities directly; adding vocabulary smoothing would solve a different problem. --- ## TEE · Q3a · Coreference and anaphora Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r25--q3a/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-r25/q3a Previous examination · complete worked answers **Question. Explain coreference and anaphoric reference with examples.** [5 marks] ### Answer **Pragmatic analysis** interprets an utterance using its situation and the speaker’s intended meaning. **Discourse analysis** studies how utterances connect across a text or conversation. Syntax and literal word meanings alone cannot settle every interpretation. | Text | Literal content | Contextual interpretation | | --- | --- | --- | | “Can you close the window?” | A question about ability. | Usually a polite request. | | “Meera bought a laptop. It was expensive.” | Two sentences. | “It” connects the second sentence to the laptop. | | “Meet me here tomorrow.” | An instruction. | Me, here and tomorrow depend on speaker, place and date. | ### Reference, mention and antecedent A **referring expression** points to an entity; a **mention** is its occurrence in the text. Its **referent** is the entity being discussed. An **antecedent** is a preceding expression used to interpret a later referring expression. Reference resolution identifies the intended referent. Coreference resolution groups mentions that refer to the same entity. “Meera”name mention → Person entitysame referent → “she” / “the engineer”other mentions | Concept | Definition | Example | | --- | --- | --- | | Anaphora | Reference back to something introduced earlier. | “Riya arrived. She sat down.” She → Riya. | | Cataphora | Reference forward to a later expression. | “Before she spoke, Riya checked the microphone.” she → Riya. | | Exophora | Reference to the situation outside the text. | “Put it there”, said while pointing. | | Coreference | Two or more mentions refer to the same entity. | “Dr Rao”, “the lecturer”, “she”. | | Pronoun resolution | Find the referent of a pronoun. | Resolve “it” in “The laptop stopped because it overheated.” | Coreference is broader than pronoun resolution: repeated names and descriptions also form a chain. Anaphora describes direction/dependence; coreference describes identity. In “I bought a car. The engine is noisy”, engine is related to the car but is not the same entity. **Yaad rakho:** Reference = kiski baat? Coreference = alag mentions, same entity. Pronoun resolution = he/she/it kis se linked? ### Cohesion, coherence and discourse markers **Cohesion** is the explicit linguistic linking of a text through reference, repeated or related words, conjunctions and other devices. In “Rita bought a book. She read it”, she and it connect the second sentence to Rita and the book. **Coherence** is the conceptual unity that makes a discourse meaningful. “The road flooded, so the match was cancelled” presents a plausible cause and consequence. A reader relates the propositions using context and knowledge. **Discourse markers** signal relationships or organize speech: however signals contrast; therefore signals a result; first and next signal sequence; for example introduces an illustration. These concepts differ: “I like apples. Therefore the moon is square” has a linking word but no sensible inference. A connector alone does not guarantee coherence. **Yaad rakho:** Cohesion = words se link; coherence = meaning se link; marker = relation ka signal. ### A practical resolution procedure First detect the pronoun and candidate noun phrases. Eliminate candidates inconsistent with person, number, grammatical binding or relevant semantic constraints. Rank the surviving candidates using syntactic prominence, recency and discourse context. Preserve ambiguity if the available evidence does not decide. | Approach | How it works | Strength | Limitation | | --- | --- | --- | --- | | Rule based / syntactic | Use parse trees, agreement and explicit search order; Hobbs is a classic example. | Can explain each decision without labelled training data. | Parsing errors and world knowledge gaps cause errors. | | Salience based | Score candidates using recency, subjecthood and other discourse features; select the strongest compatible candidate. | Captures the tendency to refer again to prominent entities. | Feature weights and competing candidates can mislead. | | Supervised statistical | Learn candidate/pronoun compatibility from labelled examples. | Combines many features from data. | Needs representative annotated examples. | | Neural / contextual | Encode the surrounding text and score mention pairs or antecedents. | Can learn contextual patterns and long-range dependencies. | Needs data/compute and can still make confident mistakes. | ### Worked analysis 1 · number and semantics Resolve “they” in: “Riya put the books on the shelf because they were heavy.” **Given** Candidates: Riya (singular person), books (plural objects), shelf (singular object); pronoun they is plural here. **To find** The antecedent and a justified resolution procedure. ### Formula and why it applies Use number agreement first, then semantic compatibility. There is no multiplication formula in this linguistic task. ### Solution #### Step 1. List candidates Riya, the books, the shelf. #### Step 2. Apply number agreement Riya and shelf are singular. Books is plural, matching they. #### Step 3. Check meaning Books can be heavy; the causal explanation fits carrying/placing books. “They” refers to “the books”. ### Worked analysis 2 · a complete coreference chain Identify the coreference chain in: “Dr Sen entered. The surgeon greeted Ravi. She opened her notebook.” **Given** Mentions: Dr Sen, the surgeon, Ravi, she, her, her notebook. Assume the surgeon is Dr Sen and she/her refers to that surgeon. **To find** The person chain, and a mention that must not be placed in it. ### Formula and why it applies Group expressions by identity of referent; a possessive expression and the owned object are different entities. ### Solution #### Step 1. Link the name and description Dr Sen and the surgeon denote the same person under the given context. #### Step 2. Link the pronouns She and her refer to the surgeon. #### Step 3. Keep the notebook separate “Her notebook” is an object owned by the person; it is not the person herself. Person chain: {Dr Sen, the surgeon, She, her}. “Her notebook” is a separate object mention. ### Worked analysis 3 · honest ambiguity Discuss the pronoun in: “Asha called Nisha after she finished work.” **Given** Both named candidates are singular; the sentence gives no further context. **To find** Whether one referent is logically guaranteed. ### Formula and why it applies Agreement filters incompatible candidates but cannot always select between equally compatible candidates. ### Solution #### Step 1. Check agreement Both Asha and Nisha are compatible with she. #### Step 2. Check the event Either person could have finished work. #### Step 3. State the limitation A heuristic may prefer a candidate, but grammar and this sentence alone do not prove the intended referent. Ambiguous without further context. State any heuristic and parse assumption before giving its preferred candidate. --- ## TEE · Q3b · Cohesion, coherence and discourse markers Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r25--q3b/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-r25/q3b Previous examination · complete worked answers **Question. Explain cohesion, coherence and discourse markers.** [5 marks] ### Answer **Pragmatic analysis** interprets an utterance using its situation and the speaker’s intended meaning. **Discourse analysis** studies how utterances connect across a text or conversation. Syntax and literal word meanings alone cannot settle every interpretation. | Text | Literal content | Contextual interpretation | | --- | --- | --- | | “Can you close the window?” | A question about ability. | Usually a polite request. | | “Meera bought a laptop. It was expensive.” | Two sentences. | “It” connects the second sentence to the laptop. | | “Meet me here tomorrow.” | An instruction. | Me, here and tomorrow depend on speaker, place and date. | ### Reference, mention and antecedent A **referring expression** points to an entity; a **mention** is its occurrence in the text. Its **referent** is the entity being discussed. An **antecedent** is a preceding expression used to interpret a later referring expression. Reference resolution identifies the intended referent. Coreference resolution groups mentions that refer to the same entity. “Meera”name mention → Person entitysame referent → “she” / “the engineer”other mentions | Concept | Definition | Example | | --- | --- | --- | | Anaphora | Reference back to something introduced earlier. | “Riya arrived. She sat down.” She → Riya. | | Cataphora | Reference forward to a later expression. | “Before she spoke, Riya checked the microphone.” she → Riya. | | Exophora | Reference to the situation outside the text. | “Put it there”, said while pointing. | | Coreference | Two or more mentions refer to the same entity. | “Dr Rao”, “the lecturer”, “she”. | | Pronoun resolution | Find the referent of a pronoun. | Resolve “it” in “The laptop stopped because it overheated.” | Coreference is broader than pronoun resolution: repeated names and descriptions also form a chain. Anaphora describes direction/dependence; coreference describes identity. In “I bought a car. The engine is noisy”, engine is related to the car but is not the same entity. **Yaad rakho:** Reference = kiski baat? Coreference = alag mentions, same entity. Pronoun resolution = he/she/it kis se linked? ### Cohesion, coherence and discourse markers **Cohesion** is the explicit linguistic linking of a text through reference, repeated or related words, conjunctions and other devices. In “Rita bought a book. She read it”, she and it connect the second sentence to Rita and the book. **Coherence** is the conceptual unity that makes a discourse meaningful. “The road flooded, so the match was cancelled” presents a plausible cause and consequence. A reader relates the propositions using context and knowledge. **Discourse markers** signal relationships or organize speech: however signals contrast; therefore signals a result; first and next signal sequence; for example introduces an illustration. These concepts differ: “I like apples. Therefore the moon is square” has a linking word but no sensible inference. A connector alone does not guarantee coherence. **Yaad rakho:** Cohesion = words se link; coherence = meaning se link; marker = relation ka signal. ### Definitions and examples requested in this part | Term | Definition | Example | | --- | --- | --- | | Cohesion | Explicit language links connecting text: pronouns, repetition, conjunctions, lexical ties | Rita bought a book. She read it. She/it refer back to Rita/book. | | Coherence | Conceptual unity and plausible relations making the text meaningful | The road flooded, so the match was cancelled: cause and consequence connect the propositions. | | Discourse marker | Expression signalling relation or organizing the discourse | however=contrast; therefore=result; first/next=sequence; for example=illustration | “I like apples. Therefore the moon is square” contains a connector but lacks a sensible inference: surface cohesion does not guarantee coherence. --- ## TEE · Q3c · Question answering system Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r25--q3c/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-r25/q3c Previous examination · complete worked answers **Question. Explain a question-answering system with its architecture and an example.** [10 marks] ### Answer **Question answering (QA) returns an answer to a natural-language question using available evidence or knowledge.** Information retrieval usually returns ranked documents; QA must identify the requested information and deliver the answer. A relevant document is useful evidence but is not automatically the answer. ### Question types and answer types | Question | Expected answer type | Required operation | | --- | --- | --- | | Who founded the company? | Person or organisation | Find the founder relation | | When did the museum open? | Date/time | Locate the opening date | | Where is the campus? | Location | Locate the campus-location relation | | How many books were issued? | Quantity | Read or compute the stated count | | Why was the train cancelled? | Reason/explanation | Find the causal evidence | | Is the office open on Sunday? | Yes/no with evidence | Check the relevant schedule statement | **Closed-domain QA** operates within a restricted area such as a university handbook. **Open-domain QA** addresses a wider range of subjects and usually needs broad retrieval. **Extractive QA** selects a span from a passage. **Generative QA** constructs answer text; it still must preserve the evidence and avoid unsupported details. Question analysis → Evidence retrieval → Passage/record understanding → Answer selection or composition → Support check → Present answer ### Explain each module | Module | What happens | Why it matters | | --- | --- | --- | | Question analysis | Identify topic, intent, constraints and answer type. | When asks for a time, not a place. | | Retrieval | Find a relevant passage or structured fact. | Reduces the search space. | | Evidence understanding | Parse relations, resolve references and ambiguous words. | The correct date must belong to the correct event. | | Answer formation | Extract a span or compose a short supported response. | Turns a document into an actual answer. | | Verification | Check whether every claimed detail follows from evidence. | Prevents filling gaps by guessing. | | Presentation | Return a concise answer with suitable context. | Avoids dumping unrelated text. | ### Worked application 1 · extract the correct fact **Question. Given “The City Museum opened in 1998 in Pune. It was renovated in 2016”, answer “When did the museum open?” Show the QA stages.** #### Given A two-sentence passage. Dates 1998 and 2016 refer to different events. #### To find The opening date, not the renovation date. #### Method and why it applies Use extractive QA: identify the requested event and expected date type, then choose the span linked to that event. ### Solution #### Step 1. Analyse the question When requires a date/time answer. The event is opening; the entity is the City Museum. #### Step 2. Retrieve the relevant sentence The first sentence mentions the museum and its opening. The second sentence discusses renovation. #### Step 3. Identify event–date relations | Event | Date | | --- | --- | | opened | 1998 | | renovated | 2016 | #### Step 4. Select the answer span 1998 is attached to opened, so select the four-character year span 1998. #### Step 5. Reject the distractor 2016 is also a date, but it answers a different question: when was the museum renovated? #### Step 6. Present the answer “The museum opened in 1998.” **Answer.** 1998. Matching only the answer type date would be insufficient without matching the event. ### Worked application 2 · resolve a pronoun before answering **Question. Given “Mira joined Orion Labs in 2022. She became its research lead in 2024”, answer “Who became Orion Labs' research lead, and when?”** #### Given The passage contains the name Mira, pronoun She, organisation Orion Labs and possessive its. Assume the normal reading that She refers to Mira and its to Orion Labs. #### To find The person and the date of becoming research lead. #### Method and why it applies Combine reference resolution with relation extraction. The question asks for two fields, both tied to the promotion/role event. ### Solution #### Step 1. Identify requested fields Who → person; when → date. The relation is became research lead of Orion Labs. #### Step 2. Resolve mentions | Mention | Referent | | --- | --- | | She | Mira | | its | Orion Labs | #### Step 3. Read the target event After resolution, the second sentence states that Mira became Orion Labs' research lead in 2024. #### Step 4. Distinguish the dates 2022 belongs to joining the organisation. 2024 belongs to becoming research lead. #### Step 5. Compose a supported answer “Mira became Orion Labs' research lead in 2024.” #### Step 6. Verify completeness The answer supplies both requested fields and does not add an unsupported department, qualification or reason. **Answer.** Mira, in 2024. Reference resolution connects the second sentence to the named person and organisation. ### Worked application 3 · detect insufficient evidence **Question. A handbook excerpt states “The library opens at 8 a.m. on weekdays.” A user asks “At what time does it close on Sunday?” Explain the correct QA response.** #### Given Only the opening time for weekdays is supplied. No closing time or Sunday schedule is present. #### To find Whether an answer is supported and how to handle the missing information. #### Method and why it applies Compare every question constraint against the evidence. Topic overlap with library is not sufficient: time type and day must also match. ### Solution #### Step 1. Extract the question constraints | Constraint | Requested | | --- | --- | | Entity | Library | | Event | Closing | | Day | Sunday | | Answer type | Time | #### Step 2. Extract the supplied fact Entity=library; event=opening; day=weekdays; time=8 a.m. #### Step 3. Identify mismatches Opening is not closing. Weekdays do not supply a Sunday schedule. #### Step 4. Avoid the unsupported substitution Do not answer 8 a.m. just because it is the only time in the passage. #### Step 5. Return an evidence-limited response “The excerpt does not state the library's Sunday closing time.” A system with further retrieval can search a relevant schedule. #### Step 6. State the benefit The system avoids converting a relevant-looking number into an incorrect answer. **Answer.** No Sunday closing time is supported by the excerpt. Report insufficient evidence or retrieve the missing schedule. ### Advantages, limits and evaluation | Design choice | Benefit | Limitation | | --- | --- | --- | | Extractive answer | The returned wording comes directly from evidence. | The answer may require combining sentences or may not be present as one span. | | Generative answer | Can combine supported facts into a readable response. | Can introduce unsupported information unless carefully checked. | | Closed-domain collection | Controlled terminology and relevant resources. | Cannot answer facts outside its knowledge source. | | Broad retrieval | Wider question coverage. | Irrelevant, outdated or conflicting sources can be retrieved. | Evaluate **exact match** when a canonical answer string is expected, **token overlap/F1** when wording may vary, and **factual correctness and completeness** when answers are composed. An answer can be grammatically fluent but wrong. Questions with no answer in the evidence should be included in evaluation so the system's restraint is also tested. For a longer exam answer, define QA, distinguish it from retrieval, explain answer types and system variants, draw the pipeline, work through a passage including distractors, and discuss ambiguity, missing evidence and evaluation. A list of chatbot names is not a QA design. **Yaad rakho:** Question ka entity + event + constraint match karo. Passage mein koi bhi number mil gaya matlab answer nahi. --- ## TEE · Q4a · NLP pipeline Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r25--q4a/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-r25/q4a Previous examination · complete worked answers **Question. Explain the stages of an NLP pipeline.** [5 marks] ### Answer **An NLP pipeline is an organized sequence of operations that transforms raw language into a task output and checks whether that output is useful.** Not every task needs every cleaning operation. Data acquisition → Preprocessing → Feature representation → Model → Evaluation → Deployment and monitoring 1. **Data acquisition:** collect relevant text and, when needed, labels. 2. **Cleaning and preprocessing:** handle markup, tokenize, normalize and decide which information to retain. 3. **Feature engineering:** represent text numerically using BoW, TF–IDF or embeddings. 4. **Modelling:** apply rules or train a statistical/neural model. 5. **Evaluation:** test on held-out data using task-appropriate measures. 6. **Deployment:** make the system available, log suitable feedback and monitor errors or data changes. ### Concrete spam-filter example Collect labelled emails → separate train/test data → tokenize text while preserving meaningful terms → fit vocabulary and IDF on training data → train classifier → evaluate spam detection on the held-out set → deploy an email service filter → monitor legitimate emails wrongly blocked. **Intrinsic evaluation** measures model/task quality, such as precision, recall or tagging accuracy. **Extrinsic evaluation** measures effect in the wider application, such as whether users find answers faster. A higher internal metric does not automatically guarantee a better user outcome. **Data leakage:** learning vocabulary statistics, labels or preprocessing decisions from the test set can make reported performance too optimistic. Fit learned transformations on training data, then apply them to validation/test data. Collect → clean → numbers → model → test → use. Test data ko training ki information mat banao. **Definition.** NLP is the study of computational methods for analyzing, understanding and generating human language. The pipeline diagram above turns raw language into a task-specific prediction or generated response. --- ## TEE · Q4b · Morphology and syntax Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r25--q4b/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-r25/q4b Previous examination · complete worked answers **Question. Explain the importance of morphology and syntax in NLP.** [5 marks] ### Answer **A phoneme** is a sound unit that can distinguish meanings in a language, such as the /p/ and /b/ contrast in “pat” and “bat”. A phoneme is a sound, not necessarily one alphabet letter. **A morpheme** is the smallest meaning-bearing grammatical unit: cats = cat + plural -s; unhappy = un- + happy. **A lexeme** is an abstract vocabulary item grouping its inflected forms: write, writes, wrote and writing belong to WRITE. A **phrase** is a group of words functioning together; a **clause** normally contains a predicate and its associated participants; a **sentence** expresses a complete grammatical unit. | Level | What it analyses | Concrete example | | --- | --- | --- | | Phonetics / phonology | Physical sounds / sound patterns in a language | Distinguish bat from pat | | Morphology / lexical analysis | Word structure and lexical items | un + help + ful → unhelpful | | Syntax | Grammatical arrangement and relationships | The dog chased the cat: subject, verb, object | | Semantics | Meaning of words and composed expressions | The dog is the chaser; the cat is chased | | Discourse | Connections across sentences | “Riya arrived. She smiled.” → She refers to Riya | | Pragmatics | Intended meaning in the situation | “It is cold here” may request closing the window | word forms → grammatical structure → sentence meaning → cross-sentence context → speaker intention ### Three worked interpretations **“The cats are sleeping.”** Morphology identifies cat + s as plural. Syntax connects “cats” to “are sleeping”. Semantics represents animals engaged in sleeping. **“Aman bought a laptop. It was expensive.”** Sentence-level syntax alone does not identify “it”. Discourse connects “it” to the laptop across the sentence boundary. **“Wonderful, another power cut!”** The word wonderful is positive literally, but situational context can indicate sarcasm and frustration. This is a pragmatic interpretation. The analysis levels are conceptual descriptions. Modern systems may learn several jointly rather than executing a compulsory separate module for every level. Morphology = word ke andar; syntax = sentence ka structure; semantics = meaning; discourse = sentences ka link; pragmatics = intention. Morphology analyzes internal word structure: unreadable→un+read+able, cats→cat+plural. Syntax analyzes how words combine: subject–verb–object relations distinguish “dog bites man” from “man bites dog”. Morphological normalization supports vocabulary matching; syntax preserves grammatical relations needed for parsing, extraction and translation. Translation needs morphological number/tense agreement and target-language word order. Sentiment needs syntax to distinguish “not good” from “good” and morphology to relate disappointed/disappointing to their base form without discarding the negative meaning. --- ## TEE · Q4c · Most probable bigram continuations Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r25--q4c/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-r25/q4c Previous examination · complete worked answers **Question. Train on I love programming ; Programming is fun ; I love coding ; Coding is great . Predict next tokens for I love ..., programming ..., coding ... using a bigram model.** [10 marks] ### Answer ### Given I love programming Programming is fun I love coding Coding is great ### To find Most probable next token for each prefix. ### Formula: which one and why $$ P(w\mid h)=\frac{c(h,w)}{\sum_v c(h,v)} $$ Lowercase lexical tokens; include sentence-end as an observed next token. Bigram prediction uses only the last token of the prefix. ### Step 1. Count each adjacent pair separately within sentences | Bigram | Count | | --- | --- | | → coding | 1 | | → i | 2 | | → programming | 1 | | coding → | 1 | | coding → is | 1 | | fun → | 1 | | great → | 1 | | i → love | 2 | | is → fun | 1 | | is → great | 1 | | love → coding | 1 | | love → programming | 1 | | programming → | 1 | | programming → is | 1 | ### Step 2. Continue I love | Candidate | Count / history count | Probability | | --- | --- | --- | | coding | 1/2 | 0.500000 | | programming | 1/2 | 0.500000 | Highest: programming, coding. This is a tie; no tie-break rule is supplied. ### Step 3. Continue programming | Candidate | Count / history count | Probability | | --- | --- | --- | | | 1/2 | 0.500000 | | is | 1/2 | 0.500000 | Highest: , is. This is a tie; no tie-break rule is supplied. ### Step 4. Continue coding | Candidate | Count / history count | Probability | | --- | --- | --- | | | 1/2 | 0.500000 | | is | 1/2 | 0.500000 | Highest: , is. This is a tie; no tie-break rule is supplied. ### Final answer I love → programming or coding programming → or is coding → or is If an examiner explicitly restricts the answer to a lexical word, exclude END and state that restricted candidate set. Do not silently remove an END tie. --- ## TEE · Q5a · Preprocess the NLP passage Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r25--q5a/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-r25/q5a Previous examination · complete worked answers **Question. Perform the requested text preprocessing on: “Natural Language Processing (NLP) is a field of artificial intelligence that focuses on the interaction between computers and humans through natural language. The ultimate objective of NLP is to read, decipher, understand, and make sense of human languages in a valuable way.” Remove the supplied stopwords {is,a,of,that,on,I,the,and,to,in}.** [6 marks] ### Answer | Stage | Result | | --- | --- | | Sentence segmentation | Natural Language Processing (NLP) is a field of artificial intelligence that focuses on the interaction between computers and humans through natural language. The ultimate objective of NLP is to read, decipher, understand, and make sense of human languages in a valuable way. | | Word tokens (punctuation omitted) | Natural, Language, Processing, NLP, is, a, field, of, artificial, intelligence, that, focuses, on, the, interaction, between, computers, and, humans, through, natural, language, The, ultimate, objective, of, NLP, is, to, read, decipher, understand, and, make, sense, of, human, languages, in, a, valuable, way | | Lowercase | natural, language, processing, nlp, is, a, field, of, artificial, intelligence, that, focuses, on, the, interaction, between, computers, and, humans, through, natural, language, the, ultimate, objective, of, nlp, is, to, read, decipher, understand, and, make, sense, of, human, languages, in, a, valuable, way | | Declared stoplist | a, and, i, in, is, of, on, that, the, to | | After exact stop removal | natural, language, processing, nlp, field, artificial, intelligence, focuses, interaction, between, computers, humans, through, natural, language, ultimate, objective, nlp, read, decipher, understand, make, sense, human, languages, valuable, way | | Operation | Why it is useful | | --- | --- | | Sentence segmentation | Defines separate units for parsing and prevents counts from crossing sentence boundaries. | | Lowercasing | Merges case variants when capitalization is not needed; retain original case for NER when useful. | | Stop removal | Can remove frequent low-information words for a bag-of-words retrieval task; it is task-dependent and can damage negation/grammar. | | Tokenization | Defines the units whose counts/features are computed; punctuation and abbreviation conventions must be consistent. | --- ## TEE · Q5b · Stemming and lemmatization tools Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r25--q5b/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-r25/q5b Previous examination · complete worked answers **Question. Differentiate stemming and lemmatization, naming two approaches/tools for each.** [4 marks] ### Answer **Stemming** applies string-based rules, often stripping suffixes, to reduce related forms to a stem. The result need not be a valid dictionary word. **Lemmatization** uses vocabulary and morphological analysis, often with POS information, to return a dictionary lemma. | Criterion | Stemming | Lemmatization | | --- | --- | --- | | Basis | Surface form/suffix rules | Lexicon and morphological/POS analysis | | Output | May be nonword, e.g. studi | Dictionary form, e.g. study | | Context/POS | Usually little or none | Often important | | Irregular forms | May miss went→go | Can map went→go with verb analysis | | Cost | Generally simpler | Usually more linguistic processing | Porter applies a sequence of suffix-removal rules subject to conditions; Snowball provides related language-specific stemming algorithms; Lancaster is often more aggressive; regex stemming uses explicit patterns; Lovins is another established stemming approach. Exact output depends on the algorithm and implementation. ### Three worked reductions 1. **studies, studying:** an English stemmer may yield studi/study depending on the rule sequence; verb lemmatization aims at study. State the exact algorithm if exact outputs are asked. 2. **better:** adjective-aware lemmatization can map it to good; naïve suffix removal cannot infer that irregular relation. 3. **saw:** as a past-tense verb it can become see; as a cutting-tool noun it remains saw. POS and context matter. **Over-stemming** merges forms that should remain distinct; **under-stemming** fails to merge related forms. Normalization can increase recall in search while losing useful distinctions, so it is a task-dependent choice. Stem = rules se cut; lemma = dictionary form, meaning/POS ka dhyan. Examples of stemmers: Porter and Lancaster (Snowball is another). Examples of lemmatization tools: WordNet lemmatizer with POS information and spaCy’s language-specific lemmatizer. Outputs depend on language and supplied POS; better→good requires lexical knowledge, unlike simple suffix cutting. --- ## TEE · Q5c · Hobbs on John and Mary Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r25--q5c/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-r25/q5c Previous examination · complete worked answers **Question. Explain Hobbs’ algorithm and resolve the pronouns in “John gave a book to Mary after he finished reading it.”** [10 marks] ### Answer **Hobbs’ algorithm is a syntax-based procedure that searches constituent parse trees in a fixed order for a compatible noun-phrase antecedent of a pronoun.** The tree determines which candidates are proposed first. Agreement and other relevant constraints determine whether a proposed candidate can be accepted. It is not simply the rule “choose the closest noun”. A nearby noun may have the wrong number, may be syntactically ineligible, or may be visited after a structurally more prominent noun. The algorithm produces a preference under the supplied parse; ambiguous language can still require context. ### The tree vocabulary you need | Term | Meaning | Example | | --- | --- | --- | | NP | Noun phrase: a phrase referring to an entity | the student; Anita; they | | S | A sentence/clause node | the node containing a subject and predicate | | VP | Verb phrase | told Ravi that she should revise | | X | The NP or S node currently being searched | An algorithm label, not a word | | Path p | The branch travelled upward from the pronoun/previous X | Used to distinguish left and right subtrees | | N-bar (N′) | The noun-centred part of an NP in a grammar using that label | The nominal-child test depends on the given tree | | Breadth-first | Visit shallower nodes before deeper ones | Level 1 before level 2 | | Left-to-right | Visit equally deep nodes in their written/tree order | Earlier left branch before right branch | Locate pronoun NP → Climb to NP/S → Propose candidates in tree order → Check compatibility → Accept or continue searching ### The complete search procedure #### Step 1. Start at the pronoun Locate the NP immediately dominating the pronoun. “Dominating” means containing it as a descendant in the tree. #### Step 2. Climb to the first NP or S Move upward to the first NP or S above that starting NP. Call this node X. Keep the upward path p. #### Step 3. Search the left branches initially Below X, traverse branches to the left of p breadth-first and left-to-right. At this initial search, propose an NP only if another NP or S lies between that candidate and X. Test proposed candidates for the required agreement and constraints. #### Step 4. If X is the highest sentence node, inspect earlier sentences Search previous sentence trees from most recent to oldest. Within each, use breadth-first, left-to-right order to propose NPs. Accept the first eligible compatible candidate, or report that no candidate was found in the available context. #### Step 5. Otherwise climb again Move from X to the next NP or S above it. Record this new X and the path followed during the climb. #### Step 6. Apply the higher-NP test If the new X is an NP, propose X when the path did not pass through its nominal N-bar child. Use the given parse labels; do not invent an N-bar branch missing from the question's structure. #### Step 7. Search its left branches Traverse branches below the new X to the left of the path, breadth-first and left-to-right, proposing NPs. The special intervening-NP/S restriction belongs to the initial search in Step 3. #### Step 8. If X is S, search the permitted right branches Search branches to the right of the path breadth-first and left-to-right. Do not descend below an encountered NP or S. Propose eligible NP candidates there. #### Step 9. Repeat or stop Return to the highest-S check and continue the climb/search procedure until a compatible candidate is accepted or the available trees are exhausted. Record the candidate order rather than only writing an unexplained name. ### Why the first left search has an extra condition In a simple clause such as “John saw him”, the subject NP John is directly under the clause S. There is no extra NP or S between John and that S. The initial-search restriction therefore prevents casually selecting John as the ordinary local antecedent of him. Compare the reflexive “John saw himself”, which is a different grammatical case; do not treat every pronoun type as interchangeable. ### Worked tree 1 · move to the previous sentence **Question. Apply Hobbs’ search to “The student arrived. She smiled.” Assume the student is compatible with she.** #### Given Previous tree: [S [NP The student] [VP arrived]]. Current tree: [S [NP She] [VP smiled]]. #### To find The selected antecedent and the search path. #### Method and why it applies Use the supplied constituent trees. The current sentence has no earlier candidate, so the highest-S rule activates previous-sentence search. ### Solution #### Step 1. Draw both sentence structures ``` Previous sentence Current sentence S S ├── NP: The student ├── NP: She ← start └── VP: arrived └── VP: smiled ``` #### Step 2. Locate and climb Start at NP She. Its first higher NP/S is the current S. Set X to that S. #### Step 3. Search left of the path She is the leftmost subject. No NP lies to its left in the current sentence. #### Step 4. Use the highest-S rule This X is the highest S. Inspect the immediately preceding sentence tree. #### Step 5. Propose its first NP Breadth-first, left-to-right finds NP The student. #### Step 6. Check compatibility and accept The question stipulates that the student agrees with she. Accept this candidate. **Answer.** She refers to the student under the supplied assumptions. ### Worked tree 2 · reject an incompatible subject **Question. Resolve they in “Riya spoke to the students. They laughed.” Use a previous-sentence search and interpret they as plural.** #### Given Previous tree has subject NP Riya, then VP spoke [PP to [NP the students]]. Current subject is NP They. #### To find The candidate order and accepted antecedent. #### Method and why it applies Breadth-first search favours a shallow subject, but agreement must still be applied; a first candidate is not automatically a valid answer. ### Solution #### Step 1. Draw the useful part of the previous tree ``` S ├── NP: Riya ← candidate 1 └── VP ├── V: spoke └── PP ├── P: to └── NP: the students ← candidate 2 ``` #### Step 2. Search the current sentence From NP They climb to its S; there is no earlier NP in “They laughed”. This is the highest S, so move to the previous tree. #### Step 3. Visit Riya first Riya is the shallower leftmost NP. Under the stated plural interpretation, this singular candidate fails number agreement. #### Step 4. Visit the students Continue breadth-first. The next relevant NP is the students. #### Step 5. Apply number agreement Students is plural and is an eligible candidate. It matches plural they. #### Step 6. State the resolution Accept the students. The method did not simply accept the first noun it saw. **Answer.** They → the students. Candidate order: Riya rejected; the students accepted. ### Worked tree 3 · climb out of an embedded clause **Question. Resolve she in “Anita told Ravi that she should revise.” Assume Anita is compatible with she and Ravi is not. Use the displayed parse.** #### Given The pronoun is the subject of an embedded S inside the main VP. #### To find The preferred antecedent found by the climb and left-branch search. #### Method and why it applies The embedded clause has no left candidate. Climbing to the main S exposes its subject Anita as the first suitable higher-level candidate. ### Solution #### Step 1. Show the clause structure ``` S (main) ├── NP: Anita ← accepted candidate └── VP ├── V: told ├── NP: Ravi └── SBAR: that └── S (embedded) ← first X ├── NP: she ← start └── VP: should revise ``` #### Step 2. Start inside the embedded clause From NP she climb to the embedded S. There is no candidate to the left of she within this embedded S. #### Step 3. Climb to the next NP/S The embedded S is not the highest S. Move upward through SBAR and VP to the main S. These intermediate nodes are not NP/S search nodes. #### Step 4. Search left of the upward path At the main S, the subject NP Anita is a shallow left candidate. The object Ravi is deeper within the VP. Breadth-first order proposes Anita first. #### Step 5. Check the stated compatibility Anita is compatible with she by the question's assumption. Accept it; no need to choose a later incompatible candidate. #### Step 6. Give the result and its limit This is the algorithm's result under the displayed parse and agreement assumptions. It does not prove the intention of every speaker using an otherwise ambiguous sentence. **Answer.** She → Anita under the stated tree and compatibility information. ### Advantages, limitations and a complete exam answer | Advantages | Limitations | | --- | --- | | Explicit candidate order makes the decision traceable. | Wrong parse structure can produce the wrong search order. | | Requires no labelled coreference-training corpus for its basic rules. | World knowledge and discourse intention may override syntactic preference. | | Uses structure rather than only surface distance. | Some ambiguous, long-distance or unusual constructions need additional treatment. | For a longer answer, write the definition, inputs, tree notation, ordered algorithm, one labelled tree with a candidate table, final antecedent, and limitations. In a worked question show separately: candidate → why proposed → agreement result → accept/reject. Do not equate the selected candidate with a logically guaranteed interpretation when the sentence is ambiguous. **Yaad rakho:** Tree order candidate deta hai; agreement usko allow/reject karta hai. Nearest noun automatically answer nahi. ### Apply to this sentence 1. Build the main clause with subject NP John; VP gave contains object NP a book and PP to Mary; the after-clause has subject he and object it under reading. 2. For he, ascend from its NP to the governing S/NP and traverse candidate NPs to the left using the Hobbs breadth-first search. Test number/person/gender and binding compatibility. 3. John is an earlier singular human candidate. Under the intended conventional name-gender reading, he→John. Mary is not chosen merely because it is nearer. Names alone do not establish real-world gender; this is the ordinary exercise reading. 4. For it, the inanimate singular a book is compatible with reading; John and Mary are human mentions. Resolve it→a book. 5. Final interpretation: John gave the book after John finished reading the book. The trees/candidate order guide the search; semantic compatibility is also needed. ### The four requested properties | Property | What it means | Apply here | | --- | --- | --- | | Binding | Syntactic restrictions on reference; ordinary personal pronouns and reflexives have different local-antecedent requirements | Do not mechanically link every nearby NP; he in an after/because clause can refer outside that local clause. | | Gender agreement | Candidate must match the pronoun under available linguistic/contextual knowledge | John/he is the exercise’s conventional reading; professor/student do not provide gender by themselves. | | Distance | Token, syntactic or sentence separation between pronoun and candidate | Closer candidates may be preferred, but cannot override incompatibility or Hobbs search order. | | Recency | Recently mentioned discourse entities are often more salient | Mary/student is later than John/professor, but recency is a preference, not a guaranteed antecedent rule. | Reference resolution preserves who performed each action; without it an extraction or QA system could wrongly attribute the reading to Mary. John is the first suitable main-clause subject under the declared parse; the inanimate book fails human he compatibility. --- ## TEE · Q6a · Corpora and acquisition Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r25--q6a/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-r25/q6a Previous examination · complete worked answers **Question. Explain types, characteristics and acquisition of text corpora.** [10 marks] ### Answer **A corpus is a collection of language data assembled for study or computational use.** Its plural is corpora. A corpus can contain books, news, reviews or conversations. It need not be labelled. **Annotation** adds linguistic information such as POS tags, entities, parse structures or sense labels. | Structure | Meaning | Use | | --- | --- | --- | | Plain | Collection without category labels | Word counts, language modelling | | Categorized | Documents grouped by genre/topic | Classification | | Overlapping categories | One document has several categories | Multi-topic news | | Temporal | Documents indexed by time | Language change | | Annotated | Linguistic labels attached | Tagging, parsing, WSD | | Parallel | Aligned texts in different languages | Translation | A **tagset** is the allowed label inventory. A **treebank** is a corpus annotated with syntax trees. **Metadata** records source, date, genre, language, annotation method and limitations. A common benchmark supports comparison under shared data and evaluation conditions. **Raw example:** “Riya works in Pune.” **POS annotation:** Riya/PROPN works/VERB in/ADP Pune/PROPN. **NER annotation:** Riya/PERSON, Pune/LOCATION. Each annotation answers a different task. A good corpus is relevant to its domain, sufficiently representative, consistently annotated and appropriately split. More data alone do not repair duplicated examples, incorrect labels or a mismatch between training and real use. Corpus=text collection. Annotation=text ke saath labels. Metadata=collection ki information. | Resource | Contents | Typical use | | --- | --- | --- | | Gutenberg selection | Literary/book texts | Long-form language and vocabulary | | Web text | Informal domain-specific text | Study usage beyond edited books | | Chat corpus | Conversational posts | Dialogue, informal forms | | Brown | English texts across genres | Genre comparison | | Reuters | News with topic labels | Text classification; overlapping topics | | Inaugural | Speeches indexed by time | Vocabulary change across years | | Wordlist | Words/names/stopwords | Lexical filtering | | Pronouncing dictionary | Words and phonetic forms | Speech, stress, rhyme | **Gutenberg versus web/chat:** book language does not represent every spelling or conversational pattern in messages. **Brown versus Reuters:** Brown emphasizes genres; Reuters news can have overlapping topic labels. **Inaugural addresses:** dates provide a time dimension. Corpus sizes depend on version, subset and tokenization. Explain their purpose and structure rather than assuming a historical count is a permanent property. Gutenberg=books; web/chat=informal; Brown=genres; Reuters=news topics; Inaugural=time. **Data acquisition** obtains language examples relevant to the task from existing data, public resources or appropriate new collection. Product instrumentation can gather useful feedback. Supervised learning additionally needs an annotation process when labels are absent. **Data augmentation** creates training variants intended to preserve the correct label. It is not simply changing words arbitrarily. | Method | Procedure | Caution | | --- | --- | --- | | Synonym replacement | Use suitable synonyms for selected words | Choose the right sense | | Back translation | Translate to another language and back | May change meanings/entities | | TF–IDF-informed replacement | Protect informative words while varying others | Do not remove label-bearing clues | | Bigram flipping | Swap adjacent words | Can break grammar or meaning | | Entity replacement | Replace by same-type entity | Must preserve the task label | | Noise injection | Simulate typos/keyboard errors | Match expected noise | | Active learning | Select informative cases for human labels | This acquires labels; it is not synthetic text | ### Three label decisions “The film is good”→“The film is excellent” may preserve a positive sentiment label. “The film is not good”→“The film is good” reverses the label and is not valid label-preserving augmentation. “Flights to Pune”→“Flights to Mumbai” may preserve a travel-topic label but changes the destination label. Split train/test data before creating related augmented variants. Otherwise near-duplicates can leak between sets and produce misleading evaluation. Keep a representative evaluation set of genuine examples. Sentence badal sakta hai; required correct label nahi. Label badla toh augmentation invalid. ### Corpus structure diagram ``` Corpus ├─ Documents (plain collection) │ └─ Sentences → tokens ├─ Categories → document IDs (one or overlapping labels) ├─ Dates → document IDs (temporal collection) ├─ Annotations → token spans / tags / parse trees └─ Parallel pairs → aligned source and target sentences ``` An annotated corpus attaches linguistic labels to the raw text: e.g. Rita/PROPN runs/VERB. A wordlist is a lexical inventory without full discourse context; a Gutenberg corpus contains books; web/chat corpora contain informal online language. Their structures determine which evidence can be queried. --- ## TEE · Q6b · HMM components and estimates Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r25--q6b/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-r25/q6b Previous examination · complete worked answers **Question. Explain HMM components, the number of possible tag sequences and limitations. Construct transition and emission tables for the supplied Alice/Bob/They/The-cat tagged corpus.** [10 marks] ### Answer **A Hidden Markov Model (HMM)** is a probabilistic sequence model in which the underlying state sequence is unobserved, while each state emits an observed item. For POS tagging, words are observed and their grammatical tags are hidden. The task is to infer a suitable tag sequence from the words. Hidden: Noun → Hidden: Verb → Hidden: Noun Observed: birds → Observed: eat → Observed: seeds Horizontal state-to-state links are **transitions**. The relation from each hidden state to its observed word is an **emission**. A transition P(V|N) asks how often a verb tag follows a noun tag. An emission P(eat|V) asks how often the observed word eat appears among verb-tagged tokens. | Component | Notation | Meaning | | --- | --- | --- | | State set | S={N,V,…} | Possible hidden tags. | | Observation vocabulary | O={birds,eat,seeds,…} | Visible word tokens. | | Initial distribution | πⱼ=P(t₁=j) | Probability of starting in tag j. | | Transition matrix | aᵢⱼ=P(tₖ=j|tₖ₋₁=i) | Probability of next tag j after i. | | Emission matrix | bⱼ(w)=P(w|tag j) | Probability that state j emits word w. | Two basic assumptions are used: the current hidden state depends only on the previous hidden state, and the current observation depends only on the current hidden state. A standard first-order HMM therefore does not explicitly condition an emitted word on the whole sentence. $$ P(t_{1:T},w_{1:T})=\pi_{t_1}b_{t_1}(w_1)\prod_{k=2}^{T}a_{t_{k-1},t_k}b_{t_k}(w_k) $$ Read the formula as **start × first emission × transition × emission × transition × emission…**. There are T emission factors and T−1 transitions, plus one initial factor. If the question supplies an explicit end-state probability, multiply that too. Do not invent an end factor when the model has none. **Three different questions.** (1) Scoring a specified tag path uses the joint product above. (2) Finding the probability of the observed words sums scores over all possible tag paths. (3) Finding the best tag path maximizes the path score, typically using Viterbi. A joint path score is not a normalized posterior P(tags|words). **Yaad rakho:** Transition = tag se agla tag. Emission = tag se visible word. If each of T observed words can receive any of N tags, there are N choices at each position. Thus the total number of complete tag sequences is N multiplied by itself T times. This counts possible assignments before grammar rules or zero probabilities eliminate any. $$ \text{Number of tag sequences}=N^T $$ **Question. A four-word sentence is tagged using three possible POS tags per word. How many complete tag sequences are possible?** #### Given N=3 possible tags per position; T=4 word positions. #### To find The number of tag sequences. #### Formula and why it applies $$ M=N^T $$ Every independent position contributes a factor of three choices. #### Solution ##### Step 1. Write one factor per position $$ M=3\times3\times3\times3 $$ ##### Step 2. Multiply $$ 3\times3=9,\quad9\times3=27,\quad27\times3=81 $$ **Final answer.** 81 possible tag sequences. **Question. A five-word sentence has two candidate tags per word. Count its tag sequences.** #### Given N=2; T=5. #### To find M. #### Formula and why it applies $$ M=N^T $$ The length is the exponent, not a multiplier. #### Solution ##### Step 1. Substitute $$ M=2^5 $$ ##### Step 2. Evaluate $$ 2\times2\times2\times2\times2=32 $$ **Final answer.** 32 possible sequences. **Question. Using 45 possible tags, compare the number of tag sequences for sentences of length 2 and 3.** #### Given N=45. First T=2, then T=3. #### To find Both counts and the growth factor. #### Formula and why it applies $$ M_2=45^2,\quad M_3=45^3 $$ Adding one position multiplies the possibilities by the number of tag choices. #### Solution ##### Step 1. Two words $$ 45^2=2025 $$ ##### Step 2. Three words $$ 45^3=2025\times45=91125 $$ ##### Step 3. Compare $$ 91125/2025=45 $$ **Final answer.** 2 words: 2025 paths. 3 words: 91125 paths. One additional word multiplies the count by 45. The slides introduce Viterbi to avoid enumerating every path. It reuses the best prefix score for each ending state. For a dense first-order HMM, it needs O(TN²) recurrence work instead of enumerating NT complete paths. **Yaad rakho:** Choices per word ki power = sentence length. ### Exact corpus calculation ### Given | Sentence | Given tags | | --- | --- | | Alice will eat cookies | N M V N | | Bob can drink tea | N M V N | | They will watch movies | Pronoun M V N | | The cat is sleeping | Det N V V | ### To find Emission and transition tables ### Formula: which one and why $$ P(w\mid t)=\frac{c(t,w)}{c(t)} $$ $$ P(t_j\mid t_i)=\frac{c(t_i,t_j)}{\sum_kc(t_i,t_k)} $$ The second formula is the declared no-END convention. Reset at sentence boundaries; never create a transition from the last word of one training sentence to the first of the next. ### Step 1. Count emissions | Tag | Total occurrences | Alice | will | eat | cookies | Bob | can | drink | tea | They | watch | movies | The | cat | is | sleeping | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | N | 6 | 1 | 0 | 0 | 1 | 1 | 0 | 0 | 1 | 0 | 0 | 1 | 0 | 1 | 0 | 0 | | M | 3 | 0 | 2 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | | V | 5 | 0 | 0 | 1 | 0 | 0 | 0 | 1 | 0 | 0 | 1 | 0 | 0 | 0 | 1 | 1 | | Pronoun | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | | Det | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | ### Step 2. Divide each emission by its tag count | Tag | Alice | will | eat | cookies | Bob | can | drink | tea | They | watch | movies | The | cat | is | sleeping | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | N | 1/6 | 0 | 0 | 1/6 | 1/6 | 0 | 0 | 1/6 | 0 | 0 | 1/6 | 0 | 1/6 | 0 | 0 | | M | 0 | 2/3 | 0 | 0 | 0 | 1/3 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | | V | 0 | 0 | 1/5 | 0 | 0 | 0 | 1/5 | 0 | 0 | 1/5 | 0 | 0 | 0 | 1/5 | 1/5 | | Pronoun | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | | Det | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | ### Step 3. Count transitions inside each sentence | From | Outgoing transitions | Sentence-final count | N | M | V | Pronoun | Det | | --- | --- | --- | --- | --- | --- | --- | --- | | N | 3 | 3 | 0 | 2 | 1 | 0 | 0 | | M | 3 | 0 | 0 | 0 | 3 | 0 | 0 | | V | 4 | 1 | 3 | 0 | 1 | 0 | 0 | | Pronoun | 1 | 0 | 0 | 1 | 0 | 0 | 0 | | Det | 1 | 0 | 1 | 0 | 0 | 0 | 0 | ### Step 4. Convert to a transition table | From / to | N | M | V | Pronoun | Det | | --- | --- | --- | --- | --- | --- | | N | 0 | 2/3 | 1/3 | 0 | 0 | | M | 0 | 0 | 1 | 0 | 0 | | V | 3/4 | 0 | 1/4 | 0 | 0 | | Pronoun | 0 | 1 | 0 | 0 | 0 | | Det | 1 | 0 | 0 | 0 | 0 | Here sentence-final tags are excluded from the no-END outgoing denominator. If END is explicitly modeled, divide all tag→tag and tag→END counts by the total tag count instead; the row then includes END. Do not mix denominators. ### Final answer The two displayed probability tables are the estimated HMM. Zero means unobserved without smoothing, not logically impossible. For T tokens and K possible tags per token, exhaustive assignments number K^T before constraints; the paper does not supply a single test length for a specific numerical total. HMM assumptions limit long-range context, first-order transitions and emission independence; sparsity and unknown words require smoothing or other handling. The supplied tag is for is and sleeping is V, so both count as V even though finer linguistic tagsets distinguish auxiliaries/participles. --- ## TEE · Q7a · Word classes, chunking and chinking Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r25--q7a/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-r25/q7a Previous examination · complete worked answers **Question. Explain open/closed word classes, chunking and chinking with examples.** [5 marks] ### Answer **Part-of-speech (POS) tagging** assigns a grammatical category to every word token in context, such as noun, verb, adjective or determiner. It is a sequence-labeling task: the input is a sequence of words, and the output is a tag for each word. A tag describes the word's role in this sentence, not its meaning in every possible sentence. | Tag | Role | Example in context | | --- | --- | --- | | NN / NNS | Singular / plural common noun | a book / two books | | NNP | Proper noun | Mary, Mumbai | | VB / VBD / VBZ | Base / past / third-person singular verb | read / read / reads | | JJ | Adjective | the red book | | RB | Adverb | runs quickly | | DT | Determiner | the, a, this | | IN | Preposition or subordinating conjunction | on the table | | PRP | Personal pronoun | she, they | | MD | Modal auxiliary | can, will, should | **Open classes** readily accept new content words: nouns, verbs, adjectives and adverbs. Examples include chatbot and livestream. **Closed classes** contain comparatively stable grammatical/function words: determiners, pronouns, prepositions and conjunctions. These are tendencies, not a claim that language never changes. Input: Book that flight → Context: an imperative request → Output: Book/VB that/DT flight/NN Compare “hand me that **book**”: book is a noun here. Similarly, “I **can** swim” has can/MD, while “the **can** is empty” has can/NN. The spelling stays the same; neighboring words and sentence structure change the correct tag. This is **POS ambiguity**. | Sentence | Tagged interpretation | Reason | | --- | --- | --- | | They watch birds. | They/PRP watch/VBP birds/NNS | watch is the action. | | The watch stopped. | The/DT watch/NN stopped/VBD | watch is the object that stopped. | | She has a light bag. | She/PRP has/VBZ a/DT light/JJ bag/NN | light describes bag. | | Light the lamp. | Light/VB the/DT lamp/NN | Light is an imperative verb. | POS tags support parsing, named entity recognition, question answering, sentiment analysis and word-sense disambiguation. They do not by themselves identify every entity or fully resolve meaning. **Yaad rakho:** Word ka naam same ho sakta hai; sentence mein uska kaam tag decide karta hai. **Chunking**, also called shallow parsing, groups adjacent tagged words into meaningful non-overlapping units, commonly noun phrases. It usually identifies local phrase boundaries without producing the complete hierarchical structure of a sentence. **Chinking** removes specified tokens from an existing candidate chunk, splitting or trimming it. The/DT small/JJ cat/NN sat/VBD on/IN the/DT mat/NN → NP: [The small cat] → Outside: sat on → NP: [the mat] A POS tagger labels individual words. A chunker groups words into phrases. A full parser shows how phrases relate recursively to form the entire sentence. These are related tasks with different outputs. | Operation | Input | Output | | --- | --- | --- | | POS tagging | The small cat | The/DT small/JJ cat/NN | | NP chunking | Tagged tokens | [NP The small cat] | | Chinking | A broad candidate chunk | Remove verbs/prepositions, keep noun-phrase segments | | Full parsing | Sentence + grammar | A nested tree with sentence, noun phrase and verb phrase nodes | **Rule-based chunking** uses tag patterns. Statistical or learned chunkers infer boundaries from labeled examples. A useful NP rule is an optional determiner, followed by zero or more adjectives, followed by a noun. In NLTK-style notation: `NP: {
?*}`. Here ? means zero or one occurrence; \* means zero or more; + means one or more when used. The braces mean “form a chunk around the matched sequence”. **Question. Apply the NP rule optional DT, zero or more JJ, then one NN to “The/DT quick/JJ brown/JJ fox/NN jumps/VBZ over/IN the/DT lazy/JJ dog/NN”.** #### Given Tagged input is supplied. Rule: DT? JJ\* NN. #### To find All noun-phrase chunks. #### Formula and why it applies `NP → DT? JJ* NN` The rule groups a noun with its preceding determiner and adjectives; a verb or preposition does not match this NP pattern. #### Solution ##### Step 1. Start at The The matches optional DT; quick and brown both match JJ\*; fox matches NN. ##### Step 2. Form the first NP **[NP The quick brown fox]** ##### Step 3. Skip nonmatching tokens jumps/VBZ and over/IN do not satisfy the NP pattern. ##### Step 4. Match the second NP the is DT, lazy is JJ, dog is NN: **[NP the lazy dog]**. **Final answer.** [NP The quick brown fox] jumps over [NP the lazy dog]. **Question. Using NP → DT? JJ\* NN, identify the chunks in “bright/JJ stars/NN shine/VBP and/CC a/DT moon/NN glows/VBZ”. Use the supplied tags exactly.** #### Given Input tags label stars as NN for this exercise. Rule requires optional DT, zero or more JJ, then NN. #### To find The matched NP spans. #### Formula and why it applies `NP → DT? JJ* NN` The optional determiner may be absent and the adjective repetition may be empty. #### Solution ##### Step 1. Match bright stars No determiner is present, which DT? permits. bright matches JJ; stars matches NN. ##### Step 2. Inspect shine and and VBP and CC fall outside the NP rule. ##### Step 3. Match a moon a matches DT; no adjective is present; moon matches NN. **Final answer.** Chunks are [NP bright stars] and [NP a moon]. With ordinary Penn tagging, stars is usually NNS; broaden the noun pattern to include NNS when that tag is used. **Question. Begin with one candidate chunk containing “The/DT cat/NN sat/VBD on/IN the/DT mat/NN”. Chink out verbs and prepositions. Show the remaining chunks.** #### Given Initial candidate: [The cat sat on the mat]. Remove tags VBD and IN. #### To find The chunk segments after exclusion. #### Formula and why it applies Chinking removes matched tokens from the existing chunk. Removing the middle verb/preposition region divides the original candidate into two remaining contiguous spans. #### Solution ##### Step 1. Locate the tokens to remove sat is VBD and on is IN. ##### Step 2. Remove those tokens from chunk membership The remaining token groups are “The cat” and “the mat”. sat and on still exist in the sentence; they are simply outside these chunks. ##### Step 3. Write the final chunked sentence **[The cat]** sat on **[the mat]**. **Final answer.** Two chunks remain: The cat; the mat. **Yaad rakho:** Chunking = andar group karo; chinking = unwanted tokens ko group se bahar karo. --- ## TEE · Q7b · Named entity recognition Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r25--q7b/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-r25/q7b Previous examination · complete worked answers **Question. Explain named-entity recognition and its approaches with examples.** [5 marks] ### Answer **Named Entity Recognition (NER)** identifies text spans referring to entities and assigns their types, such as PERSON, ORGANIZATION and LOCATION. Many practical schemas also include DATE, TIME, MONEY or domain-specific types. The exact allowed labels depend on the application's annotation scheme. NER has two parts: **boundary detection** finds where the entity starts and ends; **classification** assigns the appropriate type. A multiword name should normally be one entity span: “New York” is a location span, not two unrelated location decisions. Text: Mary joined Google in New York on Monday. → Mary → PERSON → Google → ORGANIZATION → New York → LOCATION → Monday → DATE Not every noun is a named entity. In “a developer works at Google”, developer is an ordinary job description while Google is a named organization. A specialized HR system may define a JOB\_TITLE category, but it must state that schema explicitly. NER identifies mentions; entity linking is the separate task of connecting a mention to a unique database identity. | Task | Question it answers | Example | | --- | --- | --- | | POS | What grammatical role does this token have? | Google/NNP | | Chunking | Which adjacent words form a phrase? | [NP a skilled developer] | | NER | Which span names an entity, and of what type? | [Google] ORGANIZATION | | Entity linking | Which real entity does this mention denote? | Connect Paris to the intended city or person record | **Applications.** NER structures search results and documents, extracts organizations/people from news, routes support tickets by product or organization, supports recommendation, and extracts specialized entities from healthcare or legal text. Context is essential: “Apple released a laptop” names an organization, while “she ate an apple” describes fruit. **Yaad rakho:** NER mein do kaam: naam kahan se kahan tak, aur kis category ka. | Approach | How it works | Example | Main limitation | | --- | --- | --- | --- | | Dictionary / gazetteer | Match text against lists of known names | A city list includes Mumbai and Pune | New names, spelling variants and ambiguous names are missed | | Rule-based | Use word shapes, regular expressions and context | A title such as Dr. followed by a name-like span suggests PERSON | Manual maintenance; exceptions and domain/language dependence | | Machine learning | Learn a labeling rule from annotated examples using features | Capitalization, suffixes, POS and surrounding words help a classifier/CRF | Requires labeled data and useful features | | Deep learning | Learn contextual word/character representations and predict labels | A contextual sequence encoder labels multiword entity spans | Training/domain dependence and resource demands | A **pattern-based rule** examines the form of text, such as an email pattern or a date format. A **context-based rule** examines neighboring words: “Dr.” can support a following PERSON span, while “headquartered in” can support a following LOCATION span. Capitalization alone is insufficient because sentence-initial common nouns are also capitalized. A dictionary may find “May” as a person's name, but “in May” may refer to a month. A learned model uses surrounding evidence to disambiguate. Hybrid systems can combine dictionaries, rules and learned models; these categories are practical tools rather than mutually exclusive restrictions. **Preprocessing for NER.** Tokenize while preserving token offsets and useful capitalization. Blindly removing capital letters or stopwords before recognition can damage names such as “Bank of India”. NER output should preserve the original span so the reader can locate it in the input. **Exam-ready 3-mark answer.** NER locates named-entity spans and classifies them into categories such as person, organization and location. A dictionary-based approach matches known name lists; a rule-based approach uses patterns or context; machine-learning and deep-learning approaches learn entity decisions from labeled data and representations. For example, in “Mary works at Google in Mumbai”, Mary is PERSON, Google is ORGANIZATION and Mumbai is LOCATION. --- ## TEE · Q7c · Co-occurrence with inconsistent requested comparisons Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r25--q7c/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-r25/q7c Previous examination · complete worked answers **Question. Construct a whole-sentence co-occurrence matrix for: The bird flew across the sky. The cat watched the bird. The flowers grew in the garden. The bird rested on the tree. The cat slept near the tree. Targets={bird,cat,garden,flowers}; contexts={sky,tree,garden,slept}. The printed follow-up asks dot products boy–dog and boy–sun and a plot on kicked/field dimensions.** [10 marks] ### Answer ### Given Use the exact target/context lists above. ### To find Construct the supported matrix and identify which later requests are undefined. ### Formula: which one and why Count ordered target-token/context-token pairs in the same sentence, excluding the target token itself. A single garden occurrence cannot co-occur with itself. If a classroom convention uses sentence-level presence including diagonal, only garden→garden changes from0to1; state that choice. ### Step 1. Enumerate contributions and sum | Sentence | Target occurrences | Context occurrences | | --- | --- | --- | | 1 | bird | sky | | 2 | cat, bird | | | 3 | flowers, garden | garden | | 4 | bird | tree | | 5 | cat | slept, tree | | Target / context | sky | tree | garden | slept | | --- | --- | --- | --- | --- | | bird | 1 | 1 | 0 | 0 | | cat | 0 | 1 | 0 | 1 | | garden | 0 | 0 | 0 | 0 | | flowers | 0 | 0 | 1 | 0 | ### Step 2. Check requested names before multiplying boy,dog,sun are not target rows; kicked,field are not context columns. Therefore the requested dot products and plot cannot be calculated from the supplied matrix. Zero vectors for invented words would not repair the missing givens. ### Step 3. Clearly labelled corrected illustration If the follow-up were changed to bird–cat and bird–flowers, using the displayed rows: bird·cat=1×0+1×1+0×0+0×1=1; bird·flowers=1×0+1×0+0×1+0×0=0. A sky/tree plot would use bird(1,1),cat(0,1),garden(0,0),flowers(0,0). These substitutions are teaching examples, not printed instructions. ### Final answer **The original matrix is solved; its requested boy/dog/sun comparisons and kicked/field plot are not well-defined.** Ask for corrected target/context names rather than inventing a numerical answer. **Source / convention note.** Source defect: follow-up names belong to another corpus. Exact supported matrix is complete; unsupported parts are explicitly unresolved from the original data. --- ## TEE · Q1a · TF–IDF in retrieval Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f24--q1a/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-f24/q1a Previous examination · complete worked answers **Question. Write TF–IDF expressions and explain their role in information retrieval.** [individual allocation not printed; Q1 totals 20] ### Answer **TF–IDF weights a term highly when it is frequent in a particular document but uncommon across the corpus.** The supplied Unit 2 slides use normalized term frequency and base-10 inverse document frequency: $$ \operatorname{TF}(t,d)=\frac{C(t,d)}{N_d} $$ $$ \operatorname{IDF}(t)=\log_{10}\frac{D}{\operatorname{df}(t)} $$ $$ \operatorname{TFIDF}(t,d)=\operatorname{TF}(t,d)\operatorname{IDF}(t). $$ $$ N_d $$ is the document's token count after the specified preprocessing; $$ D $$ is number of documents; $$ \operatorname{df}(t) $$ is the number of documents containing the term at least once. A word repeated ten times in one document contributes one to document frequency. **Why the log?** It compresses the ratio's scale. If a term occurs in all documents, $$ D/df=1 $$ , so IDF=0 under this unsmoothed convention. Other libraries use smoothed IDF, different TF or normalization. Use the question's formula; do not silently substitute a library default. **Question. A corpus contains 10 documents. The word model appears in 2 documents and occurs 3 times in a 12-token document. Compute TF, IDF and TF–IDF using the slide convention.** #### Given D=10, df(model)=2, C(model,d)=3, Nd=12; log base 10. #### To find The TF–IDF weight. #### Formula and why it applies $$ TF=C/N_d $$ , $$ IDF=\log_{10}(D/df) $$ , then multiply. #### Step 1. TF $$ 3/12=0.25 $$ . #### Step 2. Document ratio $$ 10/2=5 $$ . #### Step 3. IDF $$ \log_{10}5\approx0.698970 $$ . #### Step 4. Weight $$ 0.25\times0.698970\approx0.174743 $$ . **Answer.** TF=0.25, IDF≈0.698970, TF–IDF≈0.174743. **Question. A term occurs 4 times in a 20-token document and is present in all 8 corpus documents. Calculate its TF–IDF.** #### Given C=4, Nd=20, D=8, df=8. #### To find Its weight under unsmoothed base-10 IDF. #### Formula and why it applies $$ TF=C/N_d $$ , $$ IDF=\log_{10}(D/df) $$ . #### Step 1. TF $$ 4/20=0.2 $$ . #### Step 2. IDF $$ \log_{10}(8/8)=\log_{10}1=0 $$ . #### Step 3. Multiply $$ 0.2\times0=0 $$ . **Answer.** Weight 0: this term does not distinguish documents under this convention. **Question. Find TF–IDF vectors for D1=“cat cat dog”, D2=“dog fish”, D3=“cat fish fish”, using [cat,dog,fish], normalized TF and log10 IDF.** #### Given No further preprocessing. Each term occurs in exactly two of the three documents. #### To find All three vectors. #### Formula and why it applies First count df by documents, then compute each document's TF, then multiply by its term's IDF. #### Step 1. Common IDF $$ D=3 $$ , $$ df(cat)=df(dog)=df(fish)=2 $$ . Each IDF $$ =\log_{10}(3/2)\approx0.176091 $$ . #### Step 2. D1 term frequencies Length 3 → $$ [2/3,1/3,0] $$ . #### Step 3. D1 weights $$ [(2/3)0.176091,(1/3)0.176091,0]\approx[0.117394,0.058697,0] $$ . #### Step 4. D2 weights Length 2; TF $$ [0,1/2,1/2] $$ . Vector $$ \approx[0,0.088046,0.088046] $$ . #### Step 5. D3 weights Length 3; TF $$ [1/3,0,2/3] $$ . Vector $$ \approx[0.058697,0,0.117394] $$ . **Answer.** D1≈[0.117394,0.058697,0]; D2≈[0,0.088046,0.088046]; D3≈[0.058697,0,0.117394]. TF: iss document mein kitna. IDF: poore corpus mein kitna rare. Dono multiply. **Information retrieval (IR)** locates and ranks items relevant to an information need, usually expressed as a query. The collection may contain documents, product descriptions or scientific papers. Retrieval estimates relevance; it does not itself establish that every retrieved claim is true. Documents → cleaning → Build index → Query → same representation → Match & rank → Return relevant results ### The inverted index An inverted index maps a term to the documents containing it, often with positions or counts. For D1 “cats drink milk” and D2 “dogs drink water”, the posting list for drink is {D1,D2}, while milk maps only to D1. This avoids rescanning every document for every query. | Term | Posting list | | --- | --- | | cats | D1 | | drink | D1, D2 | | milk | D1 | | dogs | D2 | | water | D2 | ### Matching and ranking Boolean retrieval combines term membership using AND, OR and NOT. Vector-space retrieval represents documents and queries as weighted vectors, for example with TF-IDF, and compares them with cosine similarity. Learned retrieval can use dense embeddings for meaning-based matching. The representation used at query time must be compatible with the document index. ### Search engines and recommendation A search engine receives an explicit query and ranks matching indexed pages. A recommender may derive an implicit query from a user’s liked items or profile and retrieve similar candidates. Both require candidate retrieval and ranking, but the source of the information need differs. ### Evaluate the results For retrieval, TP is the number of retrieved relevant documents, FP retrieved irrelevant documents, and FN relevant documents not retrieved. Precision and recall use the same formulas as classification. Ranking measures also consider the positions of relevant items; a relevant result at rank 1 is more useful than one far below. ### Retrieval evaluation A search system retrieves 8 documents; 6 are relevant. There are 10 relevant documents in the collection. Find precision, recall and F1. **Given** Retrieved=8; relevant retrieved=6; all relevant=10. **To find** P, R and F1. ### Formula and why it applies $$ P=\frac{\text{relevant retrieved}}{\text{retrieved}},\quad R=\frac{\text{relevant retrieved}}{\text{all relevant}} $$ $$ F_1=\frac{2PR}{P+R} $$ The denominator is 8 for precision and 10 for recall because they ask different questions. ### Solution #### Step 1. Identify the confusion counts TP=6; FP=8−6=2; FN=10−6=4. #### Step 2. Precision $$ P=\frac68=0.75 $$ #### Step 3. Recall $$ R=\frac6{10}=0.6 $$ #### Step 4. F1 $$ F_1=\frac{2(0.75)(0.6)}{0.75+0.6}=\frac{0.9}{1.35}=\frac23\approx0.666667 $$ Precision=0.75, recall=0.60, F1≈0.666667. **Yaad rakho:** Precision: jo laaye, unmein kitne sahi? Recall: jo sahi the, unmein kitne laaye? --- ## TEE · Q1b · NER and approaches Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f24--q1b/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-f24/q1b Previous examination · complete worked answers **Question. Explain NER with examples and approaches.** [individual allocation not printed; Q1 totals 20] ### Answer **Named Entity Recognition (NER)** identifies text spans referring to entities and assigns their types, such as PERSON, ORGANIZATION and LOCATION. Many practical schemas also include DATE, TIME, MONEY or domain-specific types. The exact allowed labels depend on the application's annotation scheme. NER has two parts: **boundary detection** finds where the entity starts and ends; **classification** assigns the appropriate type. A multiword name should normally be one entity span: “New York” is a location span, not two unrelated location decisions. Text: Mary joined Google in New York on Monday. → Mary → PERSON → Google → ORGANIZATION → New York → LOCATION → Monday → DATE Not every noun is a named entity. In “a developer works at Google”, developer is an ordinary job description while Google is a named organization. A specialized HR system may define a JOB\_TITLE category, but it must state that schema explicitly. NER identifies mentions; entity linking is the separate task of connecting a mention to a unique database identity. | Task | Question it answers | Example | | --- | --- | --- | | POS | What grammatical role does this token have? | Google/NNP | | Chunking | Which adjacent words form a phrase? | [NP a skilled developer] | | NER | Which span names an entity, and of what type? | [Google] ORGANIZATION | | Entity linking | Which real entity does this mention denote? | Connect Paris to the intended city or person record | **Applications.** NER structures search results and documents, extracts organizations/people from news, routes support tickets by product or organization, supports recommendation, and extracts specialized entities from healthcare or legal text. Context is essential: “Apple released a laptop” names an organization, while “she ate an apple” describes fruit. **Yaad rakho:** NER mein do kaam: naam kahan se kahan tak, aur kis category ka. | Approach | How it works | Example | Main limitation | | --- | --- | --- | --- | | Dictionary / gazetteer | Match text against lists of known names | A city list includes Mumbai and Pune | New names, spelling variants and ambiguous names are missed | | Rule-based | Use word shapes, regular expressions and context | A title such as Dr. followed by a name-like span suggests PERSON | Manual maintenance; exceptions and domain/language dependence | | Machine learning | Learn a labeling rule from annotated examples using features | Capitalization, suffixes, POS and surrounding words help a classifier/CRF | Requires labeled data and useful features | | Deep learning | Learn contextual word/character representations and predict labels | A contextual sequence encoder labels multiword entity spans | Training/domain dependence and resource demands | A **pattern-based rule** examines the form of text, such as an email pattern or a date format. A **context-based rule** examines neighboring words: “Dr.” can support a following PERSON span, while “headquartered in” can support a following LOCATION span. Capitalization alone is insufficient because sentence-initial common nouns are also capitalized. A dictionary may find “May” as a person's name, but “in May” may refer to a month. A learned model uses surrounding evidence to disambiguate. Hybrid systems can combine dictionaries, rules and learned models; these categories are practical tools rather than mutually exclusive restrictions. **Preprocessing for NER.** Tokenize while preserving token offsets and useful capitalization. Blindly removing capital letters or stopwords before recognition can damage names such as “Bank of India”. NER output should preserve the original span so the reader can locate it in the input. **Exam-ready 3-mark answer.** NER locates named-entity spans and classifies them into categories such as person, organization and location. A dictionary-based approach matches known name lists; a rule-based approach uses patterns or context; machine-learning and deep-learning approaches learn entity decisions from labeled data and representations. For example, in “Mary works at Google in Mumbai”, Mary is PERSON, Google is ORGANIZATION and Mumbai is LOCATION. --- ## TEE · Q1c · Skip-gram and CBOW Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f24--q1c/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-f24/q1c Previous examination · complete worked answers **Question. Explain Skip-gram and CBOW.** [individual allocation not printed; Q1 totals 20] ### Answer **Word2Vec learns dense word embeddings by predicting words from nearby context.** Its two common architectures are CBOW and skip-gram. A vocabulary ID or one-hot vector is the input representation; the learned embedding table converts that identity into a short trainable vector. | Architecture | Input | Prediction | Memory cue | | --- | --- | --- | --- | | CBOW | Surrounding context words | Centre/target word | Context → centre | | Skip-gram | Centre/target word | Surrounding words | Centre → context | CBOW: the + sat → Average context embeddings → Predict: cat Skip-gram: cat → Embedding of cat → Predict: the and sat For vocabulary size $$ V $$ and embedding dimension $$ d $$ , an input one-hot row is $$ 1\times V $$ ; the input embedding matrix is $$ V\times d $$ ; the hidden row is $$ 1\times d $$ ; the output matrix is $$ d\times V $$ ; output scores are $$ 1\times V $$ . Multiplying a one-hot row by the embedding matrix simply selects one row. $$ P(w_o\mid w_i)=\frac{\exp(u_{w_o}^{T}v_{w_i})}{\sum_{w\in V}\exp(u_w^{T}v_{w_i})}. $$ $$ v_{w_i} $$ is the input word embedding, $$ u_w $$ an output-context embedding, and the dot product is a compatibility score. Softmax turns all scores into probabilities that sum to one. Training increases the probability of observed context words. Negative sampling or hierarchical softmax can reduce computation in larger vocabularies; use the algorithm specified by the question. **Question. For the sentence “the cat sat on mat”, generate CBOW and skip-gram training data for centre word sat with a window of one token on each side.** #### Given Tokens indexed 1 to 5: the, cat, sat, on, mat. Centre = sat. #### To find The CBOW example and the skip-gram pairs. #### Formula and why it applies CBOW predicts the centre from its neighbours; skip-gram predicts each neighbour from the centre. #### Step 1. Identify neighbours Left neighbour = cat; right neighbour = on. “the” and “mat” are outside this window. #### Step 2. CBOW example Input [cat,on] → target sat. #### Step 3. Skip-gram examples (sat,cat) and (sat,on), with first item as input and second as predicted context. **Answer.** CBOW: [cat,on]→sat. Skip-gram: sat→cat and sat→on. **Question. The embeddings of context words cat and on are [1,2] and [3,0]. Compute the CBOW hidden vector using mean aggregation. Output vectors for [sat,ran] are [1,0] and [0,1]. Compute softmax probabilities.** #### Given Context embeddings [1,2], [3,0]; mean aggregation; two candidate output vectors. #### To find The hidden vector, logits and probabilities. #### Formula and why it applies Mean context vector $$ h=(v_1+v_2)/2 $$ . Logit $$ z_w=u_w^Th $$ . Softmax $$ P(w)=e^{z_w}/\sum_j e^{z_j} $$ . #### Step 1. Add coordinates $$ [1+3,\,2+0]=[4,2] $$ . #### Step 2. Divide by number of words $$ h=[4/2,2/2]=[2,1] $$ . #### Step 3. Compute logits $$ z_{sat}=1(2)+0(1)=2 $$ . $$ z_{ran}=0(2)+1(1)=1 $$ . #### Step 4. Exponentials $$ e^2\approx7.389056 $$ , $$ e^1\approx2.718282 $$ ; sum $$ \approx10.107338 $$ . #### Step 5. Divide $$ P(sat)\approx0.731059 $$ , $$ P(ran)\approx0.268941 $$ . **Answer.** h=[2,1]; sat is more probable (0.731059). **Question. A skip-gram centre embedding is v=[1,−1]. Output embeddings for [eat,sleep,drive] are [1,1], [0,1], [1,0]. Calculate the predicted context probabilities.** #### Given One centre vector and three output vectors; full softmax over these three words. #### To find Three logits and three probabilities. #### Formula and why it applies $$ z_j=u_j^Tv $$ , then $$ P(j)=e^{z_j}/\sum_k e^{z_k} $$ . #### Step 1. Eat dot product $$ 1(1)+1(-1)=0 $$ . #### Step 2. Sleep and drive dot products $$ z_{sleep}=0(1)+1(-1)=-1 $$ . $$ z_{drive}=1(1)+0(-1)=1 $$ . #### Step 3. Exponentiate and sum $$ e^0=1 $$ , $$ e^{-1}\approx0.367879 $$ , $$ e^1\approx2.718282 $$ . Sum $$ \approx4.086161 $$ . #### Step 4. Normalize $$ P(eat)\approx0.244728 $$ ; $$ P(sleep)\approx0.090031 $$ ; $$ P(drive)\approx0.665241 $$ . **Answer.** drive has the highest probability in this toy learned model. **Limitations:** a standard static Word2Vec vector merges a word's senses; it has limited representation of subword morphology and out-of-vocabulary words; its local prediction objective does not directly fit a full global co-occurrence table. Similarity is learned from data and is not a guarantee of factual or causal understanding. CBOW = neighbours se missing centre. Skip-gram = centre se neighbours. --- ## TEE · Q1d · Pragmatics and discourse Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f24--q1d/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-f24/q1d Previous examination · complete worked answers **Question. Explain pragmatics and discourse in NLP.** [individual allocation not printed; Q1 totals 20] ### Answer **Pragmatic analysis** interprets an utterance using its situation and the speaker’s intended meaning. **Discourse analysis** studies how utterances connect across a text or conversation. Syntax and literal word meanings alone cannot settle every interpretation. | Text | Literal content | Contextual interpretation | | --- | --- | --- | | “Can you close the window?” | A question about ability. | Usually a polite request. | | “Meera bought a laptop. It was expensive.” | Two sentences. | “It” connects the second sentence to the laptop. | | “Meet me here tomorrow.” | An instruction. | Me, here and tomorrow depend on speaker, place and date. | ### Reference, mention and antecedent A **referring expression** points to an entity; a **mention** is its occurrence in the text. Its **referent** is the entity being discussed. An **antecedent** is a preceding expression used to interpret a later referring expression. Reference resolution identifies the intended referent. Coreference resolution groups mentions that refer to the same entity. “Meera”name mention → Person entitysame referent → “she” / “the engineer”other mentions | Concept | Definition | Example | | --- | --- | --- | | Anaphora | Reference back to something introduced earlier. | “Riya arrived. She sat down.” She → Riya. | | Cataphora | Reference forward to a later expression. | “Before she spoke, Riya checked the microphone.” she → Riya. | | Exophora | Reference to the situation outside the text. | “Put it there”, said while pointing. | | Coreference | Two or more mentions refer to the same entity. | “Dr Rao”, “the lecturer”, “she”. | | Pronoun resolution | Find the referent of a pronoun. | Resolve “it” in “The laptop stopped because it overheated.” | Coreference is broader than pronoun resolution: repeated names and descriptions also form a chain. Anaphora describes direction/dependence; coreference describes identity. In “I bought a car. The engine is noisy”, engine is related to the car but is not the same entity. **Yaad rakho:** Reference = kiski baat? Coreference = alag mentions, same entity. Pronoun resolution = he/she/it kis se linked? ### Cohesion, coherence and discourse markers **Cohesion** is the explicit linguistic linking of a text through reference, repeated or related words, conjunctions and other devices. In “Rita bought a book. She read it”, she and it connect the second sentence to Rita and the book. **Coherence** is the conceptual unity that makes a discourse meaningful. “The road flooded, so the match was cancelled” presents a plausible cause and consequence. A reader relates the propositions using context and knowledge. **Discourse markers** signal relationships or organize speech: however signals contrast; therefore signals a result; first and next signal sequence; for example introduces an illustration. These concepts differ: “I like apples. Therefore the moon is square” has a linking word but no sensible inference. A connector alone does not guarantee coherence. **Yaad rakho:** Cohesion = words se link; coherence = meaning se link; marker = relation ka signal. --- ## TEE · Q2a · Six preprocessing operations Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f24--q2a/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-f24/q2a Previous examination · complete worked answers **Question. Explain lowercasing, tokenization, stopword removal, stemming, lemmatization and POS tagging with examples.** [6 marks] ### Answer **Tokenization divides text into units called tokens for further processing.** A token may be a word, punctuation mark, character or subword, depending on the tokenizer. **Sentence segmentation** identifies sentence boundaries. Tokenization is not necessarily just splitting on spaces. | Method | Example | Trade-off | | --- | --- | --- | | Word | “cats sleep” → [cats,sleep] | Easy to interpret; unfamiliar words can be unseen tokens | | Character | cat → [c,a,t] | Handles spelling units; creates longer sequences | | Subword | unhappiness → pieces such as [un,happi,ness] in an illustrative vocabulary | Balances vocabulary size and rare-word representation | | Sentence | “Birds fly. Fish swim.” → two sentences | Abbreviations and punctuation complicate boundaries | Actual subword splits depend on the learned vocabulary; the example is illustrative, not a universal tokenizer output. Word segmentation is especially important where spaces do not reliably separate words or in compound-rich languages. **Question. Tokenize “Dr. Rao can't come today.” into words and punctuation under this rule: preserve Dr. and can't as whole tokens, and split only the final full stop.** #### Given The tokenization rule is explicitly supplied. #### To find The resulting token sequence. #### Formula and why it applies Apply the stated boundary rule; do not assume every period or apostrophe is a split point. #### Step 1. Handle abbreviation Dr. remains one token. #### Step 2. Handle contraction can't remains one token. #### Step 3. Write sequence [Dr., Rao, can't, come, today, .] **Answer.** 6 tokens under the specified rule. Another tokenizer may legitimately produce a different sequence. ### Three difficult cases **New York** may be two tokens but one named entity. **3.14** should not normally end a sentence at the decimal point. **don't** may remain whole or split into do + n't, depending on the tokenizer. Always state the convention when calculating counts. Token = chosen unit. Space split ko universal rule mat samajh. **Normalization reduces superficial variation so comparable forms can be processed consistently.** Common operations include lowercasing, Unicode normalization, appropriate spelling correction, contraction handling, and optional stemming or lemmatization. **Stopwords** are frequent words selected for removal for a particular task, such as the or a. Removing them may reduce feature size, but removing not, no or nor can destroy sentiment meaning. Punctuation can convey sentence boundaries, emotion or mathematical structure. Lowercasing can erase differences such as US versus us. **Question. Preprocess “The CATS are not sleeping!!!” by lowercasing, removing punctuation, removing only stopwords {the,are}, and lemmatizing plural cats→cat and sleeping→sleep.** #### Given Operations and stopword set are fixed. Do not remove not. #### To find The final token list. #### Formula and why it applies Apply the requested operations in order, keeping a visible intermediate result after each transformation. #### Step 1. Lowercase the cats are not sleeping!!! #### Step 2. Tokenize and remove punctuation [the,cats,are,not,sleeping] #### Step 3. Remove specified stopwords Remove the and are → [cats,not,sleeping]. #### Step 4. Lemmatize cats→cat; sleeping→sleep → [cat,not,sleep]. **Answer.** [cat,not,sleep]. The negative meaning is retained. ### Task decides preprocessing For a simple keyword-retrieval model, removing selected function words can help. For machine translation, question answering, grammar analysis or transformer input, aggressive removal may discard necessary syntax or violate the model's expected input format. Emoji can carry sentiment; convert or retain them when relevant instead of always deleting them. Clean karna = useful signal bachake noise hatao. Har word/punctuation delete karna cleaning nahi. **Stemming** applies string-based rules, often stripping suffixes, to reduce related forms to a stem. The result need not be a valid dictionary word. **Lemmatization** uses vocabulary and morphological analysis, often with POS information, to return a dictionary lemma. | Criterion | Stemming | Lemmatization | | --- | --- | --- | | Basis | Surface form/suffix rules | Lexicon and morphological/POS analysis | | Output | May be nonword, e.g. studi | Dictionary form, e.g. study | | Context/POS | Usually little or none | Often important | | Irregular forms | May miss went→go | Can map went→go with verb analysis | | Cost | Generally simpler | Usually more linguistic processing | Porter applies a sequence of suffix-removal rules subject to conditions; Snowball provides related language-specific stemming algorithms; Lancaster is often more aggressive; regex stemming uses explicit patterns; Lovins is another established stemming approach. Exact output depends on the algorithm and implementation. ### Three worked reductions 1. **studies, studying:** an English stemmer may yield studi/study depending on the rule sequence; verb lemmatization aims at study. State the exact algorithm if exact outputs are asked. 2. **better:** adjective-aware lemmatization can map it to good; naïve suffix removal cannot infer that irregular relation. 3. **saw:** as a past-tense verb it can become see; as a cutting-tool noun it remains saw. POS and context matter. **Over-stemming** merges forms that should remain distinct; **under-stemming** fails to merge related forms. Normalization can increase recall in search while losing useful distinctions, so it is a task-dependent choice. Stem = rules se cut; lemma = dictionary form, meaning/POS ka dhyan. **Part-of-speech (POS) tagging** assigns a grammatical category to every word token in context, such as noun, verb, adjective or determiner. It is a sequence-labeling task: the input is a sequence of words, and the output is a tag for each word. A tag describes the word's role in this sentence, not its meaning in every possible sentence. | Tag | Role | Example in context | | --- | --- | --- | | NN / NNS | Singular / plural common noun | a book / two books | | NNP | Proper noun | Mary, Mumbai | | VB / VBD / VBZ | Base / past / third-person singular verb | read / read / reads | | JJ | Adjective | the red book | | RB | Adverb | runs quickly | | DT | Determiner | the, a, this | | IN | Preposition or subordinating conjunction | on the table | | PRP | Personal pronoun | she, they | | MD | Modal auxiliary | can, will, should | **Open classes** readily accept new content words: nouns, verbs, adjectives and adverbs. Examples include chatbot and livestream. **Closed classes** contain comparatively stable grammatical/function words: determiners, pronouns, prepositions and conjunctions. These are tendencies, not a claim that language never changes. Input: Book that flight → Context: an imperative request → Output: Book/VB that/DT flight/NN Compare “hand me that **book**”: book is a noun here. Similarly, “I **can** swim” has can/MD, while “the **can** is empty” has can/NN. The spelling stays the same; neighboring words and sentence structure change the correct tag. This is **POS ambiguity**. | Sentence | Tagged interpretation | Reason | | --- | --- | --- | | They watch birds. | They/PRP watch/VBP birds/NNS | watch is the action. | | The watch stopped. | The/DT watch/NN stopped/VBD | watch is the object that stopped. | | She has a light bag. | She/PRP has/VBZ a/DT light/JJ bag/NN | light describes bag. | | Light the lamp. | Light/VB the/DT lamp/NN | Light is an imperative verb. | POS tags support parsing, named entity recognition, question answering, sentiment analysis and word-sense disambiguation. They do not by themselves identify every entity or fully resolve meaning. **Yaad rakho:** Word ka naam same ho sakta hai; sentence mein uska kaam tag decide karta hai. --- ## TEE · Q2b · Lexical similarity and cosine Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f24--q2b/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-f24/q2b Previous examination · complete worked answers **Question. Explain the need for lexical similarity and illustrate cosine similarity.** [4 marks] ### Answer Similarity supports retrieval, clustering, duplicate detection and semantic matching. Choose a representation first: character overlap measures spelling; context embeddings approximate semantic relatedness. An illustrative cosine computation follows; these vectors are chosen to explain the method, not given exam data. ### Given u=[1, 1, 0]; v=[1, 0, 1] ### To find Cosine similarity in the given coordinate order. ### Formula: which one and why $$ \cos(u,v)=\frac{u\cdot v}{\|u\|\|v\|} $$ ### Step 1. Multiply matching coordinates 1×1 + 1×0 + 0×1 = 1 ### Step 2. Calculate both squared lengths u: 1² + 1² + 0² = 2; v: 1² + 0² + 1² = 2 ### Step 3. Take square roots and divide 1 / (√2 × √2) = 0.500000000 ### Final answer **Cosine=0.500000000.** This measures direction, not a probability. --- ## TEE · Q2c · Animal categorical Naive Bayes Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f24--q2c/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-f24/q2c Previous examination · complete worked answers **Question. Using the printed20-animal training table, classify an animal with Give Birth=Yes, Can Fly=No, Live in Water=Yes, Have Legs=No. Treat Name only as an identifier.** [10 marks] ### Answer Rows below follow the source order: Human, Python, Salmon, Whale, Frog, Komodo, Bat, Pigeon, Cat, Leopard Shark, Turtle, Penguin, Porcupine, Eel, Salamander, Gila Monster, Platypus, Owl, Dolphin, Eagle. The source assigns Leopard Shark to Non-mammals and Platypus to Mammals; retain those labels. ### Given | Give Birth | Can Fly | Live in Water | Have Legs | Class | | --- | --- | --- | --- | --- | | Yes | No | No | Yes | Mammals | | No | No | No | No | Non-mammals | | No | No | Yes | No | Non-mammals | | Yes | No | Yes | No | Mammals | | No | No | Sometimes | Yes | Non-mammals | | No | No | No | Yes | Non-mammals | | Yes | Yes | No | Yes | Mammals | | No | Yes | No | Yes | Non-mammals | | Yes | No | No | Yes | Mammals | | Yes | No | Yes | No | Non-mammals | | No | No | Sometimes | Yes | Non-mammals | | No | No | Sometimes | Yes | Non-mammals | | Yes | No | No | Yes | Mammals | | No | No | Yes | No | Non-mammals | | No | No | Sometimes | Yes | Non-mammals | | No | No | No | Yes | Non-mammals | | No | No | No | Yes | Mammals | | No | Yes | No | Yes | Non-mammals | | Yes | No | Yes | No | Mammals | | No | Yes | No | Yes | Non-mammals | Test: {'Give Birth': 'Yes', 'Can Fly': 'No', 'Live in Water': 'Yes', 'Have Legs': 'No'} ### To find Class under categorical Naive Bayes. ### Formula: which one and why $$ \mathrm{score}(c)=P(c)\prod_jP(x_j\mid c) $$ Use empirical categorical frequencies, not a multinomial word model. No likelihood is zero for the requested test, so unsmoothed estimates are sufficient. Each feature contributes once; ignore names/identifiers as predictors. ### Step 1. Count examples in each class | Class | Count | Prior | | --- | --- | --- | | Mammals | 7 | 7/20 | | Non-mammals | 13 | 13/20 | ### Step 2. Count each requested feature within each class and multiply | Class | n | Prior | Give Birth | Can Fly | Live in Water | Have Legs | Exact product | Decimal product | | --- | --- | --- | --- | --- | --- | --- | --- | --- | | Mammals | 7 | 7/20 | 6/7 | 6/7 | 2/7 | 2/7 | 36/1715 | 0.020991253644 | | Non-mammals | 13 | 13/20 | 1/13 | 10/13 | 3/13 | 4/13 | 6/2197 | 0.002730996814 | ### Step 3. Normalize for a posterior if wanted | Class | Score / score sum | | --- | --- | | Mammals | 0.884876150 | | Non-mammals | 0.115123850 | ### Final answer **Predicted class: Mammals.** The independence assumption is conditional on class, not a claim that these biological/weather features are truly independent. --- ## TEE · Q3a · Hobbs and ambiguous professor–student reference Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f24--q3a/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-f24/q3a Previous examination · complete worked answers **Question. Explain Hobbs’ algorithm, including binding, gender, distance and recency. Resolve “The professor advised the student because she was struggling.”** [10 marks] ### Answer **Hobbs’ algorithm is a syntax-based procedure that searches constituent parse trees in a fixed order for a compatible noun-phrase antecedent of a pronoun.** The tree determines which candidates are proposed first. Agreement and other relevant constraints determine whether a proposed candidate can be accepted. It is not simply the rule “choose the closest noun”. A nearby noun may have the wrong number, may be syntactically ineligible, or may be visited after a structurally more prominent noun. The algorithm produces a preference under the supplied parse; ambiguous language can still require context. ### The tree vocabulary you need | Term | Meaning | Example | | --- | --- | --- | | NP | Noun phrase: a phrase referring to an entity | the student; Anita; they | | S | A sentence/clause node | the node containing a subject and predicate | | VP | Verb phrase | told Ravi that she should revise | | X | The NP or S node currently being searched | An algorithm label, not a word | | Path p | The branch travelled upward from the pronoun/previous X | Used to distinguish left and right subtrees | | N-bar (N′) | The noun-centred part of an NP in a grammar using that label | The nominal-child test depends on the given tree | | Breadth-first | Visit shallower nodes before deeper ones | Level 1 before level 2 | | Left-to-right | Visit equally deep nodes in their written/tree order | Earlier left branch before right branch | Locate pronoun NP → Climb to NP/S → Propose candidates in tree order → Check compatibility → Accept or continue searching ### The complete search procedure #### Step 1. Start at the pronoun Locate the NP immediately dominating the pronoun. “Dominating” means containing it as a descendant in the tree. #### Step 2. Climb to the first NP or S Move upward to the first NP or S above that starting NP. Call this node X. Keep the upward path p. #### Step 3. Search the left branches initially Below X, traverse branches to the left of p breadth-first and left-to-right. At this initial search, propose an NP only if another NP or S lies between that candidate and X. Test proposed candidates for the required agreement and constraints. #### Step 4. If X is the highest sentence node, inspect earlier sentences Search previous sentence trees from most recent to oldest. Within each, use breadth-first, left-to-right order to propose NPs. Accept the first eligible compatible candidate, or report that no candidate was found in the available context. #### Step 5. Otherwise climb again Move from X to the next NP or S above it. Record this new X and the path followed during the climb. #### Step 6. Apply the higher-NP test If the new X is an NP, propose X when the path did not pass through its nominal N-bar child. Use the given parse labels; do not invent an N-bar branch missing from the question's structure. #### Step 7. Search its left branches Traverse branches below the new X to the left of the path, breadth-first and left-to-right, proposing NPs. The special intervening-NP/S restriction belongs to the initial search in Step 3. #### Step 8. If X is S, search the permitted right branches Search branches to the right of the path breadth-first and left-to-right. Do not descend below an encountered NP or S. Propose eligible NP candidates there. #### Step 9. Repeat or stop Return to the highest-S check and continue the climb/search procedure until a compatible candidate is accepted or the available trees are exhausted. Record the candidate order rather than only writing an unexplained name. ### Why the first left search has an extra condition In a simple clause such as “John saw him”, the subject NP John is directly under the clause S. There is no extra NP or S between John and that S. The initial-search restriction therefore prevents casually selecting John as the ordinary local antecedent of him. Compare the reflexive “John saw himself”, which is a different grammatical case; do not treat every pronoun type as interchangeable. ### Worked tree 1 · move to the previous sentence **Question. Apply Hobbs’ search to “The student arrived. She smiled.” Assume the student is compatible with she.** #### Given Previous tree: [S [NP The student] [VP arrived]]. Current tree: [S [NP She] [VP smiled]]. #### To find The selected antecedent and the search path. #### Method and why it applies Use the supplied constituent trees. The current sentence has no earlier candidate, so the highest-S rule activates previous-sentence search. ### Solution #### Step 1. Draw both sentence structures ``` Previous sentence Current sentence S S ├── NP: The student ├── NP: She ← start └── VP: arrived └── VP: smiled ``` #### Step 2. Locate and climb Start at NP She. Its first higher NP/S is the current S. Set X to that S. #### Step 3. Search left of the path She is the leftmost subject. No NP lies to its left in the current sentence. #### Step 4. Use the highest-S rule This X is the highest S. Inspect the immediately preceding sentence tree. #### Step 5. Propose its first NP Breadth-first, left-to-right finds NP The student. #### Step 6. Check compatibility and accept The question stipulates that the student agrees with she. Accept this candidate. **Answer.** She refers to the student under the supplied assumptions. ### Worked tree 2 · reject an incompatible subject **Question. Resolve they in “Riya spoke to the students. They laughed.” Use a previous-sentence search and interpret they as plural.** #### Given Previous tree has subject NP Riya, then VP spoke [PP to [NP the students]]. Current subject is NP They. #### To find The candidate order and accepted antecedent. #### Method and why it applies Breadth-first search favours a shallow subject, but agreement must still be applied; a first candidate is not automatically a valid answer. ### Solution #### Step 1. Draw the useful part of the previous tree ``` S ├── NP: Riya ← candidate 1 └── VP ├── V: spoke └── PP ├── P: to └── NP: the students ← candidate 2 ``` #### Step 2. Search the current sentence From NP They climb to its S; there is no earlier NP in “They laughed”. This is the highest S, so move to the previous tree. #### Step 3. Visit Riya first Riya is the shallower leftmost NP. Under the stated plural interpretation, this singular candidate fails number agreement. #### Step 4. Visit the students Continue breadth-first. The next relevant NP is the students. #### Step 5. Apply number agreement Students is plural and is an eligible candidate. It matches plural they. #### Step 6. State the resolution Accept the students. The method did not simply accept the first noun it saw. **Answer.** They → the students. Candidate order: Riya rejected; the students accepted. ### Worked tree 3 · climb out of an embedded clause **Question. Resolve she in “Anita told Ravi that she should revise.” Assume Anita is compatible with she and Ravi is not. Use the displayed parse.** #### Given The pronoun is the subject of an embedded S inside the main VP. #### To find The preferred antecedent found by the climb and left-branch search. #### Method and why it applies The embedded clause has no left candidate. Climbing to the main S exposes its subject Anita as the first suitable higher-level candidate. ### Solution #### Step 1. Show the clause structure ``` S (main) ├── NP: Anita ← accepted candidate └── VP ├── V: told ├── NP: Ravi └── SBAR: that └── S (embedded) ← first X ├── NP: she ← start └── VP: should revise ``` #### Step 2. Start inside the embedded clause From NP she climb to the embedded S. There is no candidate to the left of she within this embedded S. #### Step 3. Climb to the next NP/S The embedded S is not the highest S. Move upward through SBAR and VP to the main S. These intermediate nodes are not NP/S search nodes. #### Step 4. Search left of the upward path At the main S, the subject NP Anita is a shallow left candidate. The object Ravi is deeper within the VP. Breadth-first order proposes Anita first. #### Step 5. Check the stated compatibility Anita is compatible with she by the question's assumption. Accept it; no need to choose a later incompatible candidate. #### Step 6. Give the result and its limit This is the algorithm's result under the displayed parse and agreement assumptions. It does not prove the intention of every speaker using an otherwise ambiguous sentence. **Answer.** She → Anita under the stated tree and compatibility information. ### Advantages, limitations and a complete exam answer | Advantages | Limitations | | --- | --- | | Explicit candidate order makes the decision traceable. | Wrong parse structure can produce the wrong search order. | | Requires no labelled coreference-training corpus for its basic rules. | World knowledge and discourse intention may override syntactic preference. | | Uses structure rather than only surface distance. | Some ambiguous, long-distance or unusual constructions need additional treatment. | For a longer answer, write the definition, inputs, tree notation, ordered algorithm, one labelled tree with a candidate table, final antecedent, and limitations. In a worked question show separately: candidate → why proposed → agreement result → accept/reject. Do not equate the selected candidate with a logically guaranteed interpretation when the sentence is ambiguous. **Yaad rakho:** Tree order candidate deta hai; agreement usko allow/reject karta hai. Nearest noun automatically answer nahi. **The intended antecedent is not uniquely determined by the printed sentence.** Both professor and student can be singular female people; neither noun supplies gender. Structural Hobbs search can propose the matrix subject professor early, while the causal meaning of advice because of struggling often supports student. Distance/recency favouring student is a preference, not proof. 1. Parse subject NP the professor, object NP the student, subordinate-clause subject she. 2. Generate structurally accessible earlier NPs through the Hobbs ascent and breadth-first leftward search. 3. Check grammatical number/person and binding constraints. Both candidates can pass here; she is not a reflexive. 4. Gender/world knowledge is unavailable, so do not invent either person’s gender. 5. Use broader discourse to select. Without it, report professor/student ambiguity; explain that contextual interpretation often prefers student but the syntax-only algorithm does not guarantee that intent. **Source / convention note.** Pronoun referent is ambiguous without extra gender/discourse information; structural algorithm preference is separated from semantic interpretation. ### The four requested properties | Property | What it means | Apply here | | --- | --- | --- | | Binding | Syntactic restrictions on reference; ordinary personal pronouns and reflexives have different local-antecedent requirements | Do not mechanically link every nearby NP; he in an after/because clause can refer outside that local clause. | | Gender agreement | Candidate must match the pronoun under available linguistic/contextual knowledge | John/he is the exercise’s conventional reading; professor/student do not provide gender by themselves. | | Distance | Token, syntactic or sentence separation between pronoun and candidate | Closer candidates may be preferred, but cannot override incompatibility or Hobbs search order. | | Recency | Recently mentioned discourse entities are often more salient | Mary/student is later than John/professor, but recency is a preference, not a guaranteed antecedent rule. | --- ## TEE · Q3b · Classification and sentiment in industry Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f24--q3b/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-f24/q3b Previous examination · complete worked answers **Question. Discuss real-world text classification and sentiment analysis, with examples showing decision-making and customer engagement.** [10 marks] ### Answer **Text classification** assigns a document to one or more predefined categories. Spam detection usually uses two classes; news classification may use several topics. In single-label multiclass classification, choose one of several categories. In multilabel classification, several labels may apply to the same document. Define labels → Prepare labelled examples → Train representation + model → Evaluate on unseen text → Predict new labels ### An exam-ready pipeline | Stage | What you do | Why it matters | | --- | --- | --- | | Task definition | Decide what counts as spam or which news topics exist. | Unclear labels create inconsistent training data. | | Data collection and annotation | Collect representative examples and assign correct labels. | The model learns from these examples. | | Split data | Separate training, validation and test data. | Test examples must not influence training choices. | | Preprocess | Tokenize and normalise where suitable; retain meaningful negation. | Reduce irrelevant variation without deleting signal. | | Represent | Use BoW, TF-IDF or learned embeddings. | Models need numerical features. | | Train | Fit Naive Bayes, logistic regression or another appropriate model. | Learn how features predict categories. | | Evaluate | Measure relevant precision, recall and F1 on held-out examples. | Accuracy alone may hide poor minority-class performance. | | Deploy and monitor | Apply the same preprocessing and monitor changing language. | Incoming data can differ from old examples. | Naive Bayes is especially useful for hand calculations. With a bag-of-words representation, its class score is the prior multiplied by the conditional probability of each token; repeated tokens repeat the factor. The “naive” assumption is conditional independence of features given the class, not that words are independent in real language. $$ s(c)=P(c)\prod_{w\in V}P(w\mid c)^{C(w,d)} $$ $$ P(w\mid c)=\frac{C(w,c)+1}{N_c+|V|} $$ C(w,d) counts word w in document d. C(w,c) counts it in training documents belonging to class c. Nc is the total token count for class c. Add-one smoothing adds one to every vocabulary count and therefore adds |V| to the denominator. This is the same arithmetic taught for WSD, with document classes instead of senses. ### Worked classification · two-token message A spam classifier has vocabulary {win, prize, meeting}. Priors are P(spam)=P(ham)=0.5. Spam token counts are [3,2,0]; ham counts are [0,0,5]. With add-one smoothing, classify “win prize”. **Given** |V|=3; N(spam)=5; N(ham)=5; message has win once and prize once. **To find** Class scores and predicted label. ### Formula and why it applies $$ s(c)=P(c)P(\text{win}\mid c)P(\text{prize}\mid c) $$ $$ P(w\mid c)=\frac{C(w,c)+1}{N_c+|V|} $$ The model is multinomial Naive Bayes; multiply the two token likelihoods and the class prior. ### Solution #### Step 1. Compute the smoothed denominators $$ 5+3=8 $$ #### Step 2. Compute spam likelihoods $$ P(\text{win}\mid S)=\frac{3+1}{8}=\frac12 $$ $$ P(\text{prize}\mid S)=\frac{2+1}{8}=\frac38 $$ #### Step 3. Compute ham likelihoods $$ P(\text{win}\mid H)=P(\text{prize}\mid H)=\frac{0+1}{8}=\frac18 $$ #### Step 4. Multiply the spam score $$ s(S)=\frac12\times\frac12\times\frac38=\frac{3}{32}=0.09375 $$ #### Step 5. Multiply the ham score $$ s(H)=\frac12\times\frac18\times\frac18=\frac{1}{128}=0.0078125 $$ #### Step 6. Compare 0.09375 > 0.0078125. These are unnormalised scores; the label is chosen by the larger score. Predicted label: spam. **Yaad rakho:** Class prior × har word ki class-likelihood; sabse bada score wins. **Sentiment analysis** identifies expressed opinion or attitude, often as positive, negative or neutral. Document-level analysis labels a complete text; sentence-level analysis labels a sentence; aspect-level analysis attaches sentiment to a specific feature or entity. | Review | Useful interpretation | | --- | --- | | “The display is excellent, but the battery is poor.” | Display: positive; battery: negative. One overall label loses this distinction. | | “The service is not good.” | Negative: negation changes good. Removing not can reverse the intended meaning. | | “Great, another two-hour delay.” | Likely sarcasm: literal positive word alone is insufficient. | ### Approaches **Lexicon based:** look up sentiment scores, combine them, and apply stated handling for negation/intensifiers. It is interpretable but fragile with context and sarcasm. **Supervised:** learn a mapping from labelled text features to sentiment labels. **Neural/contextual:** learn or use contextual representations to capture word interactions, then fine-tune a sentiment classifier. None guarantees correct interpretation of every review. ### A specified lexicon calculation Use good = +2, excellent = +3 and slow = −2. A “not” immediately before a sentiment word reverses its score. Score “The support was not good but the delivery was excellent and slow.” **Given** Three sentiment words; not modifies good only; sum word scores under the stated rule. **To find** Total sentiment score and label, taking positive totals as positive. ### Formula and why it applies $$ S=\sum_i s_i $$ Apply the given negation rule before summing. This is a simplified specified model, not a universal linguistic rule. ### Solution #### Step 1. Identify sentiment words good, excellent and slow. #### Step 2. Apply negation $$ s(\text{not good})=-2 $$ #### Step 3. Add the contributions $$ S=-2+3-2=-1 $$ #### Step 4. Map score to label The sum is negative. Total score = −1; negative under this rule. | Industry | Concrete input and output | Useful decision / limitation | | --- | --- | --- | | Retail | Review→topic and sentiment | Route delivery complaints; sarcasm/negation may mislead | | Banking | Customer message→card/fraud/loan intent | Route to the correct team; a model flag is not proof of fraud | | Healthcare | Feedback→service-quality topics | Detect recurring appointment issues; protect patient identifiers | | Media | Article→topic; public responses→sentiment | Organize feeds and monitor reaction; sampling bias prevents treating posts as a census | --- ## TEE · Q4a · Top-down syntax tree Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f24--q4a/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-f24/q4a Previous examination · complete worked answers **Question. Explain the need for syntax parsing; compare top-down and bottom-up parsing. Parse “the cat chased the dog” using S→NP VP, NP→Det N, VP→V NP, Det→the, N→cat|dog, V→chased|saw.** [10 marks] ### Answer **Top-down parsing** starts at the grammar's start symbol S and expands nonterminals until the leaves match the input words in order. **Recursive descent** commonly implements each nonterminal as a procedure that tries its productions. It builds from the root toward the leaves. Start with S → Choose an allowed S production → Expand its next nonterminal → Match the resulting terminal to the input → Continue, or backtrack if the choice fails **Algorithm.** Start with S as the current goal. Choose a production for the leftmost unexpanded nonterminal. Replace that nonterminal by the production's right side. If a terminal appears, compare it with the next input token. If it matches, consume the token; if it does not, reject that branch and try another production. Accept only when the complete input is consumed and all grammar goals are satisfied. A **leftmost derivation** always expands the leftmost remaining nonterminal first. Top-down parsing can make guesses that the actual words cannot support, so it may need backtracking. A naive recursive-descent parser can loop on left-recursive rules such as NP→NP PP unless the grammar or algorithm is changed. Predictive LL(1) parsing chooses productions using one lookahead token when the grammar permits; not every English CFG is LL(1). | Approach | Begins with | Direction | Common difficulty | | --- | --- | --- | --- | | Top-down | Start symbol S | Root to word leaves | Predictions that do not match input; left recursion | | Bottom-up | Actual input tokens | Leaves to root S | Reductions that cannot contribute to a complete sentence | **Yaad rakho:** Top-down: sentence ka structure pehle, words tak neeche aao. **Bottom-up parsing** starts from the input tokens and combines them into larger constituents until the start symbol is reached. A **shift-reduce parser** maintains a stack of recognized material and a list of remaining input tokens. | Action | Exact operation | Example | | --- | --- | --- | | Shift | Move the next input token onto the stack | Stack: the; remaining input: dog runs | | Reduce | Replace a suffix of the stack matching a rule's RHS with its LHS | Det N becomes NP using NP→Det N | | Accept | Stack is S and input is exhausted | Stack: S; input: empty | | Fail/backtrack | Chosen reductions cannot complete a parse | A noun reading of book blocks an imperative parse | Reduction uses a production in reverse: the grammar NP→Det N expands an NP top-down, but reduces Det N into NP bottom-up. The stack's **rightmost item is its top** in the traces below. Each row shows the state **after** the listed action, avoiding an off-by-one mismatch between actions and stack contents. A lexical reduction such as N→dog still counts as a reduction. Do not skip the sequence from word to category to phrase when the question asks for a complete trace. Backtracking means returning to an earlier choice and trying another applicable rule, not changing the original input. **Yaad rakho:** Shift = input se stack. Reduce = chhote pieces se bada grammatical unit. ### Exact top-down derivation 1. S 2. NP VP 3. Det N VP 4. the N VP 5. the cat VP 6. the cat V NP 7. the cat chased NP 8. the cat chased Det N 9. the cat chased the N 10. the cat chased the dog SNPDettheNcatVPVchasedNPDettheNdog [S [NP [Det the] [N cat]] [VP [V chased] [NP [Det the] [N dog]]]] Parsing assigns constituent structure and grammatical relationships for extraction, translation and ambiguity resolution. Top-down expands from S toward input; bottom-up combines observed tokens toward S. This derivation uses only supplied rules. --- ## TEE · Q4b · Data acquisition and four augmentation methods Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f24--q4b/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-f24/q4b Previous examination · complete worked answers **Question. Explain NLP data acquisition, why augmentation is needed, and any four augmentation methods.** [10 marks] ### Answer **Data acquisition** obtains language examples relevant to the task from existing data, public resources or appropriate new collection. Product instrumentation can gather useful feedback. Supervised learning additionally needs an annotation process when labels are absent. **Data augmentation** creates training variants intended to preserve the correct label. It is not simply changing words arbitrarily. | Method | Procedure | Caution | | --- | --- | --- | | Synonym replacement | Use suitable synonyms for selected words | Choose the right sense | | Back translation | Translate to another language and back | May change meanings/entities | | TF–IDF-informed replacement | Protect informative words while varying others | Do not remove label-bearing clues | | Bigram flipping | Swap adjacent words | Can break grammar or meaning | | Entity replacement | Replace by same-type entity | Must preserve the task label | | Noise injection | Simulate typos/keyboard errors | Match expected noise | | Active learning | Select informative cases for human labels | This acquires labels; it is not synthetic text | ### Three label decisions “The film is good”→“The film is excellent” may preserve a positive sentiment label. “The film is not good”→“The film is good” reverses the label and is not valid label-preserving augmentation. “Flights to Pune”→“Flights to Mumbai” may preserve a travel-topic label but changes the destination label. Split train/test data before creating related augmented variants. Otherwise near-duplicates can leak between sets and produce misleading evaluation. Keep a representative evaluation set of genuine examples. Sentence badal sakta hai; required correct label nahi. Label badla toh augmentation invalid. | Method | Example | Required label check | | --- | --- | --- | | Synonym replacement | good film→excellent film | Replacement must preserve sense and sentiment | | Back-translation | English→French→English paraphrase | Check negation, entity and factual preservation | | Contextual paraphrasing | The delivery was late→The package arrived late | Do not change class or named facts | | Controlled noise / perturbation | Small realistic spelling error or punctuation variation | Appropriate only if target should remain unchanged | Acquire licensed/public datasets, commissioned annotation, surveys/user-consented logs or permitted scraping. Augment training data only after splitting to avoid near-duplicate leakage into validation/test. More synthetic examples do not guarantee better generalization. --- ## TEE · Q5a · Feature engineering, BoW and Euclidean Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f24--q5a/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-f24/q5a Previous examination · complete worked answers **Question. Explain text feature engineering and list vectorization approaches. For the three sentences below remove stopwords {is,that,are,to,and}, create vocabulary and BoW, then compute Euclidean distance between sentences1and2: “Natural Language Processing is fascinating”; “I love Natural Language processing and like to study Natural Language”; “I like things that are fascinating”.** [10 marks] ### Answer Feature engineering converts text into informative model inputs. Common approaches: one-hot, BoW, bag of n-grams, TF–IDF and dense embeddings. This question specifically requests count-based BoW, so do not use TF–IDF in its distance. ### Given Use lowercase lexical tokens and the exact printed stoplist; I remains because it is not a stopword here. ### To find Vocabulary, count matrix and distance D1–D2. ### Formula: which one and why $$ d_E(u,v)=\sqrt{\sum_i(u_i-v_i)^2} $$ ### Step 1. Tokenize and remove the exact stoplist | Document | Retained tokens | | --- | --- | | 1 | natural, language, processing, fascinating | | 2 | i, love, natural, language, processing, like, study, natural, language | | 3 | i, like, things, fascinating | ### Step 2. Fix vocabulary and count repeats | Document | fascinating | i | language | like | love | natural | processing | study | things | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | 1 | 1 | 0 | 1 | 0 | 0 | 1 | 1 | 0 | 0 | | 2 | 0 | 1 | 2 | 1 | 1 | 2 | 1 | 1 | 0 | | 3 | 1 | 1 | 0 | 1 | 0 | 0 | 0 | 0 | 1 | ### Step 3. Subtract coordinates [1, -1, -1, -1, -1, -1, 0, -1, 0] ### Step 4. Square, sum and take the root 1 + 1 + 1 + 1 + 1 + 1 + 0 + 1 + 0 = 7; √7 = 2.645751311 ### Final answer **Euclidean distance=2.645751311.** Natural and Language each occur twice in sentence2; preserve both counts. --- ## TEE · Q5b · WSD approaches and uses Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f24--q5b/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-f24/q5b Previous examination · complete worked answers **Question. Explain WSD, its approaches and applications.** [5 marks] ### Answer **Word sense disambiguation (WSD) is the task of selecting the intended sense of an ambiguous word in its context from a set of possible senses.** The output is a sense label, not a different replacement word. Input: “He grilled the bass.” → Candidate senses: fish / low sound → Context: grilled → Output: fish sense The **target word** is the ambiguous word. The **context** is the surrounding evidence. The **sense inventory** lists candidate meanings, often using WordNet. A **feature** is a usable clue: nearby word, position, part-of-speech tag or a phrase. | Approach | What it needs | How it decides | Limitation | | --- | --- | --- | --- | | Knowledge based | Dictionary, thesaurus or semantic graph | Definition overlap, category evidence or graph scores | Short definitions and missing knowledge | | Supervised | Examples already labelled with senses | Learns context → sense mapping | Labelling cost, limited rare-sense examples | | Semi-supervised | Small labelled set plus unlabelled data | Bootstraps from reliable seed examples | Early errors can spread | | Unsupervised sense induction | Unlabelled contextual occurrences | Clusters similar contexts | Clusters are not automatically named dictionary senses | | Neural/contextual | Learned word/context representations | Predicts or compares a context representation | Needs suitable data/model and evaluation | **Bag-of-words features** record nearby words while ignoring exact positions. **Collocational features** preserve a positional phrase such as “river bank”, “bank account”, “chocolate bar” or “bar exam”. ### Why WSD matters Machine translation needs the correct target-language meaning; information retrieval should distinguish river-bank travel from banking services; text-to-speech may need different pronunciations for “bass”; information extraction should avoid connecting entities through the wrong relation. WSD: same spelling ke options mein context dekh ke sahi meaning select. --- ## TEE · Q5c · Discourse analysis of a conversation Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f24--q5c/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-f24/q5c Previous examination · complete worked answers **Question. Analyze coherence, cohesion, topic structure, information flow and discourse markers: Speaker1 “I went to the store to buy some milk.” Speaker2 “Was it crowded?” Speaker1 “Yes, I had to wait in line for10minutes.” Speaker3 “I was going to attend the meeting, but I got stuck in traffic.”** [5 marks] ### Answer | Feature | Evidence and explanation | | --- | --- | | Coherence | First3turns form a meaningful store/crowding/waiting exchange. The meeting/traffic turn shifts topic without an explicit bridge; local relevance is weaker. | | Cohesion | it refers to the store; Yes answers the preceding question; first-person I shifts referent with each speaker. | | Topic structure | Store visit→crowding→wait; then new topic of meeting attendance and traffic. | | Information flow | Known store is followed by new question crowding, then new evidence10-minute wait. The last turn introduces separate plans and obstacle. | | Discourse markers | Yes signals affirmation; but signals contrast between intention and obstructing traffic. “To buy” states purpose, not merely temporal order. | Cohesion is explicit linguistic linkage; coherence is meaningful conceptual connection. A conversation can contain grammatical connectors and still have a poorly connected topic shift. --- ## TEE · Q6a · Henry bigram continuations Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f24--q6a/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-f24/q6a Previous examination · complete worked answers **Question. From the seven supplied training sentences below, predict the most probable next token for Do ..., I like ..., and I ... using bigrams.** [10 marks] ### Answer ### Given I am Henry I like college Do Henry like college Henry I am Do I like Henry Do I like college I do like Henry ### To find Most probable next token for each prefix. ### Formula: which one and why $$ P(w\mid h)=\frac{c(h,w)}{\sum_v c(h,v)} $$ Retain case exactly; include sentence-end as an observed next token. Bigram prediction uses only the last token of the prefix. ### Step 1. Count each adjacent pair separately within sentences | Bigram | Count | | --- | --- | | → Do | 3 | | → Henry | 1 | | → I | 3 | | Do → Henry | 1 | | Do → I | 2 | | Henry → | 3 | | Henry → I | 1 | | Henry → like | 1 | | I → am | 2 | | I → do | 1 | | I → like | 3 | | am → | 1 | | am → Henry | 1 | | college → | 3 | | do → like | 1 | | like → Henry | 2 | | like → college | 3 | ### Step 2. Continue Do | Candidate | Count / history count | Probability | | --- | --- | --- | | Henry | 1/3 | 0.333333 | | I | 2/3 | 0.666667 | Highest: I. ### Step 3. Continue I like | Candidate | Count / history count | Probability | | --- | --- | --- | | Henry | 2/5 | 0.400000 | | college | 3/5 | 0.600000 | Highest: college. ### Step 4. Continue I | Candidate | Count / history count | Probability | | --- | --- | --- | | am | 2/6 | 0.333333 | | do | 1/6 | 0.166667 | | like | 3/6 | 0.500000 | Highest: like. ### Final answer Do → I I like → college I → like If an examiner explicitly restricts the answer to a lexical word, exclude END and state that restricted candidate set. Do not silently remove an END tie. **Source / convention note.** Case is retained as printed: Do and do are distinct. The corpus is not silently grammatically corrected. --- ## TEE · Q6b · Five linguistic levels Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f24--q6b/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-f24/q6b Previous examination · complete worked answers **Question. Explain the contribution of phonology, morphology, syntax, semantics and pragmatics to language understanding.** [5 marks] ### Answer **A phoneme** is a sound unit that can distinguish meanings in a language, such as the /p/ and /b/ contrast in “pat” and “bat”. A phoneme is a sound, not necessarily one alphabet letter. **A morpheme** is the smallest meaning-bearing grammatical unit: cats = cat + plural -s; unhappy = un- + happy. **A lexeme** is an abstract vocabulary item grouping its inflected forms: write, writes, wrote and writing belong to WRITE. A **phrase** is a group of words functioning together; a **clause** normally contains a predicate and its associated participants; a **sentence** expresses a complete grammatical unit. | Level | What it analyses | Concrete example | | --- | --- | --- | | Phonetics / phonology | Physical sounds / sound patterns in a language | Distinguish bat from pat | | Morphology / lexical analysis | Word structure and lexical items | un + help + ful → unhelpful | | Syntax | Grammatical arrangement and relationships | The dog chased the cat: subject, verb, object | | Semantics | Meaning of words and composed expressions | The dog is the chaser; the cat is chased | | Discourse | Connections across sentences | “Riya arrived. She smiled.” → She refers to Riya | | Pragmatics | Intended meaning in the situation | “It is cold here” may request closing the window | word forms → grammatical structure → sentence meaning → cross-sentence context → speaker intention ### Three worked interpretations **“The cats are sleeping.”** Morphology identifies cat + s as plural. Syntax connects “cats” to “are sleeping”. Semantics represents animals engaged in sleeping. **“Aman bought a laptop. It was expensive.”** Sentence-level syntax alone does not identify “it”. Discourse connects “it” to the laptop across the sentence boundary. **“Wonderful, another power cut!”** The word wonderful is positive literally, but situational context can indicate sarcasm and frustration. This is a pragmatic interpretation. The analysis levels are conceptual descriptions. Modern systems may learn several jointly rather than executing a compulsory separate module for every level. Morphology = word ke andar; syntax = sentence ka structure; semantics = meaning; discourse = sentences ka link; pragmatics = intention. --- ## TEE · Q6c · NLU and NLG working together Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f24--q6c/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-f24/q6c Previous examination · complete worked answers **Question. Explain how NLU and NLG work together for meaningful human–machine communication.** [5 marks] ### Answer **Natural language understanding (NLU)** maps language into an interpretation useful for a task: intents, entities, semantic relations or another meaning representation. **Natural language generation (NLG)** produces human-language output from data, a representation or another source. “Book a ticket to Pune tomorrow.” → NLU: intent=book; destination=Pune; date=tomorrow → Application: search available journeys → NLG: “Here are the available journeys to Pune.” | Feature | NLU | NLG | | --- | --- | --- | | Direction | Language → interpretation | Data/representation → language | | Main concern | Meaning, intent, ambiguity | Content choice, organization, fluent wording | | Example | Find destination in a request | Write the booking confirmation | | Common failure | Wrong destination or sense | Unclear, unsupported or ungrammatical output | ### NLG stages in a classical pipeline 1. **Content determination:** choose facts to communicate. 2. **Data interpretation:** identify relationships or trends. 3. **Document planning:** organize facts in a useful order. 4. **Sentence aggregation:** combine related facts without repetition. 5. **Lexical and grammatical realization:** choose words and construct grammatical sentences. 6. **Presentation:** format the generated message for its audience. For a weather report, facts {city=Pune, temperature=28°C, rain=yes} can become “Pune will have rain with a temperature of 28°C.” NLU would instead extract facts or intent from an incoming language message. NLU = language se meaning andar. NLG = data se language bahar. --- ## TEE · Q7a · POS, chunks and boundary-aware HMM Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f24--q7a/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-f24/q7a Previous examination · complete worked answers **Question. Define function/content words, chunking/chinking and POS applications. Construct transition and emission tables including start/end for: Book/Verb a/Det flight/Noun; Read/Verb the/Det book/Noun; The/Det book/Noun is/Modal-Verb green/Noun.** [10 marks] ### Answer **Part-of-speech (POS) tagging** assigns a grammatical category to every word token in context, such as noun, verb, adjective or determiner. It is a sequence-labeling task: the input is a sequence of words, and the output is a tag for each word. A tag describes the word's role in this sentence, not its meaning in every possible sentence. | Tag | Role | Example in context | | --- | --- | --- | | NN / NNS | Singular / plural common noun | a book / two books | | NNP | Proper noun | Mary, Mumbai | | VB / VBD / VBZ | Base / past / third-person singular verb | read / read / reads | | JJ | Adjective | the red book | | RB | Adverb | runs quickly | | DT | Determiner | the, a, this | | IN | Preposition or subordinating conjunction | on the table | | PRP | Personal pronoun | she, they | | MD | Modal auxiliary | can, will, should | **Open classes** readily accept new content words: nouns, verbs, adjectives and adverbs. Examples include chatbot and livestream. **Closed classes** contain comparatively stable grammatical/function words: determiners, pronouns, prepositions and conjunctions. These are tendencies, not a claim that language never changes. Input: Book that flight → Context: an imperative request → Output: Book/VB that/DT flight/NN Compare “hand me that **book**”: book is a noun here. Similarly, “I **can** swim” has can/MD, while “the **can** is empty” has can/NN. The spelling stays the same; neighboring words and sentence structure change the correct tag. This is **POS ambiguity**. | Sentence | Tagged interpretation | Reason | | --- | --- | --- | | They watch birds. | They/PRP watch/VBP birds/NNS | watch is the action. | | The watch stopped. | The/DT watch/NN stopped/VBD | watch is the object that stopped. | | She has a light bag. | She/PRP has/VBZ a/DT light/JJ bag/NN | light describes bag. | | Light the lamp. | Light/VB the/DT lamp/NN | Light is an imperative verb. | POS tags support parsing, named entity recognition, question answering, sentiment analysis and word-sense disambiguation. They do not by themselves identify every entity or fully resolve meaning. **Yaad rakho:** Word ka naam same ho sakta hai; sentence mein uska kaam tag decide karta hai. **Chunking**, also called shallow parsing, groups adjacent tagged words into meaningful non-overlapping units, commonly noun phrases. It usually identifies local phrase boundaries without producing the complete hierarchical structure of a sentence. **Chinking** removes specified tokens from an existing candidate chunk, splitting or trimming it. The/DT small/JJ cat/NN sat/VBD on/IN the/DT mat/NN → NP: [The small cat] → Outside: sat on → NP: [the mat] A POS tagger labels individual words. A chunker groups words into phrases. A full parser shows how phrases relate recursively to form the entire sentence. These are related tasks with different outputs. | Operation | Input | Output | | --- | --- | --- | | POS tagging | The small cat | The/DT small/JJ cat/NN | | NP chunking | Tagged tokens | [NP The small cat] | | Chinking | A broad candidate chunk | Remove verbs/prepositions, keep noun-phrase segments | | Full parsing | Sentence + grammar | A nested tree with sentence, noun phrase and verb phrase nodes | **Rule-based chunking** uses tag patterns. Statistical or learned chunkers infer boundaries from labeled examples. A useful NP rule is an optional determiner, followed by zero or more adjectives, followed by a noun. In NLTK-style notation: `NP: {
?*}`. Here ? means zero or one occurrence; \* means zero or more; + means one or more when used. The braces mean “form a chunk around the matched sequence”. **Question. Apply the NP rule optional DT, zero or more JJ, then one NN to “The/DT quick/JJ brown/JJ fox/NN jumps/VBZ over/IN the/DT lazy/JJ dog/NN”.** #### Given Tagged input is supplied. Rule: DT? JJ\* NN. #### To find All noun-phrase chunks. #### Formula and why it applies `NP → DT? JJ* NN` The rule groups a noun with its preceding determiner and adjectives; a verb or preposition does not match this NP pattern. #### Solution ##### Step 1. Start at The The matches optional DT; quick and brown both match JJ\*; fox matches NN. ##### Step 2. Form the first NP **[NP The quick brown fox]** ##### Step 3. Skip nonmatching tokens jumps/VBZ and over/IN do not satisfy the NP pattern. ##### Step 4. Match the second NP the is DT, lazy is JJ, dog is NN: **[NP the lazy dog]**. **Final answer.** [NP The quick brown fox] jumps over [NP the lazy dog]. **Question. Using NP → DT? JJ\* NN, identify the chunks in “bright/JJ stars/NN shine/VBP and/CC a/DT moon/NN glows/VBZ”. Use the supplied tags exactly.** #### Given Input tags label stars as NN for this exercise. Rule requires optional DT, zero or more JJ, then NN. #### To find The matched NP spans. #### Formula and why it applies `NP → DT? JJ* NN` The optional determiner may be absent and the adjective repetition may be empty. #### Solution ##### Step 1. Match bright stars No determiner is present, which DT? permits. bright matches JJ; stars matches NN. ##### Step 2. Inspect shine and and VBP and CC fall outside the NP rule. ##### Step 3. Match a moon a matches DT; no adjective is present; moon matches NN. **Final answer.** Chunks are [NP bright stars] and [NP a moon]. With ordinary Penn tagging, stars is usually NNS; broaden the noun pattern to include NNS when that tag is used. **Question. Begin with one candidate chunk containing “The/DT cat/NN sat/VBD on/IN the/DT mat/NN”. Chink out verbs and prepositions. Show the remaining chunks.** #### Given Initial candidate: [The cat sat on the mat]. Remove tags VBD and IN. #### To find The chunk segments after exclusion. #### Formula and why it applies Chinking removes matched tokens from the existing chunk. Removing the middle verb/preposition region divides the original candidate into two remaining contiguous spans. #### Solution ##### Step 1. Locate the tokens to remove sat is VBD and on is IN. ##### Step 2. Remove those tokens from chunk membership The remaining token groups are “The cat” and “the mat”. sat and on still exist in the sentence; they are simply outside these chunks. ##### Step 3. Write the final chunked sentence **[The cat]** sat on **[the mat]**. **Final answer.** Two chunks remain: The cat; the mat. **Yaad rakho:** Chunking = andar group karo; chinking = unwanted tokens ko group se bahar karo. ### Exact HMM table ### Given | Sentence | Supplied tags | | --- | --- | | Book a flight | Verb Det Noun | | Read the book | Verb Det Noun | | The book is green | Det Noun Modal-Verb Noun | ### To find Transition and emission probability tables including START and END. ### Formula: which one and why $$ a_{ij}=\frac{c(t_i,t_j)}{c(t_i)},\qquad b_i(w)=\frac{c(t_i,w)}{c(t_i)} $$ For START divide by sentence count. END has no outgoing or emission distribution. Keep the supplied tags even when their linguistic choice is unusual, and retain case-sensitive Book/book and The/the as printed. ### Step 1. Add boundary tags to every sentence START Verb Det Noun END START Verb Det Noun END START Det Noun Modal-Verb Noun END ### Step 2. Count transitions | From → to | Count | | --- | --- | | START → Verb | 2 | | Verb → Det | 2 | | Det → Noun | 3 | | Noun → END | 3 | | START → Det | 1 | | Noun → Modal-Verb | 1 | | Modal-Verb → Noun | 1 | ### Step 3. Divide by each source tag count | From | Denominator | Verb | Det | Noun | Modal-Verb | END | | --- | --- | --- | --- | --- | --- | --- | | START | 3 | 2/3 | 1/3 | 0 | 0 | 0 | | Verb | 2 | 0 | 1 | 0 | 0 | 0 | | Det | 3 | 0 | 0 | 1 | 0 | 0 | | Noun | 4 | 0 | 0 | 0 | 1/4 | 3/4 | | Modal-Verb | 1 | 0 | 0 | 1 | 0 | 0 | ### Step 4. Count and normalize emissions | Tag | Count | Book | a | flight | Read | the | book | The | is | green | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | Verb | 2 | 1/2 | 0 | 0 | 1/2 | 0 | 0 | 0 | 0 | 0 | | Det | 3 | 0 | 1/3 | 0 | 0 | 1/3 | 0 | 1/3 | 0 | 0 | | Noun | 4 | 0 | 0 | 1/4 | 0 | 0 | 1/2 | 0 | 0 | 1/4 | | Modal-Verb | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | ### Final answer Every nonterminal transition row sums to1 when END is included; every emission row sums to1. Boundary symbols do not emit lexical words. POS applications include syntax parsing, disambiguation, lemmatization, chunking and extraction. For this numerical retain printed green=Noun and is=Modal-Verb even though ordinary linguistic tagging would differ; the task is estimation from supplied labels. **Source / convention note.** Printed tags are used as training data; START and END are explicitly required and included. --- ## TEE · Q7b · Co-occurrence, dot products and plot Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f24--q7b/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-f24/q7b Previous examination · complete worked answers **Question. Define target words, context window, context words and co-occurrence matrix. For the printed boy/ball/dog/sun corpus below, use one whole sentence as the window; contexts={kicked,field,tree,ran}. Find boy·dog and boy·sun, plot on kicked/field dimensions and comment.** [10 marks] ### Answer **Target** is the word whose row we represent. **Context** is an observed neighbouring word whose column we count. A **window** defines which tokens count as neighbours; here all other tokens in the same sentence. A **co-occurrence matrix** records those counts. ### Given The boy kicked the ball into the field. The dog barked at the boy. The sun was shining over the field. The boy ran after the dog. The ball rolled to the tree. ### To find Matrix, two dot products and the requested 2D coordinates. ### Formula: which one and why $$ u\cdot v=\sum_i u_iv_i $$ Use lowercase token-pair counts within each sentence; do not cross sentence boundaries. ### Step 1. List contributions and build the matrix | Sentence | Target occurrences | Context occurrences | | --- | --- | --- | | 1 | boy, ball | kicked, field | | 2 | dog, boy | | | 3 | sun | field | | 4 | boy, dog | ran | | 5 | ball | tree | | Target / context | kicked | field | tree | ran | | --- | --- | --- | --- | --- | | boy | 1 | 1 | 0 | 1 | | ball | 1 | 1 | 1 | 0 | | dog | 0 | 0 | 0 | 1 | | sun | 0 | 1 | 0 | 0 | ### Step 2. Dot boy and dog [1, 1, 0, 1] · [0, 0, 0, 1] = 1×0 + 1×0 + 0×0 + 1×1 = 1 ### Step 3. Dot boy and sun [1, 1, 0, 1] · [0, 1, 0, 0] = 1×0 + 1×1 + 0×0 + 1×0 = 1 ### Step 4. Plot coordinates: kicked on x, field on y | Target | (x,y) | | --- | --- | | boy | (1, 1) | | ball | (1, 1) | | dog | (0, 0) | | sun | (0, 1) | kickedfielddog (0,0)sun (0,1)boy, ball (1,1) Two coordinates omit tree and ran, so boy and ball overlap here despite different full vectors. ### Final answer **boy·dog=1; boy·sun=1.** Both share one counted context contribution. Dot-product similarity reflects this corpus and magnitude, not a claim that boy is equally semantically related to dog and sun in general. --- ## TEE · Q1a · Anaphora versus cataphora Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r24--q1a/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-r24/q1a Previous examination · complete worked answers **Question. Compare anaphoric and cataphoric reference with examples.** [5 marks] ### Answer **Pragmatic analysis** interprets an utterance using its situation and the speaker’s intended meaning. **Discourse analysis** studies how utterances connect across a text or conversation. Syntax and literal word meanings alone cannot settle every interpretation. | Text | Literal content | Contextual interpretation | | --- | --- | --- | | “Can you close the window?” | A question about ability. | Usually a polite request. | | “Meera bought a laptop. It was expensive.” | Two sentences. | “It” connects the second sentence to the laptop. | | “Meet me here tomorrow.” | An instruction. | Me, here and tomorrow depend on speaker, place and date. | ### Reference, mention and antecedent A **referring expression** points to an entity; a **mention** is its occurrence in the text. Its **referent** is the entity being discussed. An **antecedent** is a preceding expression used to interpret a later referring expression. Reference resolution identifies the intended referent. Coreference resolution groups mentions that refer to the same entity. “Meera”name mention → Person entitysame referent → “she” / “the engineer”other mentions | Concept | Definition | Example | | --- | --- | --- | | Anaphora | Reference back to something introduced earlier. | “Riya arrived. She sat down.” She → Riya. | | Cataphora | Reference forward to a later expression. | “Before she spoke, Riya checked the microphone.” she → Riya. | | Exophora | Reference to the situation outside the text. | “Put it there”, said while pointing. | | Coreference | Two or more mentions refer to the same entity. | “Dr Rao”, “the lecturer”, “she”. | | Pronoun resolution | Find the referent of a pronoun. | Resolve “it” in “The laptop stopped because it overheated.” | Coreference is broader than pronoun resolution: repeated names and descriptions also form a chain. Anaphora describes direction/dependence; coreference describes identity. In “I bought a car. The engine is noisy”, engine is related to the car but is not the same entity. **Yaad rakho:** Reference = kiski baat? Coreference = alag mentions, same entity. Pronoun resolution = he/she/it kis se linked? ### Cohesion, coherence and discourse markers **Cohesion** is the explicit linguistic linking of a text through reference, repeated or related words, conjunctions and other devices. In “Rita bought a book. She read it”, she and it connect the second sentence to Rita and the book. **Coherence** is the conceptual unity that makes a discourse meaningful. “The road flooded, so the match was cancelled” presents a plausible cause and consequence. A reader relates the propositions using context and knowledge. **Discourse markers** signal relationships or organize speech: however signals contrast; therefore signals a result; first and next signal sequence; for example introduces an illustration. These concepts differ: “I like apples. Therefore the moon is square” has a linking word but no sensible inference. A connector alone does not guarantee coherence. **Yaad rakho:** Cohesion = words se link; coherence = meaning se link; marker = relation ka signal. --- ## TEE · Q1b · Five corpora Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r24--q1b/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-r24/q1b Previous examination · complete worked answers **Question. Define corpora and discuss any five corpora used in NLP.** [5 marks] ### Answer **A corpus is a collection of language data assembled for study or computational use.** Its plural is corpora. A corpus can contain books, news, reviews or conversations. It need not be labelled. **Annotation** adds linguistic information such as POS tags, entities, parse structures or sense labels. | Structure | Meaning | Use | | --- | --- | --- | | Plain | Collection without category labels | Word counts, language modelling | | Categorized | Documents grouped by genre/topic | Classification | | Overlapping categories | One document has several categories | Multi-topic news | | Temporal | Documents indexed by time | Language change | | Annotated | Linguistic labels attached | Tagging, parsing, WSD | | Parallel | Aligned texts in different languages | Translation | A **tagset** is the allowed label inventory. A **treebank** is a corpus annotated with syntax trees. **Metadata** records source, date, genre, language, annotation method and limitations. A common benchmark supports comparison under shared data and evaluation conditions. **Raw example:** “Riya works in Pune.” **POS annotation:** Riya/PROPN works/VERB in/ADP Pune/PROPN. **NER annotation:** Riya/PERSON, Pune/LOCATION. Each annotation answers a different task. A good corpus is relevant to its domain, sufficiently representative, consistently annotated and appropriately split. More data alone do not repair duplicated examples, incorrect labels or a mismatch between training and real use. Corpus=text collection. Annotation=text ke saath labels. Metadata=collection ki information. | Resource | Contents | Typical use | | --- | --- | --- | | Gutenberg selection | Literary/book texts | Long-form language and vocabulary | | Web text | Informal domain-specific text | Study usage beyond edited books | | Chat corpus | Conversational posts | Dialogue, informal forms | | Brown | English texts across genres | Genre comparison | | Reuters | News with topic labels | Text classification; overlapping topics | | Inaugural | Speeches indexed by time | Vocabulary change across years | | Wordlist | Words/names/stopwords | Lexical filtering | | Pronouncing dictionary | Words and phonetic forms | Speech, stress, rhyme | **Gutenberg versus web/chat:** book language does not represent every spelling or conversational pattern in messages. **Brown versus Reuters:** Brown emphasizes genres; Reuters news can have overlapping topic labels. **Inaugural addresses:** dates provide a time dimension. Corpus sizes depend on version, subset and tokenization. Explain their purpose and structure rather than assuming a historical count is a permanent property. Gutenberg=books; web/chat=informal; Brown=genres; Reuters=news topics; Inaugural=time. --- ## TEE · Q1c · Linguistic foundations Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r24--q1c/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-r24/q1c Previous examination · complete worked answers **Question. Define linguistic foundation and explain its importance in NLP.** [5 marks] ### Answer **A phoneme** is a sound unit that can distinguish meanings in a language, such as the /p/ and /b/ contrast in “pat” and “bat”. A phoneme is a sound, not necessarily one alphabet letter. **A morpheme** is the smallest meaning-bearing grammatical unit: cats = cat + plural -s; unhappy = un- + happy. **A lexeme** is an abstract vocabulary item grouping its inflected forms: write, writes, wrote and writing belong to WRITE. A **phrase** is a group of words functioning together; a **clause** normally contains a predicate and its associated participants; a **sentence** expresses a complete grammatical unit. | Level | What it analyses | Concrete example | | --- | --- | --- | | Phonetics / phonology | Physical sounds / sound patterns in a language | Distinguish bat from pat | | Morphology / lexical analysis | Word structure and lexical items | un + help + ful → unhelpful | | Syntax | Grammatical arrangement and relationships | The dog chased the cat: subject, verb, object | | Semantics | Meaning of words and composed expressions | The dog is the chaser; the cat is chased | | Discourse | Connections across sentences | “Riya arrived. She smiled.” → She refers to Riya | | Pragmatics | Intended meaning in the situation | “It is cold here” may request closing the window | word forms → grammatical structure → sentence meaning → cross-sentence context → speaker intention ### Three worked interpretations **“The cats are sleeping.”** Morphology identifies cat + s as plural. Syntax connects “cats” to “are sleeping”. Semantics represents animals engaged in sleeping. **“Aman bought a laptop. It was expensive.”** Sentence-level syntax alone does not identify “it”. Discourse connects “it” to the laptop across the sentence boundary. **“Wonderful, another power cut!”** The word wonderful is positive literally, but situational context can indicate sarcasm and frustration. This is a pragmatic interpretation. The analysis levels are conceptual descriptions. Modern systems may learn several jointly rather than executing a compulsory separate module for every level. Morphology = word ke andar; syntax = sentence ka structure; semantics = meaning; discourse = sentences ka link; pragmatics = intention. --- ## TEE · Q1d · Five regex operators Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r24--q1d/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-r24/q1d Previous examination · complete worked answers **Question. For word-pattern detection, explain: ^abc, abc$, [A-Z0-9], ed|ing|s, [abc].** [5 marks] ### Answer | Pattern | Exact meaning | Positive example | Important distinction | | --- | --- | --- | --- | | ^abc | abc at start of the input/line under chosen mode | abcdef | Not abc anywhere | | abc$ | abc at end | xxabc | Not starts with abc | | [A-Z0-9] | One uppercase ASCII letter or digit | Q or 7 | One character, not the whole string | | ed|ing|s | One of the alternatives ed, ing or s | matched ed | Without anchors this is not necessarily a suffix | | [abc] | One a, b or c | b | Not the three-character string abc | Use full-string matching when the entire string must satisfy a pattern; a search may find only a substring. --- ## TEE · Q2a · Wordlist puzzle and lexical resource Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r24--q2a/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-r24/q2a Previous examination · complete worked answers **Question. Explain lexical resources. Solve the wordlist puzzle with letters E,G,I / V,R,V / O,N,L. Words must have at least four letters, contain centre R, use each tile at most once, and include at least one nine-letter word.** [10 marks] ### Answer ### Given | | Column1 | Column2 | Column3 | | --- | --- | --- | --- | | Row1 | E | G | I | | Row2 | V | R | V | | Row3 | O | N | L | ### To find Find valid words and explain how a wordlist corpus supplies the candidates. ### Formula: which one and why A lexical resource stores words and linguistic information. Filter an explicitly chosen dictionary by length, mandatory letter and a multiset of tile counts; duplicate V tiles permit two Vs. ### Step 1. Count available tiles E1,G1,I1,V2,R1,O1,N1,L1. ### Step 2. Apply all predicates For each lowercase dictionary word w: length(w)≥4; r occurs; for every letter c, count\_w(c)≤count\_tiles(c). Deduplicate dictionary entries. ### Step 3. Verify examples | Word | Reason | | --- | --- | | revolving | 9 letters: r e v o l v i n g; uses both V tiles and all others once | | roving | 6 letters, includes R, within tile counts | | lover | 5 letters, includes R | | giver | 5 letters, includes R | | groin | 5 letters, includes R | | iron | 4 letters, includes R | | ring | 4 letters, includes R | | rove | 4 letters, includes R | ### Step 4. Explain the requested count The count is the number of entries surviving those checks in the chosen wordlist. No wordlist/version is supplied, so a unique total cannot be stated. The examples above are verified examples, not an exhaustive dictionary. ### Final answer **REVOLVING** is a valid nine-letter answer. Enumerate all other valid entries using the declared wordlist. **Source / convention note.** The dictionary is unspecified; an exact number of valid words would be fabricated. --- ## TEE · Q2b · NLP applications and limitations Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r24--q2b/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-r24/q2b Previous examination · complete worked answers **Question. List NLP applications and explain limitations of NLP.** [10 marks] ### Answer **Natural language processing (NLP) is the field concerned with computational methods for analysing, interpreting and generating human language.** Natural languages such as English, Hindi and Marathi evolved through human use; their meanings are not specified by the rigid rules of a programming language. An NLP system may receive text, speech converted to text, or another linguistic representation. Its output depends on the task: a category, a tag for every word, a translated sentence, a ranked document list or a generated answer. It does not always “understand” in the human sense; its performance must be evaluated for the specific task. | Task | Input | Output | | --- | --- | --- | | Sentiment analysis | “The service was excellent.” | Positive sentiment | | Text classification | An email | Spam / not spam | | POS tagging | “Birds fly.” | Birds/NOUN fly/VERB | | Named entity recognition | “Asha joined Google in Mumbai.” | Asha: person; Google: organisation; Mumbai: location | | Machine translation | A Hindi sentence | A corresponding English sentence | | Information retrieval | A search query | Ranked relevant documents | | Question answering | Question plus knowledge/context | An answer | | Summarization | A long article | A shorter text preserving key information | ### How NLP relates to AI, ML and DL **AI** is the broader study of systems performing tasks associated with intelligent behaviour. **Machine learning** learns patterns from data. **Deep learning** uses multiple trainable neural layers. NLP is an application/research field that can use hand-written rules, statistical ML or deep learning. Therefore, not every NLP system is a neural network. Linguistics supplies knowledge about language structure. Statistics supplies uncertainty models. Computer science supplies algorithms, storage and efficient implementation. A practical NLP system combines these ideas. NLP = human language ko computational task mein badlo: label, meaning, search ya generated sentence. **Ambiguity** means an expression supports more than one interpretation. Good exam answers name the ambiguity and explain both readings. | Challenge | Example | Why a system can fail | | --- | --- | --- | | Lexical/sense ambiguity | bank: finance or river edge | One written word has several senses | | Syntactic ambiguity | I saw the man with a telescope. | The telescope may modify saw or the man | | Reference ambiguity | Riya told Neha that she won. | She may refer to either person without more context | | Sarcasm | Great, another delay. | Literal positive word conveys frustration | | Negation | The film was not good. | Dropping not reverses meaning | | Domain shift | “Positive” in a medical report | Meaning/use differs by domain | | Code mixing | Kal meeting reschedule karna. | Several languages and spelling conventions | | Noisy text | tmrw, repeated letters, typos | Vocabulary/token boundaries become uncertain | | Low-resource setting | Few labelled examples in a language | Models lack representative training evidence | | Bias and privacy | Skewed data or personal details | Errors and harms can persist in deployed systems | **Homophones** sound alike but may differ in spelling (pair/pear). **Homographs** share spelling but can differ in meaning or pronunciation (lead the team / lead metal). These are different dimensions from syntactic ambiguity. Long-distance dependencies, cultural knowledge, idioms, morphology and multilingual writing systems also complicate processing. For “kick the bucket”, literal word-by-word interpretation can miss the idiomatic meaning. For “The book that the students recommended is useful”, the verb is agrees with book across an intervening phrase. Exam mein challenge ka naam + ek sentence + dono meanings/actual failure likho. --- ## TEE · Q3a · Top-down restaurant parse Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r24--q3a/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-r24/q3a Previous examination · complete worked answers **Question. Draw a top-down parse of “A restaurant serves dosas” using the printed grammar: S→NP VP | Aux NP VP | VP; NP→Pronoun | Proper-Noun | Det Nominal; Nominal→Noun | Nominal Noun | Nominal PP; VP→Verb | Verb NP | VP PP; PP→Prep NP. Lexicon: Det=A; Noun=restaurant,dosas; Verb=serves.** [10 marks] ### Answer **As printed, this grammar cannot derive the complete sentence.** The object dosas is a Noun, but NP has no rule for a bare Nominal/Noun. NP→Det Nominal would require a determiner before dosas that is absent. ### Clearly stated minimal repair Add NP→Nominal. The repaired top-down derivation is: | Step | Sentential form | | --- | --- | | 1 | S | | 2 | NP VP | | 3 | Det Nominal VP | | 4 | A Noun VP | | 5 | A restaurant Verb NP | | 6 | A restaurant serves Nominal | | 7 | A restaurant serves Noun | | 8 | A restaurant serves dosas | SNPDetANominalNounrestaurantVPVerbservesNPNominalNoundosas [S [NP [Det A] [Nominal [Noun restaurant]]] [VP [Verb serves] [NP [Nominal [Noun dosas]]]]] This tree uses the added rule only for the object. Do not silently pretend it is derivable under the original rule set. Left-recursive Nominal/VP alternatives require a terminating parser strategy; the displayed successful derivation chooses the needed finite branches. **Source / convention note.** Original CFG is incomplete for the given bare-plural object. Exact failure and explicit repaired derivation are both shown. --- ## TEE · Q3b · Preprocessing pipeline Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r24--q3b/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-r24/q3b Previous examination · complete worked answers **Question. Explain basic preprocessing steps on a blob of text with an example and pipeline diagram.** [10 marks] ### Answer **An NLP pipeline is an organized sequence of operations that transforms raw language into a task output and checks whether that output is useful.** Not every task needs every cleaning operation. Data acquisition → Preprocessing → Feature representation → Model → Evaluation → Deployment and monitoring 1. **Data acquisition:** collect relevant text and, when needed, labels. 2. **Cleaning and preprocessing:** handle markup, tokenize, normalize and decide which information to retain. 3. **Feature engineering:** represent text numerically using BoW, TF–IDF or embeddings. 4. **Modelling:** apply rules or train a statistical/neural model. 5. **Evaluation:** test on held-out data using task-appropriate measures. 6. **Deployment:** make the system available, log suitable feedback and monitor errors or data changes. ### Concrete spam-filter example Collect labelled emails → separate train/test data → tokenize text while preserving meaningful terms → fit vocabulary and IDF on training data → train classifier → evaluate spam detection on the held-out set → deploy an email service filter → monitor legitimate emails wrongly blocked. **Intrinsic evaluation** measures model/task quality, such as precision, recall or tagging accuracy. **Extrinsic evaluation** measures effect in the wider application, such as whether users find answers faster. A higher internal metric does not automatically guarantee a better user outcome. **Data leakage:** learning vocabulary statistics, labels or preprocessing decisions from the test set can make reported performance too optimistic. Fit learned transformations on training data, then apply them to validation/test data. Collect → clean → numbers → model → test → use. Test data ko training ki information mat banao. **Normalization reduces superficial variation so comparable forms can be processed consistently.** Common operations include lowercasing, Unicode normalization, appropriate spelling correction, contraction handling, and optional stemming or lemmatization. **Stopwords** are frequent words selected for removal for a particular task, such as the or a. Removing them may reduce feature size, but removing not, no or nor can destroy sentiment meaning. Punctuation can convey sentence boundaries, emotion or mathematical structure. Lowercasing can erase differences such as US versus us. **Question. Preprocess “The CATS are not sleeping!!!” by lowercasing, removing punctuation, removing only stopwords {the,are}, and lemmatizing plural cats→cat and sleeping→sleep.** #### Given Operations and stopword set are fixed. Do not remove not. #### To find The final token list. #### Formula and why it applies Apply the requested operations in order, keeping a visible intermediate result after each transformation. #### Step 1. Lowercase the cats are not sleeping!!! #### Step 2. Tokenize and remove punctuation [the,cats,are,not,sleeping] #### Step 3. Remove specified stopwords Remove the and are → [cats,not,sleeping]. #### Step 4. Lemmatize cats→cat; sleeping→sleep → [cat,not,sleep]. **Answer.** [cat,not,sleep]. The negative meaning is retained. ### Task decides preprocessing For a simple keyword-retrieval model, removing selected function words can help. For machine translation, question answering, grammar analysis or transformer input, aggressive removal may discard necessary syntax or violate the model's expected input format. Emoji can carry sentiment; convert or retain them when relevant instead of always deleting them. Clean karna = useful signal bachake noise hatao. Har word/punctuation delete karna cleaning nahi. **Stemming** applies string-based rules, often stripping suffixes, to reduce related forms to a stem. The result need not be a valid dictionary word. **Lemmatization** uses vocabulary and morphological analysis, often with POS information, to return a dictionary lemma. | Criterion | Stemming | Lemmatization | | --- | --- | --- | | Basis | Surface form/suffix rules | Lexicon and morphological/POS analysis | | Output | May be nonword, e.g. studi | Dictionary form, e.g. study | | Context/POS | Usually little or none | Often important | | Irregular forms | May miss went→go | Can map went→go with verb analysis | | Cost | Generally simpler | Usually more linguistic processing | Porter applies a sequence of suffix-removal rules subject to conditions; Snowball provides related language-specific stemming algorithms; Lancaster is often more aggressive; regex stemming uses explicit patterns; Lovins is another established stemming approach. Exact output depends on the algorithm and implementation. ### Three worked reductions 1. **studies, studying:** an English stemmer may yield studi/study depending on the rule sequence; verb lemmatization aims at study. State the exact algorithm if exact outputs are asked. 2. **better:** adjective-aware lemmatization can map it to good; naïve suffix removal cannot infer that irregular relation. 3. **saw:** as a past-tense verb it can become see; as a cutting-tool noun it remains saw. POS and context matter. **Over-stemming** merges forms that should remain distinct; **under-stemming** fails to merge related forms. Normalization can increase recall in search while losing useful distinctions, so it is a task-dependent choice. Stem = rules se cut; lemma = dictionary form, meaning/POS ka dhyan. --- ## TEE · Q4a · Embeddings and Skip-gram Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r24--q4a/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-r24/q4a Previous examination · complete worked answers **Question. Explain word embeddings and the steps in Skip-gram with a suitable diagram.** [10 marks] ### Answer **A sparse representation** has many zero entries and often one coordinate per vocabulary item. BoW and TF–IDF are common sparse document representations. **A dense embedding** has comparatively few learned real-valued coordinates; one word's properties are spread across many coordinates. | Representation | What coordinates mean | Strength | Limitation | | --- | --- | --- | --- | | One-hot | One identity position per vocabulary word | Unambiguous ID | Different words are orthogonal; no learned similarity | | Count-based context vector | Context-word co-occurrence counts | Transparent corpus evidence | Large, sparse; frequent words may dominate | | TF–IDF document vector | Weighted vocabulary terms | Useful retrieval weighting | Word order and contextual meaning are limited | | Dense word embedding | Learned latent features | Compact semantic/syntactic relationships | Coordinates are not generally human-labelled features | **Distributional** describes learning from patterns of linguistic context. **Distributed** describes representing an item across multiple coordinates rather than a single active identity bit. These are related but different ideas. A one-hot ID alone does not learn distributional similarity. ### Model design choices Choose contexts (document, sentence, fixed window), weighting (raw counts, normalization, TF–IDF-like weighting or association scores), dimensionality reduction or learned embeddings, and a similarity measure. Cosine compares direction and is less affected by length than a raw dot product; Euclidean distance also reflects magnitude. Sparse = bahut zero. Dense = chhoti learned number-list. Similar context se useful similarity seekhte hain. **Word2Vec learns dense word embeddings by predicting words from nearby context.** Its two common architectures are CBOW and skip-gram. A vocabulary ID or one-hot vector is the input representation; the learned embedding table converts that identity into a short trainable vector. | Architecture | Input | Prediction | Memory cue | | --- | --- | --- | --- | | CBOW | Surrounding context words | Centre/target word | Context → centre | | Skip-gram | Centre/target word | Surrounding words | Centre → context | CBOW: the + sat → Average context embeddings → Predict: cat Skip-gram: cat → Embedding of cat → Predict: the and sat For vocabulary size $$ V $$ and embedding dimension $$ d $$ , an input one-hot row is $$ 1\times V $$ ; the input embedding matrix is $$ V\times d $$ ; the hidden row is $$ 1\times d $$ ; the output matrix is $$ d\times V $$ ; output scores are $$ 1\times V $$ . Multiplying a one-hot row by the embedding matrix simply selects one row. $$ P(w_o\mid w_i)=\frac{\exp(u_{w_o}^{T}v_{w_i})}{\sum_{w\in V}\exp(u_w^{T}v_{w_i})}. $$ $$ v_{w_i} $$ is the input word embedding, $$ u_w $$ an output-context embedding, and the dot product is a compatibility score. Softmax turns all scores into probabilities that sum to one. Training increases the probability of observed context words. Negative sampling or hierarchical softmax can reduce computation in larger vocabularies; use the algorithm specified by the question. **Question. For the sentence “the cat sat on mat”, generate CBOW and skip-gram training data for centre word sat with a window of one token on each side.** #### Given Tokens indexed 1 to 5: the, cat, sat, on, mat. Centre = sat. #### To find The CBOW example and the skip-gram pairs. #### Formula and why it applies CBOW predicts the centre from its neighbours; skip-gram predicts each neighbour from the centre. #### Step 1. Identify neighbours Left neighbour = cat; right neighbour = on. “the” and “mat” are outside this window. #### Step 2. CBOW example Input [cat,on] → target sat. #### Step 3. Skip-gram examples (sat,cat) and (sat,on), with first item as input and second as predicted context. **Answer.** CBOW: [cat,on]→sat. Skip-gram: sat→cat and sat→on. **Question. The embeddings of context words cat and on are [1,2] and [3,0]. Compute the CBOW hidden vector using mean aggregation. Output vectors for [sat,ran] are [1,0] and [0,1]. Compute softmax probabilities.** #### Given Context embeddings [1,2], [3,0]; mean aggregation; two candidate output vectors. #### To find The hidden vector, logits and probabilities. #### Formula and why it applies Mean context vector $$ h=(v_1+v_2)/2 $$ . Logit $$ z_w=u_w^Th $$ . Softmax $$ P(w)=e^{z_w}/\sum_j e^{z_j} $$ . #### Step 1. Add coordinates $$ [1+3,\,2+0]=[4,2] $$ . #### Step 2. Divide by number of words $$ h=[4/2,2/2]=[2,1] $$ . #### Step 3. Compute logits $$ z_{sat}=1(2)+0(1)=2 $$ . $$ z_{ran}=0(2)+1(1)=1 $$ . #### Step 4. Exponentials $$ e^2\approx7.389056 $$ , $$ e^1\approx2.718282 $$ ; sum $$ \approx10.107338 $$ . #### Step 5. Divide $$ P(sat)\approx0.731059 $$ , $$ P(ran)\approx0.268941 $$ . **Answer.** h=[2,1]; sat is more probable (0.731059). **Question. A skip-gram centre embedding is v=[1,−1]. Output embeddings for [eat,sleep,drive] are [1,1], [0,1], [1,0]. Calculate the predicted context probabilities.** #### Given One centre vector and three output vectors; full softmax over these three words. #### To find Three logits and three probabilities. #### Formula and why it applies $$ z_j=u_j^Tv $$ , then $$ P(j)=e^{z_j}/\sum_k e^{z_k} $$ . #### Step 1. Eat dot product $$ 1(1)+1(-1)=0 $$ . #### Step 2. Sleep and drive dot products $$ z_{sleep}=0(1)+1(-1)=-1 $$ . $$ z_{drive}=1(1)+0(-1)=1 $$ . #### Step 3. Exponentiate and sum $$ e^0=1 $$ , $$ e^{-1}\approx0.367879 $$ , $$ e^1\approx2.718282 $$ . Sum $$ \approx4.086161 $$ . #### Step 4. Normalize $$ P(eat)\approx0.244728 $$ ; $$ P(sleep)\approx0.090031 $$ ; $$ P(drive)\approx0.665241 $$ . **Answer.** drive has the highest probability in this toy learned model. **Limitations:** a standard static Word2Vec vector merges a word's senses; it has limited representation of subword morphology and out-of-vocabulary words; its local prediction objective does not directly fit a full global co-occurrence table. Similarity is learned from data and is not a guarantee of factual or causal understanding. CBOW = neighbours se missing centre. Skip-gram = centre se neighbours. --- ## TEE · Q4b · POS, NER, chunking and TTR Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r24--q4b/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-r24/q4b Previous examination · complete worked answers **Question. Define POS tagging, entity recognition, chunking and type-token ratio with examples.** [10 marks] ### Answer **Part-of-speech (POS) tagging** assigns a grammatical category to every word token in context, such as noun, verb, adjective or determiner. It is a sequence-labeling task: the input is a sequence of words, and the output is a tag for each word. A tag describes the word's role in this sentence, not its meaning in every possible sentence. | Tag | Role | Example in context | | --- | --- | --- | | NN / NNS | Singular / plural common noun | a book / two books | | NNP | Proper noun | Mary, Mumbai | | VB / VBD / VBZ | Base / past / third-person singular verb | read / read / reads | | JJ | Adjective | the red book | | RB | Adverb | runs quickly | | DT | Determiner | the, a, this | | IN | Preposition or subordinating conjunction | on the table | | PRP | Personal pronoun | she, they | | MD | Modal auxiliary | can, will, should | **Open classes** readily accept new content words: nouns, verbs, adjectives and adverbs. Examples include chatbot and livestream. **Closed classes** contain comparatively stable grammatical/function words: determiners, pronouns, prepositions and conjunctions. These are tendencies, not a claim that language never changes. Input: Book that flight → Context: an imperative request → Output: Book/VB that/DT flight/NN Compare “hand me that **book**”: book is a noun here. Similarly, “I **can** swim” has can/MD, while “the **can** is empty” has can/NN. The spelling stays the same; neighboring words and sentence structure change the correct tag. This is **POS ambiguity**. | Sentence | Tagged interpretation | Reason | | --- | --- | --- | | They watch birds. | They/PRP watch/VBP birds/NNS | watch is the action. | | The watch stopped. | The/DT watch/NN stopped/VBD | watch is the object that stopped. | | She has a light bag. | She/PRP has/VBZ a/DT light/JJ bag/NN | light describes bag. | | Light the lamp. | Light/VB the/DT lamp/NN | Light is an imperative verb. | POS tags support parsing, named entity recognition, question answering, sentiment analysis and word-sense disambiguation. They do not by themselves identify every entity or fully resolve meaning. **Yaad rakho:** Word ka naam same ho sakta hai; sentence mein uska kaam tag decide karta hai. **Named Entity Recognition (NER)** identifies text spans referring to entities and assigns their types, such as PERSON, ORGANIZATION and LOCATION. Many practical schemas also include DATE, TIME, MONEY or domain-specific types. The exact allowed labels depend on the application's annotation scheme. NER has two parts: **boundary detection** finds where the entity starts and ends; **classification** assigns the appropriate type. A multiword name should normally be one entity span: “New York” is a location span, not two unrelated location decisions. Text: Mary joined Google in New York on Monday. → Mary → PERSON → Google → ORGANIZATION → New York → LOCATION → Monday → DATE Not every noun is a named entity. In “a developer works at Google”, developer is an ordinary job description while Google is a named organization. A specialized HR system may define a JOB\_TITLE category, but it must state that schema explicitly. NER identifies mentions; entity linking is the separate task of connecting a mention to a unique database identity. | Task | Question it answers | Example | | --- | --- | --- | | POS | What grammatical role does this token have? | Google/NNP | | Chunking | Which adjacent words form a phrase? | [NP a skilled developer] | | NER | Which span names an entity, and of what type? | [Google] ORGANIZATION | | Entity linking | Which real entity does this mention denote? | Connect Paris to the intended city or person record | **Applications.** NER structures search results and documents, extracts organizations/people from news, routes support tickets by product or organization, supports recommendation, and extracts specialized entities from healthcare or legal text. Context is essential: “Apple released a laptop” names an organization, while “she ate an apple” describes fruit. **Yaad rakho:** NER mein do kaam: naam kahan se kahan tak, aur kis category ka. **Chunking**, also called shallow parsing, groups adjacent tagged words into meaningful non-overlapping units, commonly noun phrases. It usually identifies local phrase boundaries without producing the complete hierarchical structure of a sentence. **Chinking** removes specified tokens from an existing candidate chunk, splitting or trimming it. The/DT small/JJ cat/NN sat/VBD on/IN the/DT mat/NN → NP: [The small cat] → Outside: sat on → NP: [the mat] A POS tagger labels individual words. A chunker groups words into phrases. A full parser shows how phrases relate recursively to form the entire sentence. These are related tasks with different outputs. | Operation | Input | Output | | --- | --- | --- | | POS tagging | The small cat | The/DT small/JJ cat/NN | | NP chunking | Tagged tokens | [NP The small cat] | | Chinking | A broad candidate chunk | Remove verbs/prepositions, keep noun-phrase segments | | Full parsing | Sentence + grammar | A nested tree with sentence, noun phrase and verb phrase nodes | **Rule-based chunking** uses tag patterns. Statistical or learned chunkers infer boundaries from labeled examples. A useful NP rule is an optional determiner, followed by zero or more adjectives, followed by a noun. In NLTK-style notation: `NP: {
?*}`. Here ? means zero or one occurrence; \* means zero or more; + means one or more when used. The braces mean “form a chunk around the matched sequence”. **Question. Apply the NP rule optional DT, zero or more JJ, then one NN to “The/DT quick/JJ brown/JJ fox/NN jumps/VBZ over/IN the/DT lazy/JJ dog/NN”.** #### Given Tagged input is supplied. Rule: DT? JJ\* NN. #### To find All noun-phrase chunks. #### Formula and why it applies `NP → DT? JJ* NN` The rule groups a noun with its preceding determiner and adjectives; a verb or preposition does not match this NP pattern. #### Solution ##### Step 1. Start at The The matches optional DT; quick and brown both match JJ\*; fox matches NN. ##### Step 2. Form the first NP **[NP The quick brown fox]** ##### Step 3. Skip nonmatching tokens jumps/VBZ and over/IN do not satisfy the NP pattern. ##### Step 4. Match the second NP the is DT, lazy is JJ, dog is NN: **[NP the lazy dog]**. **Final answer.** [NP The quick brown fox] jumps over [NP the lazy dog]. **Question. Using NP → DT? JJ\* NN, identify the chunks in “bright/JJ stars/NN shine/VBP and/CC a/DT moon/NN glows/VBZ”. Use the supplied tags exactly.** #### Given Input tags label stars as NN for this exercise. Rule requires optional DT, zero or more JJ, then NN. #### To find The matched NP spans. #### Formula and why it applies `NP → DT? JJ* NN` The optional determiner may be absent and the adjective repetition may be empty. #### Solution ##### Step 1. Match bright stars No determiner is present, which DT? permits. bright matches JJ; stars matches NN. ##### Step 2. Inspect shine and and VBP and CC fall outside the NP rule. ##### Step 3. Match a moon a matches DT; no adjective is present; moon matches NN. **Final answer.** Chunks are [NP bright stars] and [NP a moon]. With ordinary Penn tagging, stars is usually NNS; broaden the noun pattern to include NNS when that tag is used. **Question. Begin with one candidate chunk containing “The/DT cat/NN sat/VBD on/IN the/DT mat/NN”. Chink out verbs and prepositions. Show the remaining chunks.** #### Given Initial candidate: [The cat sat on the mat]. Remove tags VBD and IN. #### To find The chunk segments after exclusion. #### Formula and why it applies Chinking removes matched tokens from the existing chunk. Removing the middle verb/preposition region divides the original candidate into two remaining contiguous spans. #### Solution ##### Step 1. Locate the tokens to remove sat is VBD and on is IN. ##### Step 2. Remove those tokens from chunk membership The remaining token groups are “The cat” and “the mat”. sat and on still exist in the sentence; they are simply outside these chunks. ##### Step 3. Write the final chunked sentence **[The cat]** sat on **[the mat]**. **Final answer.** Two chunks remain: The cat; the mat. **Yaad rakho:** Chunking = andar group karo; chinking = unwanted tokens ko group se bahar karo. **Type-token ratio** = number of distinct tokens / total tokens. In “cats chase cats”, types={cats,chase}=2 and tokens=3, so TTR=2/3. This is an authored illustration of the definition, not extra paper data. --- ## TEE · Q5a · Lexical and vector semantics Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r24--q5a/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-r24/q5a Previous examination · complete worked answers **Question. Explain lexical semantics and vector semantics with examples.** [10 marks] ### Answer **A lexical relation connects words or senses by a meaning relationship.** State the relation and its direction before giving the example. | Relation | Exam definition | Examples and direction | | --- | --- | --- | | Homonymy | The same form has unrelated meanings. | bank: financial institution / river edge; bat: animal / sports implement | | Polysemy | One lexical item has multiple related senses. | head: body part / head of a department; paper: material / article | | Synonymy | Different words have identical or closely similar meanings in a specified sense. | sofa / couch; automobile / car | | Antonymy | Words have opposite meanings. | hot / cold; alive / dead; buy / sell | | Hyponymy | A more specific concept is a type of a more general one. | rose is a flower; car is a vehicle | | Hypernymy | A broader concept includes a more specific one. | flower is a hypernym of rose | | Meronymy | A concept is a part of another concept. | wheel is part of car; finger is part of hand | | Holonymy | A concept is the whole containing a part. | car is a holonym of wheel | ### How to distinguish the commonly confused pairs For homonymy versus polysemy, ask whether the senses share an identifiable meaning. A river bank and a financial bank are unrelated. A financial bank and a blood bank share a repository/storage idea, illustrating polysemy in the supplied notes. For hyponymy versus meronymy, use the sentence test: **“X is a kind of Y”** indicates hyponymy; **“X is part of Y”** indicates meronymy. A wheel is not a type of car. | Expression | Correct relation | Reason | | --- | --- | --- | | sparrow → bird | Hyponymy | A sparrow is a kind of bird | | bird → sparrow | Hypernymy | Bird is the broader category | | keyboard → computer | Meronymy | Keyboard is a component | | computer → keyboard | Holonymy | Computer is the containing whole | | mouth of a person / mouth of a river | Polysemy | Both senses share the idea of an opening; the river use is a related extension | Synonyms are not always interchangeable in every context or register. “Big mistake” and “large mistake” can sound different even though big and large overlap in meaning. Antonyms can be gradable (hot/cold), complementary (alive/dead) or relational converses (buy/sell). HYPO = specific type; HYPER = bigger category. MERO = part; HOLO = whole. **The distributional hypothesis states that words occurring in similar contexts tend to have similar meanings or semantic behaviour.** “Cat” and “dog” often occur near pet, feed and animal. Their context patterns can therefore be represented and compared numerically. A **vector** is an ordered list of numbers. Each position must have a fixed meaning. For context order [pet, food, road], cat=[4,3,0] means four co-occurrences with pet, three with food, none with road. A **matrix** stacks vectors as rows in a table. A **word–context co-occurrence matrix** records how often a target word occurs with each context word under a specified context definition. A window may be a sentence, document, or a fixed number of neighbouring tokens. State its width, direction and whether targets count themselves. Choose target words → Choose context/window → Count co-occurrences → Store each row as a vector → Compare vectors **Question. Build the symmetric co-occurrence matrix for “cats eat fish” using a window of one token on either side, no self-counts and vocabulary [cats,eat,fish].** #### Given One sentence, one occurrence of each token. Left and right adjacent tokens are contexts. #### To find A 3×3 target-by-context count matrix. #### Formula and why it applies $$ X_{ij} $$ counts the number of times context word $$ j $$ occurs within the chosen window of target $$ i $$ . #### Step 1. Target cats Only eat is adjacent. Row cats = [0,1,0]. #### Step 2. Target eat cats is left and fish is right. Row eat = [1,0,1]. #### Step 3. Target fish Only eat is adjacent. Row fish = [0,1,0]. #### Step 4. Assemble the matrix | Target / context | cats | eat | fish | | --- | --- | --- | --- | | cats | 0 | 1 | 0 | | eat | 1 | 0 | 1 | | fish | 0 | 1 | 0 | **Answer.** Rows are [0,1,0], [1,0,1], [0,1,0]. The symmetry follows from the symmetric window. **Question. Build a target–context matrix for sentences “cats eat fish” and “dogs eat fish”. Target words are cats and dogs; context columns [eat,fish]; use the entire sentence excluding the target.** #### Given Each sentence contains one target. Count context token occurrences. #### To find Two context vectors. #### Formula and why it applies Count each context word in every sentence containing the target. #### Step 1. Count cats contexts In its sentence: eat occurs once, fish once → [1,1]. #### Step 2. Count dogs contexts In its sentence: eat occurs once, fish once → [1,1]. #### Step 3. Compare Both targets have identical observed context patterns in this very small corpus. **Answer.** cats=[1,1], dogs=[1,1]. This corpus supplies no context evidence to distinguish them. **Question. For “a b a”, use a right-only window of one token. Build the matrix with rows/columns [a,b].** #### Given Count directed target→next-word events, not both directions. #### To find The co-occurrence matrix. #### Formula and why it applies $$ X_{ij}=C(\text{target }i\text{ immediately followed by }j) $$ . #### Step 1. First target a Next token is b → add 1 to X(a,b). #### Step 2. Target b Next token is a → add 1 to X(b,a). #### Step 3. Final a No following token → no addition. #### Step 4. Write matrix | Target / next | a | b | | --- | --- | --- | | a | 0 | 1 | | b | 1 | 0 | **Answer.** a=[0,1], b=[1,0]. A different context window would produce a different matrix. Word ka meaning: uske aas-paas kaunse words kitni baar aaye, uski number-list. --- ## TEE · Q5b · Classification label structures Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r24--q5b/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-r24/q5b Previous examination · complete worked answers **Question. Compare binary, multiclass and multilabel supervised text classification with examples.** [10 marks] ### Answer | Type | Allowed output | Example | Typical output rule | | --- | --- | --- | --- | | Binary | One of two labels | positive OR negative review | One sigmoid or two-class softmax | | Multiclass | Exactly one of several labels | sport OR business OR science | Softmax over mutually exclusive classes | | Multilabel | Zero, one or several labels | Article tagged technology AND finance | Independent sigmoid per label | Multiclass and multilabel differ in whether labels may coexist, not merely in the number of labels. A loss and evaluation measure must match the task. --- ## TEE · Q6a · TF–IDF in document representation Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r24--q6a/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-r24/q6a Previous examination · complete worked answers **Question. Explain TF–IDF in document representation and information retrieval with an example.** [10 marks] ### Answer **TF–IDF weights a term highly when it is frequent in a particular document but uncommon across the corpus.** The supplied Unit 2 slides use normalized term frequency and base-10 inverse document frequency: $$ \operatorname{TF}(t,d)=\frac{C(t,d)}{N_d} $$ $$ \operatorname{IDF}(t)=\log_{10}\frac{D}{\operatorname{df}(t)} $$ $$ \operatorname{TFIDF}(t,d)=\operatorname{TF}(t,d)\operatorname{IDF}(t). $$ $$ N_d $$ is the document's token count after the specified preprocessing; $$ D $$ is number of documents; $$ \operatorname{df}(t) $$ is the number of documents containing the term at least once. A word repeated ten times in one document contributes one to document frequency. **Why the log?** It compresses the ratio's scale. If a term occurs in all documents, $$ D/df=1 $$ , so IDF=0 under this unsmoothed convention. Other libraries use smoothed IDF, different TF or normalization. Use the question's formula; do not silently substitute a library default. **Question. A corpus contains 10 documents. The word model appears in 2 documents and occurs 3 times in a 12-token document. Compute TF, IDF and TF–IDF using the slide convention.** #### Given D=10, df(model)=2, C(model,d)=3, Nd=12; log base 10. #### To find The TF–IDF weight. #### Formula and why it applies $$ TF=C/N_d $$ , $$ IDF=\log_{10}(D/df) $$ , then multiply. #### Step 1. TF $$ 3/12=0.25 $$ . #### Step 2. Document ratio $$ 10/2=5 $$ . #### Step 3. IDF $$ \log_{10}5\approx0.698970 $$ . #### Step 4. Weight $$ 0.25\times0.698970\approx0.174743 $$ . **Answer.** TF=0.25, IDF≈0.698970, TF–IDF≈0.174743. **Question. A term occurs 4 times in a 20-token document and is present in all 8 corpus documents. Calculate its TF–IDF.** #### Given C=4, Nd=20, D=8, df=8. #### To find Its weight under unsmoothed base-10 IDF. #### Formula and why it applies $$ TF=C/N_d $$ , $$ IDF=\log_{10}(D/df) $$ . #### Step 1. TF $$ 4/20=0.2 $$ . #### Step 2. IDF $$ \log_{10}(8/8)=\log_{10}1=0 $$ . #### Step 3. Multiply $$ 0.2\times0=0 $$ . **Answer.** Weight 0: this term does not distinguish documents under this convention. **Question. Find TF–IDF vectors for D1=“cat cat dog”, D2=“dog fish”, D3=“cat fish fish”, using [cat,dog,fish], normalized TF and log10 IDF.** #### Given No further preprocessing. Each term occurs in exactly two of the three documents. #### To find All three vectors. #### Formula and why it applies First count df by documents, then compute each document's TF, then multiply by its term's IDF. #### Step 1. Common IDF $$ D=3 $$ , $$ df(cat)=df(dog)=df(fish)=2 $$ . Each IDF $$ =\log_{10}(3/2)\approx0.176091 $$ . #### Step 2. D1 term frequencies Length 3 → $$ [2/3,1/3,0] $$ . #### Step 3. D1 weights $$ [(2/3)0.176091,(1/3)0.176091,0]\approx[0.117394,0.058697,0] $$ . #### Step 4. D2 weights Length 2; TF $$ [0,1/2,1/2] $$ . Vector $$ \approx[0,0.088046,0.088046] $$ . #### Step 5. D3 weights Length 3; TF $$ [1/3,0,2/3] $$ . Vector $$ \approx[0.058697,0,0.117394] $$ . **Answer.** D1≈[0.117394,0.058697,0]; D2≈[0,0.088046,0.088046]; D3≈[0.058697,0,0.117394]. TF: iss document mein kitna. IDF: poore corpus mein kitna rare. Dono multiply. **Information retrieval (IR)** locates and ranks items relevant to an information need, usually expressed as a query. The collection may contain documents, product descriptions or scientific papers. Retrieval estimates relevance; it does not itself establish that every retrieved claim is true. Documents → cleaning → Build index → Query → same representation → Match & rank → Return relevant results ### The inverted index An inverted index maps a term to the documents containing it, often with positions or counts. For D1 “cats drink milk” and D2 “dogs drink water”, the posting list for drink is {D1,D2}, while milk maps only to D1. This avoids rescanning every document for every query. | Term | Posting list | | --- | --- | | cats | D1 | | drink | D1, D2 | | milk | D1 | | dogs | D2 | | water | D2 | ### Matching and ranking Boolean retrieval combines term membership using AND, OR and NOT. Vector-space retrieval represents documents and queries as weighted vectors, for example with TF-IDF, and compares them with cosine similarity. Learned retrieval can use dense embeddings for meaning-based matching. The representation used at query time must be compatible with the document index. ### Search engines and recommendation A search engine receives an explicit query and ranks matching indexed pages. A recommender may derive an implicit query from a user’s liked items or profile and retrieve similar candidates. Both require candidate retrieval and ranking, but the source of the information need differs. ### Evaluate the results For retrieval, TP is the number of retrieved relevant documents, FP retrieved irrelevant documents, and FN relevant documents not retrieved. Precision and recall use the same formulas as classification. Ranking measures also consider the positions of relevant items; a relevant result at rank 1 is more useful than one far below. ### Retrieval evaluation A search system retrieves 8 documents; 6 are relevant. There are 10 relevant documents in the collection. Find precision, recall and F1. **Given** Retrieved=8; relevant retrieved=6; all relevant=10. **To find** P, R and F1. ### Formula and why it applies $$ P=\frac{\text{relevant retrieved}}{\text{retrieved}},\quad R=\frac{\text{relevant retrieved}}{\text{all relevant}} $$ $$ F_1=\frac{2PR}{P+R} $$ The denominator is 8 for precision and 10 for recall because they ask different questions. ### Solution #### Step 1. Identify the confusion counts TP=6; FP=8−6=2; FN=10−6=4. #### Step 2. Precision $$ P=\frac68=0.75 $$ #### Step 3. Recall $$ R=\frac6{10}=0.6 $$ #### Step 4. F1 $$ F_1=\frac{2(0.75)(0.6)}{0.75+0.6}=\frac{0.9}{1.35}=\frac23\approx0.666667 $$ Precision=0.75, recall=0.60, F1≈0.666667. **Yaad rakho:** Precision: jo laaye, unmein kitne sahi? Recall: jo sahi the, unmein kitne laaye? **Source / convention note.** The prompt requests an example but supplies no corpus. Numbers in the explanation are authored illustrations, not original numerical givens. --- ## TEE · Q6b · HMM tables and path Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r24--q6b/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-r24/q6b Previous examination · complete worked answers **Question. For the four tagged sentences below construct transition and emission tables (3 marks each), then calculate Geeta/N will/M spot/V Rita/N using initial P(N)=0.5, P(M)=0.25, P(V)=0.25.** [10 marks] ### Answer ### Given | Sentence | Given tags | | --- | --- | | Rita can teach Will | N M V N | | Spot will see Will | N M V N | | Will Rita spot Geeta | N N V N | | Geeta will pat Spot | N M V N | Required path: Geeta will spot Rita / N M V N. Given initial probabilities: {'N': Fraction(1, 2), 'M': Fraction(1, 4), 'V': Fraction(1, 4)} ### To find Emission and transition tables and the specified tag-path joint probability ### Formula: which one and why $$ P(w\mid t)=\frac{c(t,w)}{c(t)} $$ $$ P(t_j\mid t_i)=\frac{c(t_i,t_j)}{\sum_kc(t_i,t_k)} $$ The second formula is the declared no-END convention. Reset at sentence boundaries; never create a transition from the last word of one training sentence to the first of the next. ### Step 1. Count emissions | Tag | Total occurrences | Rita | can | teach | Will | Spot | will | see | spot | Geeta | pat | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | N | 9 | 2 | 0 | 0 | 3 | 2 | 0 | 0 | 0 | 2 | 0 | | M | 3 | 0 | 1 | 0 | 0 | 0 | 2 | 0 | 0 | 0 | 0 | | V | 4 | 0 | 0 | 1 | 0 | 0 | 0 | 1 | 1 | 0 | 1 | ### Step 2. Divide each emission by its tag count | Tag | Rita | can | teach | Will | Spot | will | see | spot | Geeta | pat | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | N | 2/9 | 0 | 0 | 1/3 | 2/9 | 0 | 0 | 0 | 2/9 | 0 | | M | 0 | 1/3 | 0 | 0 | 0 | 2/3 | 0 | 0 | 0 | 0 | | V | 0 | 0 | 1/4 | 0 | 0 | 0 | 1/4 | 1/4 | 0 | 1/4 | ### Step 3. Count transitions inside each sentence | From | Outgoing transitions | Sentence-final count | N | M | V | | --- | --- | --- | --- | --- | --- | | N | 5 | 4 | 1 | 3 | 1 | | M | 3 | 0 | 0 | 0 | 3 | | V | 4 | 0 | 4 | 0 | 0 | ### Step 4. Convert to a transition table | From / to | N | M | V | | --- | --- | --- | --- | | N | 1/5 | 3/5 | 1/5 | | M | 0 | 0 | 1 | | V | 1 | 0 | 0 | Here sentence-final tags are excluded from the no-END outgoing denominator. If END is explicitly modeled, divide all tag→tag and tag→END counts by the total tag count instead; the row then includes END. Do not mix denominators. ### Step 5. Multiply the given path, in order | Factor | Probability | Running product | | --- | --- | --- | | Initial N | 1/2 | 1 × 1/2 = 1/2 | | Emit Geeta | 2/9 | 1/2 × 2/9 = 1/9 | | Transition N→M | 3/5 | 1/9 × 3/5 = 1/15 | | Emit will | 2/3 | 1/15 × 2/3 = 2/45 | | Transition M→V | 1 | 2/45 × 1 = 2/45 | | Emit spot | 1/4 | 2/45 × 1/4 = 1/90 | | Transition V→N | 1 | 1/90 × 1 = 1/90 | | Emit Rita | 2/9 | 1/90 × 2/9 = 1/405 | ### Final answer The two displayed probability tables are the estimated HMM. Zero means unobserved without smoothing, not logically impossible. **No-END path joint probability = 1/405 = 0.00246913580247.** If a normalized END-state model is explicitly adopted instead, this complete terminated sequence has probability 4/6561 = 0.000609663161103. These are different model conventions, not contradictory answers. --- ## TEE · Q7a · Healthcare and finance applications Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r24--q7a/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-r24/q7a Previous examination · complete worked answers **Question. Explain NLP use cases in healthcare and finance.** [10 marks] ### Answer | Domain / task | Input → method → useful output | Example and limit | | --- | --- | --- | | Healthcare: clinical extraction | Notes → entity/negation/relation extraction → diagnoses, medicines and doses | Distinguish no fever from fever; clinician review is needed | | Healthcare: coding and search | Clinical text → classify/retrieve → candidate billing codes or evidence | Coding suggestions are not autonomous diagnoses | | Healthcare: patient messaging | Patient question → intent + retrieval → approved explanation | Escalate uncertainty and emergencies; protect identifiers | | Finance: sentiment/event analysis | News/filings → entities/events/sentiment → issuer-specific signals | Distinguish rumor from confirmed earnings; not guaranteed returns | | Finance: document extraction | Reports/contracts → fields/relations → searchable structured records | Keep source spans and verify amounts/dates | | Finance: customer support/compliance triage | Messages → intent/classification → routed cases or flagged text | Human investigation is required; a language flag is not proof of wrongdoing | Acquire permitted text → Clean and preserve context → Extract / classify / retrieve → Evaluate on domain data → Human-reviewed use --- ## TEE · Q7b · Naive Bayes sentiment Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r24--q7b/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-r24/q7b Previous examination · complete worked answers **Question. Train Naive Bayes on: positive “amaze wonderful great”; “happy happy great”; “amaze great happy disgust”; “great happy wonderful”. Negative “disgust sad terrible great”; “sad terrible sad happy”. Predict sentence 7: “great happy disgust great”.** [10 marks] ### Answer ### Given | Training text | Class | | --- | --- | | amaze wonderful great | positive | | happy happy great | positive | | amaze great happy disgust | positive | | great happy wonderful | positive | | disgust sad terrible great | negative | | sad terrible sad happy | negative | Test: great happy disgust great ### To find Classify the test using multinomial Naive Bayes. ### Formula: which one and why Assume add-one smoothing unless the question supplies probabilities. Keep repeated words as repeated factors. Use training vocabulary only; ignore out-of-vocabulary words unless an unknown-token model is defined. $$ \mathrm{score}(c)=P(c)\prod_{w\in V}\left(\frac{c(w,c)+1}{N_c+|V|}\right)^{c(w,d)} $$ ### Step 1. Vocabulary and class priors V = {amaze, disgust, great, happy, sad, terrible, wonderful}; |V|=7. | Class | Documents | Prior | | --- | --- | --- | | negative | 2 | 1/3 | | positive | 4 | 2/3 | ### Step 2. Count every training word within each class | Class | Total tokens | amaze | disgust | great | happy | sad | terrible | wonderful | | --- | --- | --- | --- | --- | --- | --- | --- | --- | | negative | 8 | 0 | 1 | 1 | 1 | 3 | 2 | 0 | | positive | 13 | 2 | 1 | 4 | 4 | 0 | 0 | 2 | ### Step 3. Count the test words | Word | Count | | --- | --- | | great | 2 | | happy | 1 | | disgust | 1 | OOV words: none ### Step 4. Calculate likelihoods and multiply with the prior | Class | Prior | Individual factors | Exact score | Decimal score | | --- | --- | --- | --- | --- | | negative | 1/3 | great: (1+1)/(8+1×7) = 2/15, used 2 time(s); happy: (1+1)/(8+1×7) = 2/15, used 1 time(s); disgust: (1+1)/(8+1×7) = 2/15, used 1 time(s) | 16/151875 | 0.000105349794239 | | positive | 2/3 | great: (4+1)/(13+1×7) = 1/4, used 2 time(s); happy: (4+1)/(13+1×7) = 1/4, used 1 time(s); disgust: (1+1)/(13+1×7) = 1/10, used 1 time(s) | 1/960 | 0.00104166666667 | ### Step 5. Compare and optionally normalize Normalized scores: negative = 0.091846802, positive = 0.908153198 Division by the same total does not change the winning class. ### Final answer **Predicted class: positive.** **Yaad rakho:** Count → prior → smoothed word chances → repeated factors → compare. --- ## TEE · Q1a · WSD and pronoun resolution Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f23--q1a/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-f23/q1a Previous examination · complete worked answers **Question. Define Word Sense Disambiguation and Pronoun Resolution. Give an example of each.** [5 marks] ### Answer **Word sense disambiguation (WSD) is the task of selecting the intended sense of an ambiguous word in its context from a set of possible senses.** The output is a sense label, not a different replacement word. Input: “He grilled the bass.” → Candidate senses: fish / low sound → Context: grilled → Output: fish sense The **target word** is the ambiguous word. The **context** is the surrounding evidence. The **sense inventory** lists candidate meanings, often using WordNet. A **feature** is a usable clue: nearby word, position, part-of-speech tag or a phrase. | Approach | What it needs | How it decides | Limitation | | --- | --- | --- | --- | | Knowledge based | Dictionary, thesaurus or semantic graph | Definition overlap, category evidence or graph scores | Short definitions and missing knowledge | | Supervised | Examples already labelled with senses | Learns context → sense mapping | Labelling cost, limited rare-sense examples | | Semi-supervised | Small labelled set plus unlabelled data | Bootstraps from reliable seed examples | Early errors can spread | | Unsupervised sense induction | Unlabelled contextual occurrences | Clusters similar contexts | Clusters are not automatically named dictionary senses | | Neural/contextual | Learned word/context representations | Predicts or compares a context representation | Needs suitable data/model and evaluation | **Bag-of-words features** record nearby words while ignoring exact positions. **Collocational features** preserve a positional phrase such as “river bank”, “bank account”, “chocolate bar” or “bar exam”. ### Why WSD matters Machine translation needs the correct target-language meaning; information retrieval should distinguish river-bank travel from banking services; text-to-speech may need different pronunciations for “bass”; information extraction should avoid connecting entities through the wrong relation. WSD: same spelling ke options mein context dekh ke sahi meaning select. **Pronoun resolution** identifies the antecedent referred to by a pronoun. In “Rita arrived. She smiled”, she refers to Rita. WSD chooses a word meaning (bank as finance or river edge); pronoun resolution chooses a discourse entity. --- ## TEE · Q1b · WordNet and its hierarchy Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f23--q1b/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-f23/q1b Previous examination · complete worked answers **Question. Define WordNet. Describe the WordNet hierarchy.** [5 marks] ### Answer **WordNet is a lexical database that groups word senses into synonym sets called synsets and links them through semantic and lexical relations.** It is a network of meanings, not merely a list of similar spellings. A synset represents one concept. Its **lemmas** are the words that can express that concept. A **gloss** is a short definition, often accompanied by an example. An ambiguous word can occur in several synsets. vehicle (general) → car / automobile (same concept) → sports car (specific) This is a simplified teaching hierarchy, not a claim about every intermediate WordNet node. Going upward reaches hypernyms; going downward reaches hyponyms. Multiple parents can exist, so the structure need not be a single simple tree. | Operation | Meaning | Illustration | | --- | --- | --- | | Get senses of a word | Find candidate synsets | bank → financial sense, river-edge sense, … | | Get lemmas | Find synonyms within one synset | car, automobile, motorcar | | Get definition/examples | Read the gloss and uses | Choose the intended concept | | Get hypernyms | Move to broader categories | car → vehicle | | Get hyponyms | Move to narrower categories | vehicle → particular vehicle categories | | Get meronyms/holonyms | Find part/whole links | car ↔ wheel | ### WordNet versus a thesaurus A thesaurus mainly groups related expressions for vocabulary lookup. WordNet explicitly distinguishes senses and labels relations such as hypernymy, hyponymy and meronymy. This supports WSD, query expansion, lexical similarity and semantic search. ### Three uses explained 1. **Search expansion:** a query for “automobile” can retrieve relevant documents using “car”. Expansion should use the right sense. 2. **WSD:** definitions of the senses of “bank” provide words that can be compared with the sentence context. 3. **Similarity:** concepts near one another in an “is-a” hierarchy tend to be semantically related. Path-based measures depend on which senses were selected. WordNet mein spelling ka group nahi—meaning ka synset, aur meanings ke beech labelled links. --- ## TEE · Q1c · Sentiment feature extraction Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f23--q1c/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-f23/q1c Previous examination · complete worked answers **Question. Extract x₁ = log₁₀(word count), x₂ = count of positive lexicon words and x₃ = count of negative lexicon words from: “I was amazed at how the product quickly it arrived. The customer service was exceptional. The support team responded promptly and professionally. I was not sad with my decision. Overall, my experience with them was outstanding.” Positive lexicon={amazed,promptly,outstanding,great}; negative={horrible,sad,disgusting}.** [5 marks] ### Answer ### Given I was amazed at how the product quickly it arrived. The customer service was exceptional. The support team responded promptly and professionally. I was not sad with my decision. Overall, my experience with them was outstanding. ### To find The three numerical features. ### Formula: which one and why Use word tokens after stripping punctuation, keep stopwords, and match only the supplied lexicons. Raw lexicon counting does not automatically reverse a word under negation. ### Step 1. List and count every word | Position | Word | | --- | --- | | 1 | i | | 2 | was | | 3 | amazed | | 4 | at | | 5 | how | | 6 | the | | 7 | product | | 8 | quickly | | 9 | it | | 10 | arrived | | 11 | the | | 12 | customer | | 13 | service | | 14 | was | | 15 | exceptional | | 16 | the | | 17 | support | | 18 | team | | 19 | responded | | 20 | promptly | | 21 | and | | 22 | professionally | | 23 | i | | 24 | was | | 25 | not | | 26 | sad | | 27 | with | | 28 | my | | 29 | decision | | 30 | overall | | 31 | my | | 32 | experience | | 33 | with | | 34 | them | | 35 | was | | 36 | outstanding | ### Step 2. Apply the logarithm $$ x_1=\log_{10}36=1.556302501 $$ ### Step 3. Count positive matches amazed, promptly, outstanding each occur once. great does not occur. x₂=3. exceptional is not in the supplied lexicon, so it is not counted. ### Step 4. Count negative matches sad occurs once; horrible and disgusting do not. x₃=1. The phrase “not sad” remains one raw negative-lexicon occurrence under the specified feature definition. ### Final answer **(x₁,x₂,x₃)=(1.556302501,3,1).** --- ## TEE · Q1d · Binary, multiclass and multilabel Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f23--q1d/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-f23/q1d Previous examination · complete worked answers **Question. Compare binary, multiclass and multilabel supervised text classification with examples.** [5 marks] ### Answer | Type | Allowed output | Example | Typical output rule | | --- | --- | --- | --- | | Binary | One of two labels | positive OR negative review | One sigmoid or two-class softmax | | Multiclass | Exactly one of several labels | sport OR business OR science | Softmax over mutually exclusive classes | | Multilabel | Zero, one or several labels | Article tagged technology AND finance | Independent sigmoid per label | Multiclass and multilabel differ in whether labels may coexist, not merely in the number of labels. A loss and evaluation measure must match the task. --- ## TEE · Q2a · Text corpus structure and applications Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f23--q2a/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-f23/q2a Previous examination · complete worked answers **Question. Explain text-corpus structure and the applications of text corpora in NLP.** [10 marks] ### Answer **A corpus is a collection of language data assembled for study or computational use.** Its plural is corpora. A corpus can contain books, news, reviews or conversations. It need not be labelled. **Annotation** adds linguistic information such as POS tags, entities, parse structures or sense labels. | Structure | Meaning | Use | | --- | --- | --- | | Plain | Collection without category labels | Word counts, language modelling | | Categorized | Documents grouped by genre/topic | Classification | | Overlapping categories | One document has several categories | Multi-topic news | | Temporal | Documents indexed by time | Language change | | Annotated | Linguistic labels attached | Tagging, parsing, WSD | | Parallel | Aligned texts in different languages | Translation | A **tagset** is the allowed label inventory. A **treebank** is a corpus annotated with syntax trees. **Metadata** records source, date, genre, language, annotation method and limitations. A common benchmark supports comparison under shared data and evaluation conditions. **Raw example:** “Riya works in Pune.” **POS annotation:** Riya/PROPN works/VERB in/ADP Pune/PROPN. **NER annotation:** Riya/PERSON, Pune/LOCATION. Each annotation answers a different task. A good corpus is relevant to its domain, sufficiently representative, consistently annotated and appropriately split. More data alone do not repair duplicated examples, incorrect labels or a mismatch between training and real use. Corpus=text collection. Annotation=text ke saath labels. Metadata=collection ki information. | Resource | Contents | Typical use | | --- | --- | --- | | Gutenberg selection | Literary/book texts | Long-form language and vocabulary | | Web text | Informal domain-specific text | Study usage beyond edited books | | Chat corpus | Conversational posts | Dialogue, informal forms | | Brown | English texts across genres | Genre comparison | | Reuters | News with topic labels | Text classification; overlapping topics | | Inaugural | Speeches indexed by time | Vocabulary change across years | | Wordlist | Words/names/stopwords | Lexical filtering | | Pronouncing dictionary | Words and phonetic forms | Speech, stress, rhyme | **Gutenberg versus web/chat:** book language does not represent every spelling or conversational pattern in messages. **Brown versus Reuters:** Brown emphasizes genres; Reuters news can have overlapping topic labels. **Inaugural addresses:** dates provide a time dimension. Corpus sizes depend on version, subset and tokenization. Explain their purpose and structure rather than assuming a historical count is a permanent property. Gutenberg=books; web/chat=informal; Brown=genres; Reuters=news topics; Inaugural=time. Word-sequence counts support n-gram language models and smoothing. POS-annotated sentences support supervised taggers. Sense-labelled contexts support Naive Bayes WSD. Word-context counts support distributional semantics and GloVe. Parallel aligned sentences support translation. For a question asking the importance of corpora, give three points: empirical language evidence, examples for training/evaluation, and reproducible comparison. For an annotated-corpus question, define the added labels and give a concrete example. For resource comparisons, distinguish running text, lexical lists, pronunciation entries and semantic relations. Same text se task ke hisaab se counts, tags, senses ya context patterns nikaalte hain. --- ## TEE · Q2b · Lesk versus Walker and bank disambiguation Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f23--q2b/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-f23/q2b Previous examination · complete worked answers **Question. Differentiate Lesk and Walker WSD. Calculate sense scores and disambiguate “bank” in “Money in this bank fetches an interest of 8% per annum.”** [10 marks] ### Answer **The Walker approach uses thesaurus categories as semantic evidence.** Associate each candidate sense with a category. Count or weight context words that support that category. Choose the largest total. $$ \operatorname{score}(s)=\sum_{w\in C} I(w,\operatorname{category}(s)) $$ $$ I=1 $$ if the supplied category mapping says the context word supports the sense, otherwise $$ I=0 $$ . The summation symbol means add all those contributions. Weighted questions may replace 1 with a supplied weight. Category membership must be supplied or justified; it is not guessed from word frequency. **Question. Disambiguate bank using Walker scoring. Context words are money, interest, annum, fetch. The finance category supports money, interest and annum; the location category supports none of these words.** #### Given Two categories: Finance and Location. Each supported word contributes 1. #### To find The two totals and selected bank sense. #### Formula and why it applies Sum the category indicators for each sense. #### Step 1. Make a contribution table | Word | Finance | Location | | --- | --- | --- | | money | 1 | 0 | | interest | 1 | 0 | | annum | 1 | 0 | | fetch | 0 | 0 | #### Step 2. Add Finance $$ 1+1+1+0=3 $$ . #### Step 3. Add Location $$ 0+0+0+0=0 $$ . #### Step 4. Select 3 > 0, so select the financial institution. **Answer.** Finance, with score 3 versus 0. **Question. Use Walker scoring for “The fisherman caught bass beside the lake.” Context {fisherman,caught,lake}. The Fish category contains all three clues; the Music category contains none.** #### Given Fish and Music senses, binary category membership given. #### To find The correct bass sense. #### Formula and why it applies Score = sum of context-category matches. #### Step 1. Record Fish matches fisherman → 1; caught → 1; lake → 1. #### Step 2. Add Fish score $$ 1+1+1=3 $$ . #### Step 3. Add Music score $$ 0+0+0=0 $$ . #### Step 4. Choose Fish has the larger score. **Answer.** Bass refers to a fish. **Question. Score “The player used a bat in the match.” Context {player,match}. Sport supports both words with weights 2 and 1; Animal supports neither. Use the supplied weighted Walker scheme.** #### Given Sport weights {player:2, match:1}; Animal weights {player:0, match:0}. #### To find Weighted sense scores. #### Formula and why it applies Use the sum of supplied weights, not an unweighted count, because the question explicitly gives weights. #### Step 1. Sport contribution of player Add 2. #### Step 2. Sport contribution of match Add 1. Sport total = 2 + 1 = 3. #### Step 3. Animal total 0 + 0 = 0. #### Step 4. Compare 3 > 0 selects the sporting implement. **Answer.** Bat is the sports implement. Walker: dictionary ke exact words nahi, context kis topic/category ko support karta hai woh count. | Aspect | Lesk | Walker/thesaurus method | | --- | --- | --- | | Evidence | Overlap of context with dictionary glosses | Context-word membership in semantic subject categories | | Score | Number of shared normalized words | Number/weight of context words supporting each sense category | | Dependency | Gloss definitions and preprocessing | Thesaurus categories and their assignments | ### Given Context is the printed sentence. No thesaurus category list is supplied. ### To find State the sense and give a transparent illustrative score. ### Formula: which one and why Declare the categories first; count only declared evidence. A numerical Walker score is not uniquely determined without the thesaurus assignments. ### Step 1. Choose explicit illustrative categories Assume money and interest belong to FINANCE; river, water and shore belong to RIVER. Ignore all other tokens as unassigned. ### Step 2. Count support | Candidate bank sense | Matching clues | Score | | --- | --- | --- | | Financial institution | money, interest | 2 | | River edge | none | 0 | ### Step 3. Compare 2 > 0, so choose the financial-institution sense under this stated assignment. ### Final answer **bank = financial institution.** The score 2 versus 0 is conditional on the declared illustrative thesaurus, not supplied numerical data. **Source / convention note.** The original gives a sentence but no thesaurus/category mapping, so a unique numerical score cannot be recovered. --- ## TEE · Q3a · Hobbs algorithm and limitations Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f23--q3a/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-f23/q3a Previous examination · complete worked answers **Question. Explain Hobbs’ algorithm with an example and state its limitations.** [10 marks] ### Answer **Hobbs’ algorithm is a syntax-based procedure that searches constituent parse trees in a fixed order for a compatible noun-phrase antecedent of a pronoun.** The tree determines which candidates are proposed first. Agreement and other relevant constraints determine whether a proposed candidate can be accepted. It is not simply the rule “choose the closest noun”. A nearby noun may have the wrong number, may be syntactically ineligible, or may be visited after a structurally more prominent noun. The algorithm produces a preference under the supplied parse; ambiguous language can still require context. ### The tree vocabulary you need | Term | Meaning | Example | | --- | --- | --- | | NP | Noun phrase: a phrase referring to an entity | the student; Anita; they | | S | A sentence/clause node | the node containing a subject and predicate | | VP | Verb phrase | told Ravi that she should revise | | X | The NP or S node currently being searched | An algorithm label, not a word | | Path p | The branch travelled upward from the pronoun/previous X | Used to distinguish left and right subtrees | | N-bar (N′) | The noun-centred part of an NP in a grammar using that label | The nominal-child test depends on the given tree | | Breadth-first | Visit shallower nodes before deeper ones | Level 1 before level 2 | | Left-to-right | Visit equally deep nodes in their written/tree order | Earlier left branch before right branch | Locate pronoun NP → Climb to NP/S → Propose candidates in tree order → Check compatibility → Accept or continue searching ### The complete search procedure #### Step 1. Start at the pronoun Locate the NP immediately dominating the pronoun. “Dominating” means containing it as a descendant in the tree. #### Step 2. Climb to the first NP or S Move upward to the first NP or S above that starting NP. Call this node X. Keep the upward path p. #### Step 3. Search the left branches initially Below X, traverse branches to the left of p breadth-first and left-to-right. At this initial search, propose an NP only if another NP or S lies between that candidate and X. Test proposed candidates for the required agreement and constraints. #### Step 4. If X is the highest sentence node, inspect earlier sentences Search previous sentence trees from most recent to oldest. Within each, use breadth-first, left-to-right order to propose NPs. Accept the first eligible compatible candidate, or report that no candidate was found in the available context. #### Step 5. Otherwise climb again Move from X to the next NP or S above it. Record this new X and the path followed during the climb. #### Step 6. Apply the higher-NP test If the new X is an NP, propose X when the path did not pass through its nominal N-bar child. Use the given parse labels; do not invent an N-bar branch missing from the question's structure. #### Step 7. Search its left branches Traverse branches below the new X to the left of the path, breadth-first and left-to-right, proposing NPs. The special intervening-NP/S restriction belongs to the initial search in Step 3. #### Step 8. If X is S, search the permitted right branches Search branches to the right of the path breadth-first and left-to-right. Do not descend below an encountered NP or S. Propose eligible NP candidates there. #### Step 9. Repeat or stop Return to the highest-S check and continue the climb/search procedure until a compatible candidate is accepted or the available trees are exhausted. Record the candidate order rather than only writing an unexplained name. ### Why the first left search has an extra condition In a simple clause such as “John saw him”, the subject NP John is directly under the clause S. There is no extra NP or S between John and that S. The initial-search restriction therefore prevents casually selecting John as the ordinary local antecedent of him. Compare the reflexive “John saw himself”, which is a different grammatical case; do not treat every pronoun type as interchangeable. ### Worked tree 1 · move to the previous sentence **Question. Apply Hobbs’ search to “The student arrived. She smiled.” Assume the student is compatible with she.** #### Given Previous tree: [S [NP The student] [VP arrived]]. Current tree: [S [NP She] [VP smiled]]. #### To find The selected antecedent and the search path. #### Method and why it applies Use the supplied constituent trees. The current sentence has no earlier candidate, so the highest-S rule activates previous-sentence search. ### Solution #### Step 1. Draw both sentence structures ``` Previous sentence Current sentence S S ├── NP: The student ├── NP: She ← start └── VP: arrived └── VP: smiled ``` #### Step 2. Locate and climb Start at NP She. Its first higher NP/S is the current S. Set X to that S. #### Step 3. Search left of the path She is the leftmost subject. No NP lies to its left in the current sentence. #### Step 4. Use the highest-S rule This X is the highest S. Inspect the immediately preceding sentence tree. #### Step 5. Propose its first NP Breadth-first, left-to-right finds NP The student. #### Step 6. Check compatibility and accept The question stipulates that the student agrees with she. Accept this candidate. **Answer.** She refers to the student under the supplied assumptions. ### Worked tree 2 · reject an incompatible subject **Question. Resolve they in “Riya spoke to the students. They laughed.” Use a previous-sentence search and interpret they as plural.** #### Given Previous tree has subject NP Riya, then VP spoke [PP to [NP the students]]. Current subject is NP They. #### To find The candidate order and accepted antecedent. #### Method and why it applies Breadth-first search favours a shallow subject, but agreement must still be applied; a first candidate is not automatically a valid answer. ### Solution #### Step 1. Draw the useful part of the previous tree ``` S ├── NP: Riya ← candidate 1 └── VP ├── V: spoke └── PP ├── P: to └── NP: the students ← candidate 2 ``` #### Step 2. Search the current sentence From NP They climb to its S; there is no earlier NP in “They laughed”. This is the highest S, so move to the previous tree. #### Step 3. Visit Riya first Riya is the shallower leftmost NP. Under the stated plural interpretation, this singular candidate fails number agreement. #### Step 4. Visit the students Continue breadth-first. The next relevant NP is the students. #### Step 5. Apply number agreement Students is plural and is an eligible candidate. It matches plural they. #### Step 6. State the resolution Accept the students. The method did not simply accept the first noun it saw. **Answer.** They → the students. Candidate order: Riya rejected; the students accepted. ### Worked tree 3 · climb out of an embedded clause **Question. Resolve she in “Anita told Ravi that she should revise.” Assume Anita is compatible with she and Ravi is not. Use the displayed parse.** #### Given The pronoun is the subject of an embedded S inside the main VP. #### To find The preferred antecedent found by the climb and left-branch search. #### Method and why it applies The embedded clause has no left candidate. Climbing to the main S exposes its subject Anita as the first suitable higher-level candidate. ### Solution #### Step 1. Show the clause structure ``` S (main) ├── NP: Anita ← accepted candidate └── VP ├── V: told ├── NP: Ravi └── SBAR: that └── S (embedded) ← first X ├── NP: she ← start └── VP: should revise ``` #### Step 2. Start inside the embedded clause From NP she climb to the embedded S. There is no candidate to the left of she within this embedded S. #### Step 3. Climb to the next NP/S The embedded S is not the highest S. Move upward through SBAR and VP to the main S. These intermediate nodes are not NP/S search nodes. #### Step 4. Search left of the upward path At the main S, the subject NP Anita is a shallow left candidate. The object Ravi is deeper within the VP. Breadth-first order proposes Anita first. #### Step 5. Check the stated compatibility Anita is compatible with she by the question's assumption. Accept it; no need to choose a later incompatible candidate. #### Step 6. Give the result and its limit This is the algorithm's result under the displayed parse and agreement assumptions. It does not prove the intention of every speaker using an otherwise ambiguous sentence. **Answer.** She → Anita under the stated tree and compatibility information. ### Advantages, limitations and a complete exam answer | Advantages | Limitations | | --- | --- | | Explicit candidate order makes the decision traceable. | Wrong parse structure can produce the wrong search order. | | Requires no labelled coreference-training corpus for its basic rules. | World knowledge and discourse intention may override syntactic preference. | | Uses structure rather than only surface distance. | Some ambiguous, long-distance or unusual constructions need additional treatment. | For a longer answer, write the definition, inputs, tree notation, ordered algorithm, one labelled tree with a candidate table, final antecedent, and limitations. In a worked question show separately: candidate → why proposed → agreement result → accept/reject. Do not equate the selected candidate with a logically guaranteed interpretation when the sentence is ambiguous. **Yaad rakho:** Tree order candidate deta hai; agreement usko allow/reject karta hai. Nearest noun automatically answer nahi. --- ## TEE · Q3b · Stemming and lemmatization Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f23--q3b/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-f23/q3b Previous examination · complete worked answers **Question. Discuss the need for stemming, list different types of stemming, and differentiate stemming from lemmatization.** [10 marks] ### Answer Stemming merges inflected/derived surface variants for matching and reduces vocabulary size. Types include rule/affix-removal stemmers (Porter, Snowball, Lancaster), dictionary/lookup stemmers, statistical stemmers and hybrid methods. More aggressive reduction increases both merging and overstemming risk. **Stemming** applies string-based rules, often stripping suffixes, to reduce related forms to a stem. The result need not be a valid dictionary word. **Lemmatization** uses vocabulary and morphological analysis, often with POS information, to return a dictionary lemma. | Criterion | Stemming | Lemmatization | | --- | --- | --- | | Basis | Surface form/suffix rules | Lexicon and morphological/POS analysis | | Output | May be nonword, e.g. studi | Dictionary form, e.g. study | | Context/POS | Usually little or none | Often important | | Irregular forms | May miss went→go | Can map went→go with verb analysis | | Cost | Generally simpler | Usually more linguistic processing | Porter applies a sequence of suffix-removal rules subject to conditions; Snowball provides related language-specific stemming algorithms; Lancaster is often more aggressive; regex stemming uses explicit patterns; Lovins is another established stemming approach. Exact output depends on the algorithm and implementation. ### Three worked reductions 1. **studies, studying:** an English stemmer may yield studi/study depending on the rule sequence; verb lemmatization aims at study. State the exact algorithm if exact outputs are asked. 2. **better:** adjective-aware lemmatization can map it to good; naïve suffix removal cannot infer that irregular relation. 3. **saw:** as a past-tense verb it can become see; as a cutting-tool noun it remains saw. POS and context matter. **Over-stemming** merges forms that should remain distinct; **under-stemming** fails to merge related forms. Normalization can increase recall in search while losing useful distinctions, so it is a task-dependent choice. Stem = rules se cut; lemma = dictionary form, meaning/POS ka dhyan. --- ## TEE · Q4a · Lexical and vector semantics Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f23--q4a/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-f23/q4a Previous examination · complete worked answers **Question. Explain lexical semantics and vector semantics with suitable examples.** [10 marks] ### Answer **A lexical relation connects words or senses by a meaning relationship.** State the relation and its direction before giving the example. | Relation | Exam definition | Examples and direction | | --- | --- | --- | | Homonymy | The same form has unrelated meanings. | bank: financial institution / river edge; bat: animal / sports implement | | Polysemy | One lexical item has multiple related senses. | head: body part / head of a department; paper: material / article | | Synonymy | Different words have identical or closely similar meanings in a specified sense. | sofa / couch; automobile / car | | Antonymy | Words have opposite meanings. | hot / cold; alive / dead; buy / sell | | Hyponymy | A more specific concept is a type of a more general one. | rose is a flower; car is a vehicle | | Hypernymy | A broader concept includes a more specific one. | flower is a hypernym of rose | | Meronymy | A concept is a part of another concept. | wheel is part of car; finger is part of hand | | Holonymy | A concept is the whole containing a part. | car is a holonym of wheel | ### How to distinguish the commonly confused pairs For homonymy versus polysemy, ask whether the senses share an identifiable meaning. A river bank and a financial bank are unrelated. A financial bank and a blood bank share a repository/storage idea, illustrating polysemy in the supplied notes. For hyponymy versus meronymy, use the sentence test: **“X is a kind of Y”** indicates hyponymy; **“X is part of Y”** indicates meronymy. A wheel is not a type of car. | Expression | Correct relation | Reason | | --- | --- | --- | | sparrow → bird | Hyponymy | A sparrow is a kind of bird | | bird → sparrow | Hypernymy | Bird is the broader category | | keyboard → computer | Meronymy | Keyboard is a component | | computer → keyboard | Holonymy | Computer is the containing whole | | mouth of a person / mouth of a river | Polysemy | Both senses share the idea of an opening; the river use is a related extension | Synonyms are not always interchangeable in every context or register. “Big mistake” and “large mistake” can sound different even though big and large overlap in meaning. Antonyms can be gradable (hot/cold), complementary (alive/dead) or relational converses (buy/sell). HYPO = specific type; HYPER = bigger category. MERO = part; HOLO = whole. **The distributional hypothesis states that words occurring in similar contexts tend to have similar meanings or semantic behaviour.** “Cat” and “dog” often occur near pet, feed and animal. Their context patterns can therefore be represented and compared numerically. A **vector** is an ordered list of numbers. Each position must have a fixed meaning. For context order [pet, food, road], cat=[4,3,0] means four co-occurrences with pet, three with food, none with road. A **matrix** stacks vectors as rows in a table. A **word–context co-occurrence matrix** records how often a target word occurs with each context word under a specified context definition. A window may be a sentence, document, or a fixed number of neighbouring tokens. State its width, direction and whether targets count themselves. Choose target words → Choose context/window → Count co-occurrences → Store each row as a vector → Compare vectors **Question. Build the symmetric co-occurrence matrix for “cats eat fish” using a window of one token on either side, no self-counts and vocabulary [cats,eat,fish].** #### Given One sentence, one occurrence of each token. Left and right adjacent tokens are contexts. #### To find A 3×3 target-by-context count matrix. #### Formula and why it applies $$ X_{ij} $$ counts the number of times context word $$ j $$ occurs within the chosen window of target $$ i $$ . #### Step 1. Target cats Only eat is adjacent. Row cats = [0,1,0]. #### Step 2. Target eat cats is left and fish is right. Row eat = [1,0,1]. #### Step 3. Target fish Only eat is adjacent. Row fish = [0,1,0]. #### Step 4. Assemble the matrix | Target / context | cats | eat | fish | | --- | --- | --- | --- | | cats | 0 | 1 | 0 | | eat | 1 | 0 | 1 | | fish | 0 | 1 | 0 | **Answer.** Rows are [0,1,0], [1,0,1], [0,1,0]. The symmetry follows from the symmetric window. **Question. Build a target–context matrix for sentences “cats eat fish” and “dogs eat fish”. Target words are cats and dogs; context columns [eat,fish]; use the entire sentence excluding the target.** #### Given Each sentence contains one target. Count context token occurrences. #### To find Two context vectors. #### Formula and why it applies Count each context word in every sentence containing the target. #### Step 1. Count cats contexts In its sentence: eat occurs once, fish once → [1,1]. #### Step 2. Count dogs contexts In its sentence: eat occurs once, fish once → [1,1]. #### Step 3. Compare Both targets have identical observed context patterns in this very small corpus. **Answer.** cats=[1,1], dogs=[1,1]. This corpus supplies no context evidence to distinguish them. **Question. For “a b a”, use a right-only window of one token. Build the matrix with rows/columns [a,b].** #### Given Count directed target→next-word events, not both directions. #### To find The co-occurrence matrix. #### Formula and why it applies $$ X_{ij}=C(\text{target }i\text{ immediately followed by }j) $$ . #### Step 1. First target a Next token is b → add 1 to X(a,b). #### Step 2. Target b Next token is a → add 1 to X(b,a). #### Step 3. Final a No following token → no addition. #### Step 4. Write matrix | Target / next | a | b | | --- | --- | --- | | a | 0 | 1 | | b | 1 | 0 | **Answer.** a=[0,1], b=[1,0]. A different context window would produce a different matrix. Word ka meaning: uske aas-paas kaunse words kitni baar aaye, uski number-list. --- ## TEE · Q4b · Three-document TF–IDF Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f23--q4b/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-f23/q4b Previous examination · complete worked answers **Question. Calculate TF–IDF for: (1) Inflation increased unemployment (2) Company increased sales (3) Fear increased pulse.** [10 marks] ### Answer ### Given | Document | Original text | | --- | --- | | 1 | Inflation increased unemployment | | 2 | Company increased sales | | 3 | Fear increased pulse | Stopwords: none supplied ### To find BoW, IDF and TF–IDF ### Formula: which one and why Use lowercase alphabetic tokens and exactly the stated stoplist. If no stoplist is supplied, retain function words. Adopt raw-count TF and natural-log IDF; the paper does not prescribe another TF/log convention. $$ \mathrm{TFIDF}(w,d)=c(w,d)\ln\frac{N}{\mathrm{df}(w)} $$ A term shared by all N documents has IDF ln(1)=0. Counts and TF–IDF weights are different representations. ### Step 1. Tokenize each document | Document | Tokens | | --- | --- | | 1 | inflation, increased, unemployment | | 2 | company, increased, sales | | 3 | fear, increased, pulse | ### Step 2. Fix the vocabulary order company, fear, increased, inflation, pulse, sales, unemployment ### Step 3. Count occurrences (Bag of Words) | Document | company | fear | increased | inflation | pulse | sales | unemployment | | --- | --- | --- | --- | --- | --- | --- | --- | | 1 | 0 | 0 | 1 | 1 | 0 | 0 | 1 | | 2 | 1 | 0 | 1 | 0 | 0 | 1 | 0 | | 3 | 0 | 1 | 1 | 0 | 1 | 0 | 0 | ### Step 4. Calculate document frequency and IDF | Term | df | N / df | ln(N/df) | | --- | --- | --- | --- | | company | 1 | 3 | 1.098612289 | | fear | 1 | 3 | 1.098612289 | | increased | 3 | 1 | 0.000000000 | | inflation | 1 | 3 | 1.098612289 | | pulse | 1 | 3 | 1.098612289 | | sales | 1 | 3 | 1.098612289 | | unemployment | 1 | 3 | 1.098612289 | ### Step 5. Multiply each TF by its IDF | Document | company | fear | increased | inflation | pulse | sales | unemployment | | --- | --- | --- | --- | --- | --- | --- | --- | | 1 | 0.000000 | 0.000000 | 0.000000 | 1.098612 | 0.000000 | 0.000000 | 1.098612 | | 2 | 1.098612 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 1.098612 | 0.000000 | | 3 | 0.000000 | 1.098612 | 0.000000 | 0.000000 | 1.098612 | 0.000000 | 0.000000 | ### Final answer The final table is the TF–IDF matrix in the declared vocabulary order. A different explicitly stated log base rescales all IDFs; length-normalized TF changes each document scale. --- ## TEE · Q5a · Word embeddings and CBOW Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f23--q5a/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-f23/q5a Previous examination · complete worked answers **Question. Elaborate word embeddings. Explain the steps in CBOW with a suitable diagram.** [10 marks] ### Answer **A sparse representation** has many zero entries and often one coordinate per vocabulary item. BoW and TF–IDF are common sparse document representations. **A dense embedding** has comparatively few learned real-valued coordinates; one word's properties are spread across many coordinates. | Representation | What coordinates mean | Strength | Limitation | | --- | --- | --- | --- | | One-hot | One identity position per vocabulary word | Unambiguous ID | Different words are orthogonal; no learned similarity | | Count-based context vector | Context-word co-occurrence counts | Transparent corpus evidence | Large, sparse; frequent words may dominate | | TF–IDF document vector | Weighted vocabulary terms | Useful retrieval weighting | Word order and contextual meaning are limited | | Dense word embedding | Learned latent features | Compact semantic/syntactic relationships | Coordinates are not generally human-labelled features | **Distributional** describes learning from patterns of linguistic context. **Distributed** describes representing an item across multiple coordinates rather than a single active identity bit. These are related but different ideas. A one-hot ID alone does not learn distributional similarity. ### Model design choices Choose contexts (document, sentence, fixed window), weighting (raw counts, normalization, TF–IDF-like weighting or association scores), dimensionality reduction or learned embeddings, and a similarity measure. Cosine compares direction and is less affected by length than a raw dot product; Euclidean distance also reflects magnitude. Sparse = bahut zero. Dense = chhoti learned number-list. Similar context se useful similarity seekhte hain. **Word2Vec learns dense word embeddings by predicting words from nearby context.** Its two common architectures are CBOW and skip-gram. A vocabulary ID or one-hot vector is the input representation; the learned embedding table converts that identity into a short trainable vector. | Architecture | Input | Prediction | Memory cue | | --- | --- | --- | --- | | CBOW | Surrounding context words | Centre/target word | Context → centre | | Skip-gram | Centre/target word | Surrounding words | Centre → context | CBOW: the + sat → Average context embeddings → Predict: cat Skip-gram: cat → Embedding of cat → Predict: the and sat For vocabulary size $$ V $$ and embedding dimension $$ d $$ , an input one-hot row is $$ 1\times V $$ ; the input embedding matrix is $$ V\times d $$ ; the hidden row is $$ 1\times d $$ ; the output matrix is $$ d\times V $$ ; output scores are $$ 1\times V $$ . Multiplying a one-hot row by the embedding matrix simply selects one row. $$ P(w_o\mid w_i)=\frac{\exp(u_{w_o}^{T}v_{w_i})}{\sum_{w\in V}\exp(u_w^{T}v_{w_i})}. $$ $$ v_{w_i} $$ is the input word embedding, $$ u_w $$ an output-context embedding, and the dot product is a compatibility score. Softmax turns all scores into probabilities that sum to one. Training increases the probability of observed context words. Negative sampling or hierarchical softmax can reduce computation in larger vocabularies; use the algorithm specified by the question. **Question. For the sentence “the cat sat on mat”, generate CBOW and skip-gram training data for centre word sat with a window of one token on each side.** #### Given Tokens indexed 1 to 5: the, cat, sat, on, mat. Centre = sat. #### To find The CBOW example and the skip-gram pairs. #### Formula and why it applies CBOW predicts the centre from its neighbours; skip-gram predicts each neighbour from the centre. #### Step 1. Identify neighbours Left neighbour = cat; right neighbour = on. “the” and “mat” are outside this window. #### Step 2. CBOW example Input [cat,on] → target sat. #### Step 3. Skip-gram examples (sat,cat) and (sat,on), with first item as input and second as predicted context. **Answer.** CBOW: [cat,on]→sat. Skip-gram: sat→cat and sat→on. **Question. The embeddings of context words cat and on are [1,2] and [3,0]. Compute the CBOW hidden vector using mean aggregation. Output vectors for [sat,ran] are [1,0] and [0,1]. Compute softmax probabilities.** #### Given Context embeddings [1,2], [3,0]; mean aggregation; two candidate output vectors. #### To find The hidden vector, logits and probabilities. #### Formula and why it applies Mean context vector $$ h=(v_1+v_2)/2 $$ . Logit $$ z_w=u_w^Th $$ . Softmax $$ P(w)=e^{z_w}/\sum_j e^{z_j} $$ . #### Step 1. Add coordinates $$ [1+3,\,2+0]=[4,2] $$ . #### Step 2. Divide by number of words $$ h=[4/2,2/2]=[2,1] $$ . #### Step 3. Compute logits $$ z_{sat}=1(2)+0(1)=2 $$ . $$ z_{ran}=0(2)+1(1)=1 $$ . #### Step 4. Exponentials $$ e^2\approx7.389056 $$ , $$ e^1\approx2.718282 $$ ; sum $$ \approx10.107338 $$ . #### Step 5. Divide $$ P(sat)\approx0.731059 $$ , $$ P(ran)\approx0.268941 $$ . **Answer.** h=[2,1]; sat is more probable (0.731059). **Question. A skip-gram centre embedding is v=[1,−1]. Output embeddings for [eat,sleep,drive] are [1,1], [0,1], [1,0]. Calculate the predicted context probabilities.** #### Given One centre vector and three output vectors; full softmax over these three words. #### To find Three logits and three probabilities. #### Formula and why it applies $$ z_j=u_j^Tv $$ , then $$ P(j)=e^{z_j}/\sum_k e^{z_k} $$ . #### Step 1. Eat dot product $$ 1(1)+1(-1)=0 $$ . #### Step 2. Sleep and drive dot products $$ z_{sleep}=0(1)+1(-1)=-1 $$ . $$ z_{drive}=1(1)+0(-1)=1 $$ . #### Step 3. Exponentiate and sum $$ e^0=1 $$ , $$ e^{-1}\approx0.367879 $$ , $$ e^1\approx2.718282 $$ . Sum $$ \approx4.086161 $$ . #### Step 4. Normalize $$ P(eat)\approx0.244728 $$ ; $$ P(sleep)\approx0.090031 $$ ; $$ P(drive)\approx0.665241 $$ . **Answer.** drive has the highest probability in this toy learned model. **Limitations:** a standard static Word2Vec vector merges a word's senses; it has limited representation of subword morphology and out-of-vocabulary words; its local prediction objective does not directly fit a full global co-occurrence table. Similarity is learned from data and is not a guarantee of factual or causal understanding. CBOW = neighbours se missing centre. Skip-gram = centre se neighbours. --- ## TEE · Q5b · HMM tables and given path Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f23--q5b/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-f23/q5b Previous examination · complete worked answers **Question. For the four tagged sentences below, construct transition (3 marks) and emission (3 marks) tables. Calculate the path Geeta/N will/M spot/V Rita/N, given initial P(N)=0.5, P(M)=0.25, P(V)=0.25 (4 marks).** [10 marks] ### Answer ### Given | Sentence | Given tags | | --- | --- | | Rita can teach Will | N M V N | | Spot will see Will | N M V N | | Will Rita spot Geeta | N N V N | | Geeta will pat Spot | N M V N | Required path: Geeta will spot Rita / N M V N. Given initial probabilities: {'N': Fraction(1, 2), 'M': Fraction(1, 4), 'V': Fraction(1, 4)} ### To find Emission and transition tables and the specified tag-path joint probability ### Formula: which one and why $$ P(w\mid t)=\frac{c(t,w)}{c(t)} $$ $$ P(t_j\mid t_i)=\frac{c(t_i,t_j)}{\sum_kc(t_i,t_k)} $$ The second formula is the declared no-END convention. Reset at sentence boundaries; never create a transition from the last word of one training sentence to the first of the next. ### Step 1. Count emissions | Tag | Total occurrences | Rita | can | teach | Will | Spot | will | see | spot | Geeta | pat | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | N | 9 | 2 | 0 | 0 | 3 | 2 | 0 | 0 | 0 | 2 | 0 | | M | 3 | 0 | 1 | 0 | 0 | 0 | 2 | 0 | 0 | 0 | 0 | | V | 4 | 0 | 0 | 1 | 0 | 0 | 0 | 1 | 1 | 0 | 1 | ### Step 2. Divide each emission by its tag count | Tag | Rita | can | teach | Will | Spot | will | see | spot | Geeta | pat | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | N | 2/9 | 0 | 0 | 1/3 | 2/9 | 0 | 0 | 0 | 2/9 | 0 | | M | 0 | 1/3 | 0 | 0 | 0 | 2/3 | 0 | 0 | 0 | 0 | | V | 0 | 0 | 1/4 | 0 | 0 | 0 | 1/4 | 1/4 | 0 | 1/4 | ### Step 3. Count transitions inside each sentence | From | Outgoing transitions | Sentence-final count | N | M | V | | --- | --- | --- | --- | --- | --- | | N | 5 | 4 | 1 | 3 | 1 | | M | 3 | 0 | 0 | 0 | 3 | | V | 4 | 0 | 4 | 0 | 0 | ### Step 4. Convert to a transition table | From / to | N | M | V | | --- | --- | --- | --- | | N | 1/5 | 3/5 | 1/5 | | M | 0 | 0 | 1 | | V | 1 | 0 | 0 | Here sentence-final tags are excluded from the no-END outgoing denominator. If END is explicitly modeled, divide all tag→tag and tag→END counts by the total tag count instead; the row then includes END. Do not mix denominators. ### Step 5. Multiply the given path, in order | Factor | Probability | Running product | | --- | --- | --- | | Initial N | 1/2 | 1 × 1/2 = 1/2 | | Emit Geeta | 2/9 | 1/2 × 2/9 = 1/9 | | Transition N→M | 3/5 | 1/9 × 3/5 = 1/15 | | Emit will | 2/3 | 1/15 × 2/3 = 2/45 | | Transition M→V | 1 | 2/45 × 1 = 2/45 | | Emit spot | 1/4 | 2/45 × 1/4 = 1/90 | | Transition V→N | 1 | 1/90 × 1 = 1/90 | | Emit Rita | 2/9 | 1/90 × 2/9 = 1/405 | ### Final answer The two displayed probability tables are the estimated HMM. Zero means unobserved without smoothing, not logically impossible. **No-END path joint probability = 1/405 = 0.00246913580247.** If a normalized END-state model is explicitly adopted instead, this complete terminated sequence has probability 4/6561 = 0.000609663161103. These are different model conventions, not contradictory answers. --- ## TEE · Q6a · Text classification pipeline Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f23--q6a/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-f23/q6a Previous examination · complete worked answers **Question. Explain the text-classification pipeline in detail with a suitable block diagram.** [10 marks] ### Answer **Text classification** assigns a document to one or more predefined categories. Spam detection usually uses two classes; news classification may use several topics. In single-label multiclass classification, choose one of several categories. In multilabel classification, several labels may apply to the same document. Define labels → Prepare labelled examples → Train representation + model → Evaluate on unseen text → Predict new labels ### An exam-ready pipeline | Stage | What you do | Why it matters | | --- | --- | --- | | Task definition | Decide what counts as spam or which news topics exist. | Unclear labels create inconsistent training data. | | Data collection and annotation | Collect representative examples and assign correct labels. | The model learns from these examples. | | Split data | Separate training, validation and test data. | Test examples must not influence training choices. | | Preprocess | Tokenize and normalise where suitable; retain meaningful negation. | Reduce irrelevant variation without deleting signal. | | Represent | Use BoW, TF-IDF or learned embeddings. | Models need numerical features. | | Train | Fit Naive Bayes, logistic regression or another appropriate model. | Learn how features predict categories. | | Evaluate | Measure relevant precision, recall and F1 on held-out examples. | Accuracy alone may hide poor minority-class performance. | | Deploy and monitor | Apply the same preprocessing and monitor changing language. | Incoming data can differ from old examples. | Naive Bayes is especially useful for hand calculations. With a bag-of-words representation, its class score is the prior multiplied by the conditional probability of each token; repeated tokens repeat the factor. The “naive” assumption is conditional independence of features given the class, not that words are independent in real language. $$ s(c)=P(c)\prod_{w\in V}P(w\mid c)^{C(w,d)} $$ $$ P(w\mid c)=\frac{C(w,c)+1}{N_c+|V|} $$ C(w,d) counts word w in document d. C(w,c) counts it in training documents belonging to class c. Nc is the total token count for class c. Add-one smoothing adds one to every vocabulary count and therefore adds |V| to the denominator. This is the same arithmetic taught for WSD, with document classes instead of senses. ### Worked classification · two-token message A spam classifier has vocabulary {win, prize, meeting}. Priors are P(spam)=P(ham)=0.5. Spam token counts are [3,2,0]; ham counts are [0,0,5]. With add-one smoothing, classify “win prize”. **Given** |V|=3; N(spam)=5; N(ham)=5; message has win once and prize once. **To find** Class scores and predicted label. ### Formula and why it applies $$ s(c)=P(c)P(\text{win}\mid c)P(\text{prize}\mid c) $$ $$ P(w\mid c)=\frac{C(w,c)+1}{N_c+|V|} $$ The model is multinomial Naive Bayes; multiply the two token likelihoods and the class prior. ### Solution #### Step 1. Compute the smoothed denominators $$ 5+3=8 $$ #### Step 2. Compute spam likelihoods $$ P(\text{win}\mid S)=\frac{3+1}{8}=\frac12 $$ $$ P(\text{prize}\mid S)=\frac{2+1}{8}=\frac38 $$ #### Step 3. Compute ham likelihoods $$ P(\text{win}\mid H)=P(\text{prize}\mid H)=\frac{0+1}{8}=\frac18 $$ #### Step 4. Multiply the spam score $$ s(S)=\frac12\times\frac12\times\frac38=\frac{3}{32}=0.09375 $$ #### Step 5. Multiply the ham score $$ s(H)=\frac12\times\frac18\times\frac18=\frac{1}{128}=0.0078125 $$ #### Step 6. Compare 0.09375 > 0.0078125. These are unnormalised scores; the label is chosen by the larger score. Predicted label: spam. **Yaad rakho:** Class prior × har word ki class-likelihood; sabse bada score wins. --- ## TEE · Q6b · VSM, TF and IDF in retrieval Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f23--q6b/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-f23/q6b Previous examination · complete worked answers **Question. Define Vector Space Model, Term Frequency and Inverse Document Frequency with reference to information retrieval.** [10 marks] ### Answer **A vocabulary** is the ordered collection of distinct terms allowed by a representation. Its order defines what every coordinate means. A **vector** is simply an ordered number-list. A **document–term matrix** puts one document in each row and one vocabulary term in each column. With vocabulary [cat,dog,fish], the count vector [2,0,1] means cat appears twice, dog zero times and fish once. With [dog,cat,fish], the same text becomes [0,2,1]. A vector without its vocabulary order is incomplete. | Document | cat | dog | fish | | --- | --- | --- | --- | | cat cat fish | 2 | 0 | 1 | | dog fish | 0 | 1 | 1 | **Nominal categories** have no inherent order, such as cities. **Ordinal categories** do have an order, such as low < medium < high. Assigning arbitrary word IDs does not create meaningful numerical distances between words. Vector ka har box ek fixed word ka hai. Pehle column order fix, phir numbers. **TF–IDF weights a term highly when it is frequent in a particular document but uncommon across the corpus.** The supplied Unit 2 slides use normalized term frequency and base-10 inverse document frequency: $$ \operatorname{TF}(t,d)=\frac{C(t,d)}{N_d} $$ $$ \operatorname{IDF}(t)=\log_{10}\frac{D}{\operatorname{df}(t)} $$ $$ \operatorname{TFIDF}(t,d)=\operatorname{TF}(t,d)\operatorname{IDF}(t). $$ $$ N_d $$ is the document's token count after the specified preprocessing; $$ D $$ is number of documents; $$ \operatorname{df}(t) $$ is the number of documents containing the term at least once. A word repeated ten times in one document contributes one to document frequency. **Why the log?** It compresses the ratio's scale. If a term occurs in all documents, $$ D/df=1 $$ , so IDF=0 under this unsmoothed convention. Other libraries use smoothed IDF, different TF or normalization. Use the question's formula; do not silently substitute a library default. **Question. A corpus contains 10 documents. The word model appears in 2 documents and occurs 3 times in a 12-token document. Compute TF, IDF and TF–IDF using the slide convention.** #### Given D=10, df(model)=2, C(model,d)=3, Nd=12; log base 10. #### To find The TF–IDF weight. #### Formula and why it applies $$ TF=C/N_d $$ , $$ IDF=\log_{10}(D/df) $$ , then multiply. #### Step 1. TF $$ 3/12=0.25 $$ . #### Step 2. Document ratio $$ 10/2=5 $$ . #### Step 3. IDF $$ \log_{10}5\approx0.698970 $$ . #### Step 4. Weight $$ 0.25\times0.698970\approx0.174743 $$ . **Answer.** TF=0.25, IDF≈0.698970, TF–IDF≈0.174743. **Question. A term occurs 4 times in a 20-token document and is present in all 8 corpus documents. Calculate its TF–IDF.** #### Given C=4, Nd=20, D=8, df=8. #### To find Its weight under unsmoothed base-10 IDF. #### Formula and why it applies $$ TF=C/N_d $$ , $$ IDF=\log_{10}(D/df) $$ . #### Step 1. TF $$ 4/20=0.2 $$ . #### Step 2. IDF $$ \log_{10}(8/8)=\log_{10}1=0 $$ . #### Step 3. Multiply $$ 0.2\times0=0 $$ . **Answer.** Weight 0: this term does not distinguish documents under this convention. **Question. Find TF–IDF vectors for D1=“cat cat dog”, D2=“dog fish”, D3=“cat fish fish”, using [cat,dog,fish], normalized TF and log10 IDF.** #### Given No further preprocessing. Each term occurs in exactly two of the three documents. #### To find All three vectors. #### Formula and why it applies First count df by documents, then compute each document's TF, then multiply by its term's IDF. #### Step 1. Common IDF $$ D=3 $$ , $$ df(cat)=df(dog)=df(fish)=2 $$ . Each IDF $$ =\log_{10}(3/2)\approx0.176091 $$ . #### Step 2. D1 term frequencies Length 3 → $$ [2/3,1/3,0] $$ . #### Step 3. D1 weights $$ [(2/3)0.176091,(1/3)0.176091,0]\approx[0.117394,0.058697,0] $$ . #### Step 4. D2 weights Length 2; TF $$ [0,1/2,1/2] $$ . Vector $$ \approx[0,0.088046,0.088046] $$ . #### Step 5. D3 weights Length 3; TF $$ [1/3,0,2/3] $$ . Vector $$ \approx[0.058697,0,0.117394] $$ . **Answer.** D1≈[0.117394,0.058697,0]; D2≈[0,0.088046,0.088046]; D3≈[0.058697,0,0.117394]. TF: iss document mein kitna. IDF: poore corpus mein kitna rare. Dono multiply. **Information retrieval (IR)** locates and ranks items relevant to an information need, usually expressed as a query. The collection may contain documents, product descriptions or scientific papers. Retrieval estimates relevance; it does not itself establish that every retrieved claim is true. Documents → cleaning → Build index → Query → same representation → Match & rank → Return relevant results ### The inverted index An inverted index maps a term to the documents containing it, often with positions or counts. For D1 “cats drink milk” and D2 “dogs drink water”, the posting list for drink is {D1,D2}, while milk maps only to D1. This avoids rescanning every document for every query. | Term | Posting list | | --- | --- | | cats | D1 | | drink | D1, D2 | | milk | D1 | | dogs | D2 | | water | D2 | ### Matching and ranking Boolean retrieval combines term membership using AND, OR and NOT. Vector-space retrieval represents documents and queries as weighted vectors, for example with TF-IDF, and compares them with cosine similarity. Learned retrieval can use dense embeddings for meaning-based matching. The representation used at query time must be compatible with the document index. ### Search engines and recommendation A search engine receives an explicit query and ranks matching indexed pages. A recommender may derive an implicit query from a user’s liked items or profile and retrieve similar candidates. Both require candidate retrieval and ranking, but the source of the information need differs. ### Evaluate the results For retrieval, TP is the number of retrieved relevant documents, FP retrieved irrelevant documents, and FN relevant documents not retrieved. Precision and recall use the same formulas as classification. Ranking measures also consider the positions of relevant items; a relevant result at rank 1 is more useful than one far below. ### Retrieval evaluation A search system retrieves 8 documents; 6 are relevant. There are 10 relevant documents in the collection. Find precision, recall and F1. **Given** Retrieved=8; relevant retrieved=6; all relevant=10. **To find** P, R and F1. ### Formula and why it applies $$ P=\frac{\text{relevant retrieved}}{\text{retrieved}},\quad R=\frac{\text{relevant retrieved}}{\text{all relevant}} $$ $$ F_1=\frac{2PR}{P+R} $$ The denominator is 8 for precision and 10 for recall because they ask different questions. ### Solution #### Step 1. Identify the confusion counts TP=6; FP=8−6=2; FN=10−6=4. #### Step 2. Precision $$ P=\frac68=0.75 $$ #### Step 3. Recall $$ R=\frac6{10}=0.6 $$ #### Step 4. F1 $$ F_1=\frac{2(0.75)(0.6)}{0.75+0.6}=\frac{0.9}{1.35}=\frac23\approx0.666667 $$ Precision=0.75, recall=0.60, F1≈0.666667. **Yaad rakho:** Precision: jo laaye, unmein kitne sahi? Recall: jo sahi the, unmein kitne laaye? --- ## TEE · Q1a · Normalization changes vocabulary Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r23--q1a/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-r23/q1a Previous examination · complete worked answers **Question. “The vocabulary of a corpus remains the same before and after normalization.” State true or false and justify with an example.** [5 marks] ### Answer **False in general.** Vocabulary is the set of distinct tokens under a chosen tokenizer. In “Cats cats running”, raw case-sensitive vocabulary={Cats,cats,running}, size3. Lowercase merges Cats/cats: {cats,running}, size2. Lemmatization may yield {cat,run}; stopword removal can remove entries. For an already normalized corpus the set may happen to remain unchanged, so the correct claim is that normalization can change it, not that it always must. --- ## TEE · Q1b · Good corpus characteristics Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r23--q1b/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-r23/q1b Previous examination · complete worked answers **Question. State the characteristics of a good corpus.** [5 marks] ### Answer A useful corpus is suited to its task and population: representative genres/languages, adequate size and diversity, balanced or intentionally documented sampling, reliable consistent annotations, clean encoding/tokenization, metadata and provenance, legal/ethical permission, documented train/test splits, and versioned reproducibility. “Large” alone does not imply representative. Example: a medical-tagging corpus needs clinical language and negation annotation; only movie reviews would be unsuitable. --- ## TEE · Q1c · Word2Vec versus Doc2Vec Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r23--q1c/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-r23/q1c Previous examination · complete worked answers **Question. Differentiate Word2Vec and Doc2Vec.** [5 marks] ### Answer | Aspect | Word2Vec | Doc2Vec | | --- | --- | --- | | Representation | One learned vector per word type | Learns document/paragraph vectors as well as, in variants, word vectors | | Training | CBOW predicts centre from context; Skip-gram predicts context from centre | PV-DM combines document/context to predict words; PV-DBOW predicts sampled words from document ID | | Unit | Word/token type | Sentence, paragraph or document | | New text | Retrieve known word vectors; combine via a separate pooling choice | Infer a new document vector while learned model weights stay fixed | | Use | Word similarity, features | Document similarity, retrieval or classification | Averaging Word2Vec word vectors is a possible document representation, but it is not the Doc2Vec training algorithm. This older paper names Doc2Vec explicitly; the current course names word and sentence embeddings but does not explicitly list Doc2Vec. --- ## TEE · Q1d · Ambiguity in NLP Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r23--q1d/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-r23/q1d Previous examination · complete worked answers **Question. Explain the types of ambiguity in NLP with examples.** [5 marks] ### Answer **Ambiguity** means an expression supports more than one interpretation. Good exam answers name the ambiguity and explain both readings. | Challenge | Example | Why a system can fail | | --- | --- | --- | | Lexical/sense ambiguity | bank: finance or river edge | One written word has several senses | | Syntactic ambiguity | I saw the man with a telescope. | The telescope may modify saw or the man | | Reference ambiguity | Riya told Neha that she won. | She may refer to either person without more context | | Sarcasm | Great, another delay. | Literal positive word conveys frustration | | Negation | The film was not good. | Dropping not reverses meaning | | Domain shift | “Positive” in a medical report | Meaning/use differs by domain | | Code mixing | Kal meeting reschedule karna. | Several languages and spelling conventions | | Noisy text | tmrw, repeated letters, typos | Vocabulary/token boundaries become uncertain | | Low-resource setting | Few labelled examples in a language | Models lack representative training evidence | | Bias and privacy | Skewed data or personal details | Errors and harms can persist in deployed systems | **Homophones** sound alike but may differ in spelling (pair/pear). **Homographs** share spelling but can differ in meaning or pronunciation (lead the team / lead metal). These are different dimensions from syntactic ambiguity. Long-distance dependencies, cultural knowledge, idioms, morphology and multilingual writing systems also complicate processing. For “kick the bucket”, literal word-by-word interpretation can miss the idiomatic meaning. For “The book that the students recommended is useful”, the verb is agrees with book across an intervening phrase. Exam mein challenge ka naam + ek sentence + dono meanings/actual failure likho. **Structural ambiguity** occurs when a sentence has more than one valid parse tree. In “I saw the man with a telescope”, the prepositional phrase “with a telescope” can attach to saw (the instrument I used) or to the man (the man possessing the telescope). The words and their order stay the same, but the grouping changes. | Reading | Attachment | Meaning | | --- | --- | --- | | I [saw the man] [with a telescope] | PP modifies VP | I used a telescope to see the man. | | I saw [the man with a telescope] | PP modifies NP | The man had a telescope. | A parser may need context, probabilities or semantic constraints to choose between trees. A plain CFG describes permitted structures but does not itself assign a probability to each tree. Do not invent rule probabilities for an ordinary CFG question. Lexical ambiguity also matters: book can be a noun or verb. In “Book the flight”, the imperative analysis begins S→VP, followed by VP→V NP. A parser that initially chooses book as a noun may reach a dead end and must try another permitted analysis. **Yaad rakho:** Same words, alag grouping → alag meaning. --- ## TEE · Q2a · Normalize the Tanisha–Simran passage Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r23--q2a/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-r23/q2a Previous examination · complete worked answers **Question. Perform sentence segmentation, word tokenization, stopword removal, lowercasing and stemming on the following paragraph; show vocabulary before and after: “Tanisha and Simran are best friends. They play together with other friends. Tanisha likes to play badminton but Simran prefers to play online games. Tanisha wants to be a badminton player. Simran wants to become an online gamer.”** [10 marks] ### Answer No stoplist/stemmer is specified. For a reproducible answer choose stoplist {and,are,they,with,other,to,but,be,a,an} after lowercase; use the Porter stemmer. Other declared lists produce different valid counts. | Stage | Result | | --- | --- | | Sentence segmentation | Tanisha and Simran are best friends. They play together with other friends. Tanisha likes to play badminton but Simran prefers to play online games. Tanisha wants to be a badminton player. Simran wants to become an online gamer. | | Word tokens (punctuation omitted) | Tanisha, and, Simran, are, best, friends, They, play, together, with, other, friends, Tanisha, likes, to, play, badminton, but, Simran, prefers, to, play, online, games, Tanisha, wants, to, be, a, badminton, player, Simran, wants, to, become, an, online, gamer | | Lowercase | tanisha, and, simran, are, best, friends, they, play, together, with, other, friends, tanisha, likes, to, play, badminton, but, simran, prefers, to, play, online, games, tanisha, wants, to, be, a, badminton, player, simran, wants, to, become, an, online, gamer | | Declared stoplist | a, an, and, are, be, but, other, they, to, with | | After exact stop removal | tanisha, simran, best, friends, play, together, friends, tanisha, likes, play, badminton, simran, prefers, play, online, games, tanisha, wants, badminton, player, simran, wants, become, online, gamer | | Retained word | Porter stem | | --- | --- | | tanisha | tanisha | | simran | simran | | best | best | | friends | friend | | play | play | | together | togeth | | likes | like | | badminton | badminton | | prefers | prefer | | online | onlin | | games | game | | wants | want | | player | player | | become | becom | | gamer | gamer | Full final token sequence: tanisha, simran, best, friend, play, togeth, friend, tanisha, like, play, badminton, simran, prefer, play, onlin, game, tanisha, want, badminton, player, simran, want, becom, onlin, gamer Raw case-sensitive vocabulary (25): Simran, Tanisha, They, a, an, and, are, badminton, be, become, best, but, friends, gamer, games, likes, online, other, play, player, prefers, to, together, wants, with. After lowercase only: 25 distinct words. Final stem vocabulary (15): badminton, becom, best, friend, game, gamer, like, onlin, play, player, prefer, simran, tanisha, togeth, want. **Source / convention note.** Stoplist and stemming algorithm are choices, not supplied numerical givens; both are stated. --- ## TEE · Q2b · Text vectorization alternatives Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r23--q2b/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-r23/q2b Previous examination · complete worked answers **Question. Define word embeddings and explain one-hot, Bag of Words, bag of n-grams and TF–IDF with advantages and disadvantages.** [10 marks] ### Answer **A sparse representation** has many zero entries and often one coordinate per vocabulary item. BoW and TF–IDF are common sparse document representations. **A dense embedding** has comparatively few learned real-valued coordinates; one word's properties are spread across many coordinates. | Representation | What coordinates mean | Strength | Limitation | | --- | --- | --- | --- | | One-hot | One identity position per vocabulary word | Unambiguous ID | Different words are orthogonal; no learned similarity | | Count-based context vector | Context-word co-occurrence counts | Transparent corpus evidence | Large, sparse; frequent words may dominate | | TF–IDF document vector | Weighted vocabulary terms | Useful retrieval weighting | Word order and contextual meaning are limited | | Dense word embedding | Learned latent features | Compact semantic/syntactic relationships | Coordinates are not generally human-labelled features | **Distributional** describes learning from patterns of linguistic context. **Distributed** describes representing an item across multiple coordinates rather than a single active identity bit. These are related but different ideas. A one-hot ID alone does not learn distributional similarity. ### Model design choices Choose contexts (document, sentence, fixed window), weighting (raw counts, normalization, TF–IDF-like weighting or association scores), dimensionality reduction or learned embeddings, and a similarity measure. Cosine compares direction and is less affected by length than a raw dot product; Euclidean distance also reflects magnitude. Sparse = bahut zero. Dense = chhoti learned number-list. Similar context se useful similarity seekhte hain. | Method | Construction and example | Strength | Limitation | | --- | --- | --- | --- | | One-hot | One1in vocabulary coordinate: cat=(1,0,0) | Simple unique identity | Sparse; all distinct words equally distant | | BoW | Counts per vocabulary coordinate | Simple interpretable frequency | Loses order and context | | Bag of n-grams | Counts contiguous word groups, e.g. not good | Retains local order/phrases | Vocabulary grows; sparse unseen phrases | | TF–IDF | Term frequency weighted by inverse document frequency | Downweights ubiquitous terms for retrieval | No contextual meaning; depends on corpus statistics | Dense learned embeddings differ from these sparse count encodings: similarity may reflect learned context. Explain the representation before computing any metric. --- ## TEE · Q3a · Semantics, pragmatics and two ambiguities Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r23--q3a/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-r23/q3a Previous examination · complete worked answers **Question. Differentiate semantics and pragmatics. Explain ambiguity in “Time flies like an arrow” and “He crushed the key to my heart”.** [10 marks] ### Answer **Semantics** concerns conventional meaning and composition; **pragmatics** interprets an utterance using speaker intention and situation. “It is cold here” can semantically report temperature and pragmatically request closing a window. | Sentence | Readings / cause | | --- | --- | | Time flies like an arrow | Ordinary reading: time(N) flies(V), with like an arrow as comparison. Alternative imperative: time(V) the flies(N) as one times an arrow; lexical-category and structural ambiguity. | | He crushed the key to my heart | Literal key/crushing versus figurative destruction of affection or emotional access. Lexical/semantic ambiguity and metaphor need pragmatic context; there is no supplied situation proving a single intended reading. | --- ## TEE · Q3b · Naive Bayes text classifier Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r23--q3b/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-r23/q3b Previous examination · complete worked answers **Question. Explain Naive Bayes text classification with a suitable example.** [10 marks] ### Answer The paper supplies no numerical training corpus for this part. The following clearly labelled illustrative corpus demonstrates the complete calculation. ### Given | Training text | Class | | --- | --- | | Chinese Beijing Chinese | yes | | Chinese Chinese Shanghai | yes | | Chinese Macao | yes | | Tokyo Japan Chinese | no | Test: Chinese Chinese Chinese Tokyo Japan ### To find Classify the test using multinomial Naive Bayes. ### Formula: which one and why Assume add-one smoothing unless the question supplies probabilities. Keep repeated words as repeated factors. Use training vocabulary only; ignore out-of-vocabulary words unless an unknown-token model is defined. $$ \mathrm{score}(c)=P(c)\prod_{w\in V}\left(\frac{c(w,c)+1}{N_c+|V|}\right)^{c(w,d)} $$ ### Step 1. Vocabulary and class priors V = {beijing, chinese, japan, macao, shanghai, tokyo}; |V|=6. | Class | Documents | Prior | | --- | --- | --- | | no | 1 | 1/4 | | yes | 3 | 3/4 | ### Step 2. Count every training word within each class | Class | Total tokens | beijing | chinese | japan | macao | shanghai | tokyo | | --- | --- | --- | --- | --- | --- | --- | --- | | no | 3 | 0 | 1 | 1 | 0 | 0 | 1 | | yes | 8 | 1 | 5 | 0 | 1 | 1 | 0 | ### Step 3. Count the test words | Word | Count | | --- | --- | | chinese | 3 | | tokyo | 1 | | japan | 1 | OOV words: none ### Step 4. Calculate likelihoods and multiply with the prior | Class | Prior | Individual factors | Exact score | Decimal score | | --- | --- | --- | --- | --- | | no | 1/4 | chinese: (1+1)/(3+1×6) = 2/9, used 3 time(s); tokyo: (1+1)/(3+1×6) = 2/9, used 1 time(s); japan: (1+1)/(3+1×6) = 2/9, used 1 time(s) | 8/59049 | 0.000135480702467 | | yes | 3/4 | chinese: (5+1)/(8+1×6) = 3/7, used 3 time(s); tokyo: (0+1)/(8+1×6) = 1/14, used 1 time(s); japan: (0+1)/(8+1×6) = 1/14, used 1 time(s) | 81/268912 | 0.000301213779973 | ### Step 5. Compare and optionally normalize Normalized scores: no = 0.310241388, yes = 0.689758612 Division by the same total does not change the winning class. ### Final answer **Predicted class: yes.** **Yaad rakho:** Count → prior → smoothed word chances → repeated factors → compare. **Source / convention note.** Chinese corpus is an authored/reused teaching illustration here; it is actual given data in F22 Q5b, not this prompt. --- ## TEE · Q4a · Jaccard, cosine and Euclidean without preprocessing Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r23--q4a/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-r23/q4a Previous examination · complete worked answers **Question. Explain Jaccard, cosine and Euclidean measures. Calculate them without preprocessing for D₁=“Data is the new oil of the digital economy” and D₂=“Data is a new oil”.** [10 marks] ### Answer ### Given Use exactly the given case and words; do not remove stopwords. ### To find Three measures in explicitly stated representations. ### Formula: which one and why $$ J(A,B)=\frac{|A\cap B|}{|A\cup B|},\quad d_E(u,v)=\sqrt{\sum_i(u_i-v_i)^2} $$ Jaccard uses sets of distinct tokens. Cosine and Euclidean use raw-count BoW vectors in one shared order; do not secretly turn repeated the into a single count. ### Step 1. Build the common vocabulary and counts | Document | Data | is | the | new | oil | of | digital | economy | a | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | D1 | 1 | 1 | 2 | 1 | 1 | 1 | 1 | 1 | 0 | | D2 | 1 | 1 | 0 | 1 | 1 | 0 | 0 | 0 | 1 | ### Step 2. Jaccard sets Intersection={Data,is,new,oil}, size4; union size9. J=4/9=0.444444; Jaccard distance=1−J=5/9. ### Step 3. Cosine Dot product=4. D1 squared length=1+1+4+1+1+1+1+1=11; D2 squared length=5. Cosine=4/√55=0.539359890. ### Step 4. Euclidean Coordinate differences: [0, 0, 2, 0, 0, 1, 1, 1, -1]. Sum of squares=8. Distance=√8=2.828427125. Smaller distance means greater proximity; this is a distance, not a probability. ### Final answer **Jaccard=0.444444; cosine=0.539360; Euclidean distance=2.828427.** --- ## TEE · Q4b · CBOW and Skip-gram Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r23--q4b/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-r23/q4b Previous examination · complete worked answers **Question. Explain the CBOW and Skip-gram architectures.** [10 marks] ### Answer **Word2Vec learns dense word embeddings by predicting words from nearby context.** Its two common architectures are CBOW and skip-gram. A vocabulary ID or one-hot vector is the input representation; the learned embedding table converts that identity into a short trainable vector. | Architecture | Input | Prediction | Memory cue | | --- | --- | --- | --- | | CBOW | Surrounding context words | Centre/target word | Context → centre | | Skip-gram | Centre/target word | Surrounding words | Centre → context | CBOW: the + sat → Average context embeddings → Predict: cat Skip-gram: cat → Embedding of cat → Predict: the and sat For vocabulary size $$ V $$ and embedding dimension $$ d $$ , an input one-hot row is $$ 1\times V $$ ; the input embedding matrix is $$ V\times d $$ ; the hidden row is $$ 1\times d $$ ; the output matrix is $$ d\times V $$ ; output scores are $$ 1\times V $$ . Multiplying a one-hot row by the embedding matrix simply selects one row. $$ P(w_o\mid w_i)=\frac{\exp(u_{w_o}^{T}v_{w_i})}{\sum_{w\in V}\exp(u_w^{T}v_{w_i})}. $$ $$ v_{w_i} $$ is the input word embedding, $$ u_w $$ an output-context embedding, and the dot product is a compatibility score. Softmax turns all scores into probabilities that sum to one. Training increases the probability of observed context words. Negative sampling or hierarchical softmax can reduce computation in larger vocabularies; use the algorithm specified by the question. **Question. For the sentence “the cat sat on mat”, generate CBOW and skip-gram training data for centre word sat with a window of one token on each side.** #### Given Tokens indexed 1 to 5: the, cat, sat, on, mat. Centre = sat. #### To find The CBOW example and the skip-gram pairs. #### Formula and why it applies CBOW predicts the centre from its neighbours; skip-gram predicts each neighbour from the centre. #### Step 1. Identify neighbours Left neighbour = cat; right neighbour = on. “the” and “mat” are outside this window. #### Step 2. CBOW example Input [cat,on] → target sat. #### Step 3. Skip-gram examples (sat,cat) and (sat,on), with first item as input and second as predicted context. **Answer.** CBOW: [cat,on]→sat. Skip-gram: sat→cat and sat→on. **Question. The embeddings of context words cat and on are [1,2] and [3,0]. Compute the CBOW hidden vector using mean aggregation. Output vectors for [sat,ran] are [1,0] and [0,1]. Compute softmax probabilities.** #### Given Context embeddings [1,2], [3,0]; mean aggregation; two candidate output vectors. #### To find The hidden vector, logits and probabilities. #### Formula and why it applies Mean context vector $$ h=(v_1+v_2)/2 $$ . Logit $$ z_w=u_w^Th $$ . Softmax $$ P(w)=e^{z_w}/\sum_j e^{z_j} $$ . #### Step 1. Add coordinates $$ [1+3,\,2+0]=[4,2] $$ . #### Step 2. Divide by number of words $$ h=[4/2,2/2]=[2,1] $$ . #### Step 3. Compute logits $$ z_{sat}=1(2)+0(1)=2 $$ . $$ z_{ran}=0(2)+1(1)=1 $$ . #### Step 4. Exponentials $$ e^2\approx7.389056 $$ , $$ e^1\approx2.718282 $$ ; sum $$ \approx10.107338 $$ . #### Step 5. Divide $$ P(sat)\approx0.731059 $$ , $$ P(ran)\approx0.268941 $$ . **Answer.** h=[2,1]; sat is more probable (0.731059). **Question. A skip-gram centre embedding is v=[1,−1]. Output embeddings for [eat,sleep,drive] are [1,1], [0,1], [1,0]. Calculate the predicted context probabilities.** #### Given One centre vector and three output vectors; full softmax over these three words. #### To find Three logits and three probabilities. #### Formula and why it applies $$ z_j=u_j^Tv $$ , then $$ P(j)=e^{z_j}/\sum_k e^{z_k} $$ . #### Step 1. Eat dot product $$ 1(1)+1(-1)=0 $$ . #### Step 2. Sleep and drive dot products $$ z_{sleep}=0(1)+1(-1)=-1 $$ . $$ z_{drive}=1(1)+0(-1)=1 $$ . #### Step 3. Exponentiate and sum $$ e^0=1 $$ , $$ e^{-1}\approx0.367879 $$ , $$ e^1\approx2.718282 $$ . Sum $$ \approx4.086161 $$ . #### Step 4. Normalize $$ P(eat)\approx0.244728 $$ ; $$ P(sleep)\approx0.090031 $$ ; $$ P(drive)\approx0.665241 $$ . **Answer.** drive has the highest probability in this toy learned model. **Limitations:** a standard static Word2Vec vector merges a word's senses; it has limited representation of subword morphology and out-of-vocabulary words; its local prediction objective does not directly fit a full global co-occurrence table. Similarity is learned from data and is not a guarantee of factual or causal understanding. CBOW = neighbours se missing centre. Skip-gram = centre se neighbours. --- ## TEE · Q5a · Password regex Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r23--q5a/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-r23/q5a Previous examination · complete worked answers **Question. Write a regex for an 8–15 character password containing letters and digits, at least one uppercase letter and at least one special character from !%@\*, with no other characters.** [10 marks] ### Answer `^(?=.{8,15}$)(?=.*[A-Z])(?=.*[0-9])(?=.*[!%@*])[A-Za-z0-9!%@*]+$` | Part | Purpose | | --- | --- | | (?=.{8,15}$) | Length8–15 | | (?=.\*[A-Z]) | At least one uppercase; this also meets the letter requirement | | (?=.\*[0-9]) | At least one digit | | (?=.\*[!%@\*]) | At least one permitted special | | [A-Za-z0-9!%@\*]+ | Only permitted characters, including lowercase if desired | Use full-string matching. Abcdef1! passes; abcdef1! fails uppercase, Abcdefgh fails digit/special, Abc1! is too short, Abcdef1# contains forbidden #. A lowercase character is not separately required by the printed conditions. --- ## TEE · Q5b · Two NLP applications Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r23--q5b/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-r23/q5b Previous examination · complete worked answers **Question. Briefly explain any two applications of NLP.** [10 marks] ### Answer ### 1. Machine translation Translate meaning from source to target language using a model conditioned on the source sequence. Example: English “Where is the station?” to Hindi; word order and ambiguity require context. Train on parallel corpora and evaluate adequacy/fluency. ### 2. Information extraction Convert text into structured entities and relations. Example: “Acme acquired Beta in 2022” becomes acquirer=Acme, acquired=Beta, year=2022. Pipeline: tokenize → NER → relation extraction → validate output with source spans. --- ## TEE · Q6a · Mention-pair model Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r23--q6a/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-r23/q6a Previous examination · complete worked answers **Question. Explain the mention-pair model for reference resolution.** [10 marks] ### Answer A mention is a span referring to an entity. A mention-pair model classifies a pair (earlier candidate antecedent, later mention) as coreferent or not. Pairwise decisions are subsequently linked into entity clusters; a positive score alone does not ensure globally consistent clusters. Detect mentions → Generate earlier candidate pairs → Extract pair features → Score same-entity relation → Select/link antecedents → Form entity clusters 1. Text: “Rita bought a car. She likes it.” Detect Rita, a car, She, it. 2. For She, compare (Rita,She) and (a car,She). Features include person/number compatibility, animate type, syntactic role, sentence distance and lexical agreement. 3. Rita–She has compatible singular human features; car–She fails the ordinary inanimate reading. Select Rita. 4. For it, compare earlier mentions: car–it is singular/inanimate and contextually fits likes. Cluster {Rita,She} and {a car,it}. 5. Train a classifier on labelled mention pairs; at prediction, choose a sufficiently supported antecedent or leave a new entity. Avoid reflexive/binding violations and document leakage during evaluation. Advantages: interpretable pair features and supervised learning. Limitations: quadratic candidate growth, error propagation from mention detection, non-transitive pair predictions, and weak long-range/world-knowledge reasoning. A greedy nearest-candidate choice is not the same as a learned mention-pair model. --- ## TEE · Q6b · Classification evaluation including ROC/AUC Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r23--q6b/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-r23/q6b Previous examination · complete worked answers **Question. Explain accuracy, precision, recall, F1 score, ROC and AUC.** [10 marks] ### Answer Evaluation compares predicted results with known answers. A **true positive** is a correctly predicted positive; a **false positive** is a negative item predicted positive; a **false negative** is a missed positive; a **true negative** is a correctly rejected negative. $$ \text{Precision}=\frac{TP}{TP+FP} $$ $$ \text{Recall}=\frac{TP}{TP+FN} $$ $$ F_1=\frac{2PR}{P+R}=\frac{2TP}{2TP+FP+FN} $$ $$ \text{Accuracy}=\frac{TP+TN}{TP+TN+FP+FN} $$ Precision asks how many predicted positives were correct. Recall asks how many actual positives were found. F1 balances them by a harmonic mean. If a denominator is zero, state the convention rather than dividing by zero. ### Example 1 · evaluate a classifier A binary text classifier gives TP=6, FP=2, FN=4, TN=8. Compute precision, recall, F1 and accuracy. **Given** TP=6; FP=2; FN=4; TN=8. **To find** Four evaluation measures. ### Formula and why it applies $$ P=\frac{TP}{TP+FP},\quad R=\frac{TP}{TP+FN} $$ $$ F_1=\frac{2TP}{2TP+FP+FN},\quad A=\frac{TP+TN}{TP+FP+FN+TN} $$ Use predicted positives in precision, actual positives in recall, and all examples in accuracy. ### Solution #### Step 1. Precision $$ P=\frac{6}{6+2}=0.750000 $$ #### Step 2. Recall $$ R=\frac{6}{6+4}=0.600000 $$ #### Step 3. F1 $$ F_1=\frac{12}{12+2+4}=0.666667 $$ #### Step 4. Accuracy $$ A=\frac{6+8}{6+2+4+8}=0.700000 $$ Precision 0.7500; recall 0.6000; F1 0.6667; accuracy 0.7000. ### Example 2 · evaluate a classifier A binary text classifier gives TP=9, FP=3, FN=1, TN=7. Compute precision, recall, F1 and accuracy. **Given** TP=9; FP=3; FN=1; TN=7. **To find** Four evaluation measures. ### Formula and why it applies $$ P=\frac{TP}{TP+FP},\quad R=\frac{TP}{TP+FN} $$ $$ F_1=\frac{2TP}{2TP+FP+FN},\quad A=\frac{TP+TN}{TP+FP+FN+TN} $$ Use predicted positives in precision, actual positives in recall, and all examples in accuracy. ### Solution #### Step 1. Precision $$ P=\frac{9}{9+3}=0.750000 $$ #### Step 2. Recall $$ R=\frac{9}{9+1}=0.900000 $$ #### Step 3. F1 $$ F_1=\frac{18}{18+3+1}=0.818182 $$ #### Step 4. Accuracy $$ A=\frac{9+7}{9+3+1+7}=0.800000 $$ Precision 0.7500; recall 0.9000; F1 0.8182; accuracy 0.8000. ### Example 3 · evaluate a classifier A binary text classifier gives TP=4, FP=0, FN=4, TN=12. Compute precision, recall, F1 and accuracy. **Given** TP=4; FP=0; FN=4; TN=12. **To find** Four evaluation measures. ### Formula and why it applies $$ P=\frac{TP}{TP+FP},\quad R=\frac{TP}{TP+FN} $$ $$ F_1=\frac{2TP}{2TP+FP+FN},\quad A=\frac{TP+TN}{TP+FP+FN+TN} $$ Use predicted positives in precision, actual positives in recall, and all examples in accuracy. ### Solution #### Step 1. Precision $$ P=\frac{4}{4+0}=1.000000 $$ #### Step 2. Recall $$ R=\frac{4}{4+4}=0.500000 $$ #### Step 3. F1 $$ F_1=\frac{8}{8+0+4}=0.666667 $$ #### Step 4. Accuracy $$ A=\frac{4+12}{4+0+4+12}=0.800000 $$ Precision 1.0000; recall 0.5000; F1 0.6667; accuracy 0.8000. ### ROC and AUC $$ \mathrm{TPR}=\frac{TP}{TP+FN},\qquad\mathrm{FPR}=\frac{FP}{FP+TN} $$ A classifier produces scores. Sweep the decision threshold; at each threshold calculate TPR and FPR. Plot FPR horizontally and TPR vertically. AUC is the area under that curve, equivalently the probability that a randomly chosen positive has a higher score than a randomly chosen negative (ties conventionally contribute half). AUC1 is perfect ranking;0.5 is chance ranking. This evaluates ranking over thresholds, not the accuracy at one threshold. | Illustrative threshold | TPR | FPR | | --- | --- | --- | | Above all scores | 0 | 0 | | Middle threshold | 0.8 | 0.2 | | Below all scores | 1 | 1 | For straight segments through (0,0),(.2,.8),(1,1), trapezoid area=.2×(0+.8)/2+.8×(.8+1)/2=.08+.72=.80. These points illustrate integration; they are not supplied exam data. Precision–recall may be more informative for rare positives. --- ## TEE · Q7a · Organization and facility mentions Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r23--q7a/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-r23/q7a Previous examination · complete worked answers **Question. Identify organization and facility mentions in this White House visitor passage using the supplied ACE categories: “After the 9/11 attacks, the system changed radically. Now, anyone who wants to tour the White House must apply through the office of his or her representative in Congress, which forwards the names to the White House for clearance. Once they get the green light, visitors show up at the appointed time on 15th Street between E and F Streets and join the line to enter through the southeast gate. Anyone who has flown on an airline in recent years will recognize the familiar territory of identity checks and electronic scans, although you do get to keep your shoes on. At the head of the line, rangers from the National Park Service check photo IDs against a list of names.”** [10 marks] ### Answer | Mention in order | Type | Reason | | --- | --- | --- | | White House (tour) | FAC | Physical building being toured | | office of his or her representative | ORG under administrative-office reading | Staff/organizational office receiving applications; no visit to the physical office is described | | Congress | ORG | Legislative organization | | White House (clearance) | ORG | Institution/staff granting clearance, a metonymic use | | 15th Street | FAC | Transport infrastructure | | E and F Streets | FAC (two coordinated streets) | Roads, not organizations | | southeast gate | FAC | Physical entrance component | | an airline | ORG generic mention | Organization operating flights; generic, not a proper named company | | National Park Service | ORG | Government agency | The prompt uses ACE entity mentions: nominal/generic mentions can be included, unlike a named-only tagger. 9/11 is an event/date mention, not ORG/FAC; visitors and rangers are people. If a narrower classroom guideline asks only named entities, omit the generic office/airline/gate spans and explicitly state that convention. Context decides whether White House means building or administration. **Source / convention note.** Boundary and generic-mention decisions depend on the provided ACE annotation convention; the answer distinguishes them from a named-only interpretation. --- ## TEE · Q7b · Tagging and two complete parse trees Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r23--q7b/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-r23/q7b Previous examination · complete worked answers **Question. Explain POS tagging. Tag and parse (1) John and Mary bought a refrigerator with three doors. (2) It was purchased from a very small store near their house.** [10 marks] ### Answer POS tagging assigns grammatical categories in context; constituency parsing groups those words hierarchically. Here with three doors modifies refrigerator; near their house modifies store. These attachments fit the natural reading; the paper gives no fixed CFG. `John/NNP and/CC Mary/NNP bought/VBD a/DT refrigerator/NN with/IN three/CD doors/NNS` SNPNNPJohnCCandNNPMaryVPVBDboughtNPDTaNNrefrigeratorPPINwithNPCDthreeNNSdoors [S [NP [NNP John] [CC and] [NNP Mary]] [VP [VBD bought] [NP [DT a] [NN refrigerator] [PP [IN with] [NP [CD three] [NNS doors]]]]]] `It/PRP was/VBD purchased/VBN from/IN a/DT very/RB small/JJ store/NN near/IN their/PRP$ house/NN` SNPPRPItVPVBDwasVPVBNpurchasedPPINfromNPDTaADJPRBveryJJsmallNNstorePPINnearNPPRP$theirNNhouse [S [NP [PRP It]] [VP [VBD was] [VP [VBN purchased] [PP [IN from] [NP [DT a] [ADJP [RB very] [JJ small]] [NN store] [PP [IN near] [NP [PRP$ their] [NN house]]]]]]]] The first subject is coordinated; bought is past tense. The second VP is passive auxiliary was plus past participle purchased. Very modifies small, and their is a possessive determiner. --- ## TEE · Q1a · Need and difficulties of NLP Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f22--q1a/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-f22/q1a Previous examination · complete worked answers **Question. Explain the need for NLP and the difficulties in natural language processing.** [5 marks] ### Answer **Natural language processing (NLP) is the field concerned with computational methods for analysing, interpreting and generating human language.** Natural languages such as English, Hindi and Marathi evolved through human use; their meanings are not specified by the rigid rules of a programming language. An NLP system may receive text, speech converted to text, or another linguistic representation. Its output depends on the task: a category, a tag for every word, a translated sentence, a ranked document list or a generated answer. It does not always “understand” in the human sense; its performance must be evaluated for the specific task. | Task | Input | Output | | --- | --- | --- | | Sentiment analysis | “The service was excellent.” | Positive sentiment | | Text classification | An email | Spam / not spam | | POS tagging | “Birds fly.” | Birds/NOUN fly/VERB | | Named entity recognition | “Asha joined Google in Mumbai.” | Asha: person; Google: organisation; Mumbai: location | | Machine translation | A Hindi sentence | A corresponding English sentence | | Information retrieval | A search query | Ranked relevant documents | | Question answering | Question plus knowledge/context | An answer | | Summarization | A long article | A shorter text preserving key information | ### How NLP relates to AI, ML and DL **AI** is the broader study of systems performing tasks associated with intelligent behaviour. **Machine learning** learns patterns from data. **Deep learning** uses multiple trainable neural layers. NLP is an application/research field that can use hand-written rules, statistical ML or deep learning. Therefore, not every NLP system is a neural network. Linguistics supplies knowledge about language structure. Statistics supplies uncertainty models. Computer science supplies algorithms, storage and efficient implementation. A practical NLP system combines these ideas. NLP = human language ko computational task mein badlo: label, meaning, search ya generated sentence. **Ambiguity** means an expression supports more than one interpretation. Good exam answers name the ambiguity and explain both readings. | Challenge | Example | Why a system can fail | | --- | --- | --- | | Lexical/sense ambiguity | bank: finance or river edge | One written word has several senses | | Syntactic ambiguity | I saw the man with a telescope. | The telescope may modify saw or the man | | Reference ambiguity | Riya told Neha that she won. | She may refer to either person without more context | | Sarcasm | Great, another delay. | Literal positive word conveys frustration | | Negation | The film was not good. | Dropping not reverses meaning | | Domain shift | “Positive” in a medical report | Meaning/use differs by domain | | Code mixing | Kal meeting reschedule karna. | Several languages and spelling conventions | | Noisy text | tmrw, repeated letters, typos | Vocabulary/token boundaries become uncertain | | Low-resource setting | Few labelled examples in a language | Models lack representative training evidence | | Bias and privacy | Skewed data or personal details | Errors and harms can persist in deployed systems | **Homophones** sound alike but may differ in spelling (pair/pear). **Homographs** share spelling but can differ in meaning or pronunciation (lead the team / lead metal). These are different dimensions from syntactic ambiguity. Long-distance dependencies, cultural knowledge, idioms, morphology and multilingual writing systems also complicate processing. For “kick the bucket”, literal word-by-word interpretation can miss the idiomatic meaning. For “The book that the students recommended is useful”, the verb is agrees with book across an intervening phrase. Exam mein challenge ka naam + ek sentence + dono meanings/actual failure likho. --- ## TEE · Q1b · Levels of language analysis Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f22--q1b/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-f22/q1b Previous examination · complete worked answers **Question. Explain the levels of language analysis with examples.** [5 marks] ### Answer **A phoneme** is a sound unit that can distinguish meanings in a language, such as the /p/ and /b/ contrast in “pat” and “bat”. A phoneme is a sound, not necessarily one alphabet letter. **A morpheme** is the smallest meaning-bearing grammatical unit: cats = cat + plural -s; unhappy = un- + happy. **A lexeme** is an abstract vocabulary item grouping its inflected forms: write, writes, wrote and writing belong to WRITE. A **phrase** is a group of words functioning together; a **clause** normally contains a predicate and its associated participants; a **sentence** expresses a complete grammatical unit. | Level | What it analyses | Concrete example | | --- | --- | --- | | Phonetics / phonology | Physical sounds / sound patterns in a language | Distinguish bat from pat | | Morphology / lexical analysis | Word structure and lexical items | un + help + ful → unhelpful | | Syntax | Grammatical arrangement and relationships | The dog chased the cat: subject, verb, object | | Semantics | Meaning of words and composed expressions | The dog is the chaser; the cat is chased | | Discourse | Connections across sentences | “Riya arrived. She smiled.” → She refers to Riya | | Pragmatics | Intended meaning in the situation | “It is cold here” may request closing the window | word forms → grammatical structure → sentence meaning → cross-sentence context → speaker intention ### Three worked interpretations **“The cats are sleeping.”** Morphology identifies cat + s as plural. Syntax connects “cats” to “are sleeping”. Semantics represents animals engaged in sleeping. **“Aman bought a laptop. It was expensive.”** Sentence-level syntax alone does not identify “it”. Discourse connects “it” to the laptop across the sentence boundary. **“Wonderful, another power cut!”** The word wonderful is positive literally, but situational context can indicate sarcasm and frustration. This is a pragmatic interpretation. The analysis levels are conceptual descriptions. Modern systems may learn several jointly rather than executing a compulsory separate module for every level. Morphology = word ke andar; syntax = sentence ka structure; semantics = meaning; discourse = sentences ka link; pragmatics = intention. --- ## TEE · Q1c · Medical chatbot Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f22--q1c/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-f22/q1c Previous examination · complete worked answers **Question. Briefly explain a medical chatbot and its NLP processing.** [5 marks] ### Answer A medical chatbot accepts a patient’s natural-language message and provides an appropriately bounded response, such as appointment information or approved health education. It must distinguish intents and preserve symptoms, negations and durations. Patient message → Tokenization + intent/entity detection → Retrieve approved information → Generate grounded response → Escalate to clinician when needed Example: “I have no fever but a cough for three days” yields cough present, fever absent and duration three days. A safe administrative bot can suggest an appointment; it must not turn uncertain text into a definitive diagnosis. Evaluate intent accuracy, entity/negation accuracy, response correctness, privacy and escalation behaviour. --- ## TEE · Q1d · Sentiment with logistic regression Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f22--q1d/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-f22/q1d Previous examination · complete worked answers **Question. Explain sentiment analysis using logistic regression with an example.** [5 marks] ### Answer Binary logistic regression maps text features x to a probability: $$ z=w^Tx+b,\qquad P(y=1\mid x)=\sigma(z)=\frac{1}{1+e^{-z}} $$ Use positive-word count and negative-word count as an illustrative feature vector. With x=(2,1), w=(1,−1), b=0: z=1×2+(−1)×1=1; P(positive)=1/(1+e⁻¹)=0.731059. A declared 0.5 threshold predicts positive. Fit weights from labelled reviews by minimizing cross-entropy; the hand-chosen weights here only illustrate the computation. Labelled reviews → Tokenize / vectorize → Fit logistic weights → Probability → Threshold / class --- ## TEE · Q2a · CBOW and Skip-gram architecture Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f22--q2a/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-f22/q2a Previous examination · complete worked answers **Question. Explain CBOW and Skip-gram with suitable architecture diagrams.** [10 marks] ### Answer **Word2Vec learns dense word embeddings by predicting words from nearby context.** Its two common architectures are CBOW and skip-gram. A vocabulary ID or one-hot vector is the input representation; the learned embedding table converts that identity into a short trainable vector. | Architecture | Input | Prediction | Memory cue | | --- | --- | --- | --- | | CBOW | Surrounding context words | Centre/target word | Context → centre | | Skip-gram | Centre/target word | Surrounding words | Centre → context | CBOW: the + sat → Average context embeddings → Predict: cat Skip-gram: cat → Embedding of cat → Predict: the and sat For vocabulary size $$ V $$ and embedding dimension $$ d $$ , an input one-hot row is $$ 1\times V $$ ; the input embedding matrix is $$ V\times d $$ ; the hidden row is $$ 1\times d $$ ; the output matrix is $$ d\times V $$ ; output scores are $$ 1\times V $$ . Multiplying a one-hot row by the embedding matrix simply selects one row. $$ P(w_o\mid w_i)=\frac{\exp(u_{w_o}^{T}v_{w_i})}{\sum_{w\in V}\exp(u_w^{T}v_{w_i})}. $$ $$ v_{w_i} $$ is the input word embedding, $$ u_w $$ an output-context embedding, and the dot product is a compatibility score. Softmax turns all scores into probabilities that sum to one. Training increases the probability of observed context words. Negative sampling or hierarchical softmax can reduce computation in larger vocabularies; use the algorithm specified by the question. **Question. For the sentence “the cat sat on mat”, generate CBOW and skip-gram training data for centre word sat with a window of one token on each side.** #### Given Tokens indexed 1 to 5: the, cat, sat, on, mat. Centre = sat. #### To find The CBOW example and the skip-gram pairs. #### Formula and why it applies CBOW predicts the centre from its neighbours; skip-gram predicts each neighbour from the centre. #### Step 1. Identify neighbours Left neighbour = cat; right neighbour = on. “the” and “mat” are outside this window. #### Step 2. CBOW example Input [cat,on] → target sat. #### Step 3. Skip-gram examples (sat,cat) and (sat,on), with first item as input and second as predicted context. **Answer.** CBOW: [cat,on]→sat. Skip-gram: sat→cat and sat→on. **Question. The embeddings of context words cat and on are [1,2] and [3,0]. Compute the CBOW hidden vector using mean aggregation. Output vectors for [sat,ran] are [1,0] and [0,1]. Compute softmax probabilities.** #### Given Context embeddings [1,2], [3,0]; mean aggregation; two candidate output vectors. #### To find The hidden vector, logits and probabilities. #### Formula and why it applies Mean context vector $$ h=(v_1+v_2)/2 $$ . Logit $$ z_w=u_w^Th $$ . Softmax $$ P(w)=e^{z_w}/\sum_j e^{z_j} $$ . #### Step 1. Add coordinates $$ [1+3,\,2+0]=[4,2] $$ . #### Step 2. Divide by number of words $$ h=[4/2,2/2]=[2,1] $$ . #### Step 3. Compute logits $$ z_{sat}=1(2)+0(1)=2 $$ . $$ z_{ran}=0(2)+1(1)=1 $$ . #### Step 4. Exponentials $$ e^2\approx7.389056 $$ , $$ e^1\approx2.718282 $$ ; sum $$ \approx10.107338 $$ . #### Step 5. Divide $$ P(sat)\approx0.731059 $$ , $$ P(ran)\approx0.268941 $$ . **Answer.** h=[2,1]; sat is more probable (0.731059). **Question. A skip-gram centre embedding is v=[1,−1]. Output embeddings for [eat,sleep,drive] are [1,1], [0,1], [1,0]. Calculate the predicted context probabilities.** #### Given One centre vector and three output vectors; full softmax over these three words. #### To find Three logits and three probabilities. #### Formula and why it applies $$ z_j=u_j^Tv $$ , then $$ P(j)=e^{z_j}/\sum_k e^{z_k} $$ . #### Step 1. Eat dot product $$ 1(1)+1(-1)=0 $$ . #### Step 2. Sleep and drive dot products $$ z_{sleep}=0(1)+1(-1)=-1 $$ . $$ z_{drive}=1(1)+0(-1)=1 $$ . #### Step 3. Exponentiate and sum $$ e^0=1 $$ , $$ e^{-1}\approx0.367879 $$ , $$ e^1\approx2.718282 $$ . Sum $$ \approx4.086161 $$ . #### Step 4. Normalize $$ P(eat)\approx0.244728 $$ ; $$ P(sleep)\approx0.090031 $$ ; $$ P(drive)\approx0.665241 $$ . **Answer.** drive has the highest probability in this toy learned model. **Limitations:** a standard static Word2Vec vector merges a word's senses; it has limited representation of subword morphology and out-of-vocabulary words; its local prediction objective does not directly fit a full global co-occurrence table. Similarity is learned from data and is not a guarantee of factual or causal understanding. CBOW = neighbours se missing centre. Skip-gram = centre se neighbours. --- ## TEE · Q2b · Seven regex constructions Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f22--q2b/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-f22/q2b Previous examination · complete worked answers **Question. Define regular expressions and their uses. Write expressions for (i) all alphabetic strings, (ii) lowercase alphabetic strings ending in b, (iii) a/b strings where each a is immediately preceded and followed by b, (iv) consecutive repeated words, (v) a line starting with an integer and ending with a word, (vi) strings containing the whole words grotto and raven, (vii) capture the first English word, handling opening punctuation.** [10 marks] ### Answer A regular expression specifies a pattern for matching text. NLP uses it for token extraction, cleaning, validation and rule-based entities. The following uses a Python/PCRE-like engine with lookaheads and backreferences; these extended features go beyond a strictly finite-state regular language. | Task | Pattern | Why it works | | --- | --- | --- | | All alphabetic strings | `^[A-Za-z]+$` | Only letters; + requires at least one. Use \* if empty string is allowed. | | Lowercase strings ending in b | `^[a-z]*b$` | The final b is mandatory. | | a/b strings: every a surrounded by b | `^(?!a)(?!.*aa)(?!.*a$)[ab]*$` | Forbid starting a, adjacent aa, ending a. Then every a has a b on both sides. | | Two consecutive identical words | `\b([A-Za-z]+)\s+\1\b` | Capture word, whitespace, same captured word. Case-sensitive unless ignore-case is enabled. | | Start with integer; end with word | `^[0-9]+\b.*\b[A-Za-z]+$` | Same line; signed integers need optional [+-]. | | Whole grotto and whole raven anywhere | `^(?=.*\bgrotto\b)(?=.*\braven\b).*$` | Two positive lookaheads require both whole words. Dot excludes newlines unless DOTALL. | | Capture first word despite opening punctuation | `^[^A-Za-z]*([A-Za-z]+(?:[\x27’][A-Za-z]+)*)` | Group1 is first alphabetic word, with internal apostrophes allowed. | Tests: b and bab pass rule3; a, ab, ba and baab fail. “Humbert Humbert” matches rule4; “the big bug” does not. “grottos raven” fails rule6. Regex word rules are practical ASCII conventions here, not a full linguistic tokenizer. --- ## TEE · Q3a · Text preprocessing Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f22--q3a/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-f22/q3a Previous examination · complete worked answers **Question. Explain text preprocessing with applications and examples.** [10 marks] ### Answer **Tokenization divides text into units called tokens for further processing.** A token may be a word, punctuation mark, character or subword, depending on the tokenizer. **Sentence segmentation** identifies sentence boundaries. Tokenization is not necessarily just splitting on spaces. | Method | Example | Trade-off | | --- | --- | --- | | Word | “cats sleep” → [cats,sleep] | Easy to interpret; unfamiliar words can be unseen tokens | | Character | cat → [c,a,t] | Handles spelling units; creates longer sequences | | Subword | unhappiness → pieces such as [un,happi,ness] in an illustrative vocabulary | Balances vocabulary size and rare-word representation | | Sentence | “Birds fly. Fish swim.” → two sentences | Abbreviations and punctuation complicate boundaries | Actual subword splits depend on the learned vocabulary; the example is illustrative, not a universal tokenizer output. Word segmentation is especially important where spaces do not reliably separate words or in compound-rich languages. **Question. Tokenize “Dr. Rao can't come today.” into words and punctuation under this rule: preserve Dr. and can't as whole tokens, and split only the final full stop.** #### Given The tokenization rule is explicitly supplied. #### To find The resulting token sequence. #### Formula and why it applies Apply the stated boundary rule; do not assume every period or apostrophe is a split point. #### Step 1. Handle abbreviation Dr. remains one token. #### Step 2. Handle contraction can't remains one token. #### Step 3. Write sequence [Dr., Rao, can't, come, today, .] **Answer.** 6 tokens under the specified rule. Another tokenizer may legitimately produce a different sequence. ### Three difficult cases **New York** may be two tokens but one named entity. **3.14** should not normally end a sentence at the decimal point. **don't** may remain whole or split into do + n't, depending on the tokenizer. Always state the convention when calculating counts. Token = chosen unit. Space split ko universal rule mat samajh. **Normalization reduces superficial variation so comparable forms can be processed consistently.** Common operations include lowercasing, Unicode normalization, appropriate spelling correction, contraction handling, and optional stemming or lemmatization. **Stopwords** are frequent words selected for removal for a particular task, such as the or a. Removing them may reduce feature size, but removing not, no or nor can destroy sentiment meaning. Punctuation can convey sentence boundaries, emotion or mathematical structure. Lowercasing can erase differences such as US versus us. **Question. Preprocess “The CATS are not sleeping!!!” by lowercasing, removing punctuation, removing only stopwords {the,are}, and lemmatizing plural cats→cat and sleeping→sleep.** #### Given Operations and stopword set are fixed. Do not remove not. #### To find The final token list. #### Formula and why it applies Apply the requested operations in order, keeping a visible intermediate result after each transformation. #### Step 1. Lowercase the cats are not sleeping!!! #### Step 2. Tokenize and remove punctuation [the,cats,are,not,sleeping] #### Step 3. Remove specified stopwords Remove the and are → [cats,not,sleeping]. #### Step 4. Lemmatize cats→cat; sleeping→sleep → [cat,not,sleep]. **Answer.** [cat,not,sleep]. The negative meaning is retained. ### Task decides preprocessing For a simple keyword-retrieval model, removing selected function words can help. For machine translation, question answering, grammar analysis or transformer input, aggressive removal may discard necessary syntax or violate the model's expected input format. Emoji can carry sentiment; convert or retain them when relevant instead of always deleting them. Clean karna = useful signal bachake noise hatao. Har word/punctuation delete karna cleaning nahi. **Stemming** applies string-based rules, often stripping suffixes, to reduce related forms to a stem. The result need not be a valid dictionary word. **Lemmatization** uses vocabulary and morphological analysis, often with POS information, to return a dictionary lemma. | Criterion | Stemming | Lemmatization | | --- | --- | --- | | Basis | Surface form/suffix rules | Lexicon and morphological/POS analysis | | Output | May be nonword, e.g. studi | Dictionary form, e.g. study | | Context/POS | Usually little or none | Often important | | Irregular forms | May miss went→go | Can map went→go with verb analysis | | Cost | Generally simpler | Usually more linguistic processing | Porter applies a sequence of suffix-removal rules subject to conditions; Snowball provides related language-specific stemming algorithms; Lancaster is often more aggressive; regex stemming uses explicit patterns; Lovins is another established stemming approach. Exact output depends on the algorithm and implementation. ### Three worked reductions 1. **studies, studying:** an English stemmer may yield studi/study depending on the rule sequence; verb lemmatization aims at study. State the exact algorithm if exact outputs are asked. 2. **better:** adjective-aware lemmatization can map it to good; naïve suffix removal cannot infer that irregular relation. 3. **saw:** as a past-tense verb it can become see; as a cutting-tool noun it remains saw. POS and context matter. **Over-stemming** merges forms that should remain distinct; **under-stemming** fails to merge related forms. Normalization can increase recall in search while losing useful distinctions, so it is a task-dependent choice. Stem = rules se cut; lemma = dictionary form, meaning/POS ka dhyan. --- ## TEE · Q3b · Penn Treebank tagging Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f22--q3b/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-f22/q3b Previous examination · complete worked answers **Question. Using Penn Treebank tags, tag every word and ignore punctuation: (1) It is a nice night. (2) This crap game is over a garage in Fifty-second Street. (3) Nobody ever takes the newspapers she sells. (4) I am sitting in Mindy’s restaurant putting on the gefilte fish, which is a dish I am very fond of. (5) What flights do you have from Milwaukee to Tampa?** [10 marks] ### Answer Read the role of each word in its sentence. Split possessive ’s into POS; punctuation has no requested tag. The original supplies a Penn-tag annex. ### Sentence 1 `It/PRP is/VBZ a/DT nice/JJ night/NN` ### Sentence 2 `This/DT crap/NN game/NN is/VBZ over/IN a/DT garage/NN in/IN Fifty-second/JJ Street/NNP` ### Sentence 3 `Nobody/NN ever/RB takes/VBZ the/DT newspapers/NNS she/PRP sells/VBZ` ### Sentence 4 `I/PRP am/VBP sitting/VBG in/IN Mindy/NNP ’s/POS restaurant/NN putting/VBG on/RP the/DT gefilte/JJ fish/NN which/WDT is/VBZ a/DT dish/NN I/PRP am/VBP very/RB fond/JJ of/IN` ### Sentence 5 `What/WDT flights/NNS do/VBP you/PRP have/VB from/IN Milwaukee/NNP to/IN Tampa/NNP` | Tag | Meaning | Why here | | --- | --- | --- | | PRP | Personal pronoun | It, I, she, you | | DT / WDT | Determiner / wh-determiner | a/the versus What before flights; which relative modifier | | VB/VBP/VBZ/VBG | Base/non-3sg-present/3sg-present/gerund-participle | have after do; am/do; is/takes; sitting/putting | | NN/NNS/NNP | Common singular/plural/proper noun | game/newspapers/Milwaukee | | IN/RP | Preposition or subordinator / particle | in/over/from; on in the idiomatic putting on reading | | JJ/RB/POS | Adjective/adverb/possessive ending | nice/very/’s | Annotation ambiguities: “crap” is a noun adjunct under NN; an adjectival analysis can use JJ. “Fifty-second” may be NNP within an official street name instead of JJ; “gefilte” can be a noun adjunct in some taggers. Explain the chosen contextual reading rather than claiming all taggers must agree. **Source / convention note.** Context-sensitive tags and possessive tokenization are explicitly stated; reasonable corpus guidelines can differ on noun adjuncts. --- ## TEE · Q4a · Mention-pair coreference Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f22--q4a/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-f22/q4a Previous examination · complete worked answers **Question. Explain the mention-pair model with a detailed example.** [10 marks] ### Answer A mention is a span referring to an entity. A mention-pair model classifies a pair (earlier candidate antecedent, later mention) as coreferent or not. Pairwise decisions are subsequently linked into entity clusters; a positive score alone does not ensure globally consistent clusters. Detect mentions → Generate earlier candidate pairs → Extract pair features → Score same-entity relation → Select/link antecedents → Form entity clusters 1. Text: “Rita bought a car. She likes it.” Detect Rita, a car, She, it. 2. For She, compare (Rita,She) and (a car,She). Features include person/number compatibility, animate type, syntactic role, sentence distance and lexical agreement. 3. Rita–She has compatible singular human features; car–She fails the ordinary inanimate reading. Select Rita. 4. For it, compare earlier mentions: car–it is singular/inanimate and contextually fits likes. Cluster {Rita,She} and {a car,it}. 5. Train a classifier on labelled mention pairs; at prediction, choose a sufficiently supported antecedent or leave a new entity. Avoid reflexive/binding violations and document leakage during evaluation. Advantages: interpretable pair features and supervised learning. Limitations: quadratic candidate growth, error propagation from mention detection, non-transitive pair predictions, and weak long-range/world-knowledge reasoning. A greedy nearest-candidate choice is not the same as a learned mention-pair model. --- ## TEE · Q4b · Text classification pipeline Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f22--q4b/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-f22/q4b Previous examination · complete worked answers **Question. Explain the text-classification pipeline with a diagram.** [10 marks] ### Answer **Text classification** assigns a document to one or more predefined categories. Spam detection usually uses two classes; news classification may use several topics. In single-label multiclass classification, choose one of several categories. In multilabel classification, several labels may apply to the same document. Define labels → Prepare labelled examples → Train representation + model → Evaluate on unseen text → Predict new labels ### An exam-ready pipeline | Stage | What you do | Why it matters | | --- | --- | --- | | Task definition | Decide what counts as spam or which news topics exist. | Unclear labels create inconsistent training data. | | Data collection and annotation | Collect representative examples and assign correct labels. | The model learns from these examples. | | Split data | Separate training, validation and test data. | Test examples must not influence training choices. | | Preprocess | Tokenize and normalise where suitable; retain meaningful negation. | Reduce irrelevant variation without deleting signal. | | Represent | Use BoW, TF-IDF or learned embeddings. | Models need numerical features. | | Train | Fit Naive Bayes, logistic regression or another appropriate model. | Learn how features predict categories. | | Evaluate | Measure relevant precision, recall and F1 on held-out examples. | Accuracy alone may hide poor minority-class performance. | | Deploy and monitor | Apply the same preprocessing and monitor changing language. | Incoming data can differ from old examples. | Naive Bayes is especially useful for hand calculations. With a bag-of-words representation, its class score is the prior multiplied by the conditional probability of each token; repeated tokens repeat the factor. The “naive” assumption is conditional independence of features given the class, not that words are independent in real language. $$ s(c)=P(c)\prod_{w\in V}P(w\mid c)^{C(w,d)} $$ $$ P(w\mid c)=\frac{C(w,c)+1}{N_c+|V|} $$ C(w,d) counts word w in document d. C(w,c) counts it in training documents belonging to class c. Nc is the total token count for class c. Add-one smoothing adds one to every vocabulary count and therefore adds |V| to the denominator. This is the same arithmetic taught for WSD, with document classes instead of senses. ### Worked classification · two-token message A spam classifier has vocabulary {win, prize, meeting}. Priors are P(spam)=P(ham)=0.5. Spam token counts are [3,2,0]; ham counts are [0,0,5]. With add-one smoothing, classify “win prize”. **Given** |V|=3; N(spam)=5; N(ham)=5; message has win once and prize once. **To find** Class scores and predicted label. ### Formula and why it applies $$ s(c)=P(c)P(\text{win}\mid c)P(\text{prize}\mid c) $$ $$ P(w\mid c)=\frac{C(w,c)+1}{N_c+|V|} $$ The model is multinomial Naive Bayes; multiply the two token likelihoods and the class prior. ### Solution #### Step 1. Compute the smoothed denominators $$ 5+3=8 $$ #### Step 2. Compute spam likelihoods $$ P(\text{win}\mid S)=\frac{3+1}{8}=\frac12 $$ $$ P(\text{prize}\mid S)=\frac{2+1}{8}=\frac38 $$ #### Step 3. Compute ham likelihoods $$ P(\text{win}\mid H)=P(\text{prize}\mid H)=\frac{0+1}{8}=\frac18 $$ #### Step 4. Multiply the spam score $$ s(S)=\frac12\times\frac12\times\frac38=\frac{3}{32}=0.09375 $$ #### Step 5. Multiply the ham score $$ s(H)=\frac12\times\frac18\times\frac18=\frac{1}{128}=0.0078125 $$ #### Step 6. Compare 0.09375 > 0.0078125. These are unnormalised scores; the label is chosen by the larger score. Predicted label: spam. **Yaad rakho:** Class prior × har word ki class-likelihood; sabse bada score wins. --- ## TEE · Q5a · Corpora and structure Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f22--q5a/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-f22/q5a Previous examination · complete worked answers **Question. Define a corpus. Explain corpus types, characteristics and structure.** [10 marks] ### Answer **A corpus is a collection of language data assembled for study or computational use.** Its plural is corpora. A corpus can contain books, news, reviews or conversations. It need not be labelled. **Annotation** adds linguistic information such as POS tags, entities, parse structures or sense labels. | Structure | Meaning | Use | | --- | --- | --- | | Plain | Collection without category labels | Word counts, language modelling | | Categorized | Documents grouped by genre/topic | Classification | | Overlapping categories | One document has several categories | Multi-topic news | | Temporal | Documents indexed by time | Language change | | Annotated | Linguistic labels attached | Tagging, parsing, WSD | | Parallel | Aligned texts in different languages | Translation | A **tagset** is the allowed label inventory. A **treebank** is a corpus annotated with syntax trees. **Metadata** records source, date, genre, language, annotation method and limitations. A common benchmark supports comparison under shared data and evaluation conditions. **Raw example:** “Riya works in Pune.” **POS annotation:** Riya/PROPN works/VERB in/ADP Pune/PROPN. **NER annotation:** Riya/PERSON, Pune/LOCATION. Each annotation answers a different task. A good corpus is relevant to its domain, sufficiently representative, consistently annotated and appropriately split. More data alone do not repair duplicated examples, incorrect labels or a mismatch between training and real use. Corpus=text collection. Annotation=text ke saath labels. Metadata=collection ki information. | Resource | Contents | Typical use | | --- | --- | --- | | Gutenberg selection | Literary/book texts | Long-form language and vocabulary | | Web text | Informal domain-specific text | Study usage beyond edited books | | Chat corpus | Conversational posts | Dialogue, informal forms | | Brown | English texts across genres | Genre comparison | | Reuters | News with topic labels | Text classification; overlapping topics | | Inaugural | Speeches indexed by time | Vocabulary change across years | | Wordlist | Words/names/stopwords | Lexical filtering | | Pronouncing dictionary | Words and phonetic forms | Speech, stress, rhyme | **Gutenberg versus web/chat:** book language does not represent every spelling or conversational pattern in messages. **Brown versus Reuters:** Brown emphasizes genres; Reuters news can have overlapping topic labels. **Inaugural addresses:** dates provide a time dimension. Corpus sizes depend on version, subset and tokenization. Explain their purpose and structure rather than assuming a historical count is a permanent property. Gutenberg=books; web/chat=informal; Brown=genres; Reuters=news topics; Inaugural=time. Corpus readers expose several views. **raw** returns a character string, **words** returns token occurrences, and **sents** returns a list of sentences where each sentence contains tokens. | Conceptual method | Returned view | Its length counts | | --- | --- | --- | | fileids() | Document IDs | Files | | raw(fileid) | Character string | Characters | | words(fileid) | Token list | Token occurrences | | sents(fileid) | Sentence lists | Sentences | | categories() | Genre/topic labels | Categories | | tagged\_words(fileid) | Word–tag pairs | Annotated tokens | **Question. For sentence lists [the,cat,sleeps,.] and [the,dog,runs,.], count all tokens, sentences and lowercase alphabetic word types.** #### Given Punctuation is included in token count but excluded from word types. #### To find For sentence lists [the,cat,sleeps,.] and [the,dog,runs,.], count all tokens, sentences and lowercase alphabetic word types. #### Formula and why it applies Count every token occurrence, then distinct alphabetic forms. Different counting rules give different totals. #### Step 1. Token count 4+4=8. #### Step 2. Sentence count Two outer lists → 2. #### Step 3. Types {the,cat,sleeps,dog,runs} → 5. **Answer.** 8 tokens, 2 sentences, 5 word types. For “Cats run. Dogs sleep.”, a reader may return six tokens including two periods, but only four alphabetic word tokens. State the convention before calculating TTR or probabilities. raw=characters, words=tokens, sents=sentences. Teen alag counts. --- ## TEE · Q5b · Multinomial Naive Bayes Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f22--q5b/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-f22/q5b Previous examination · complete worked answers **Question. Explain Naive Bayes for text classification. Using the four documents below, classify “Chinese Chinese Chinese Tokyo Japan”: Chinese Beijing Chinese=yes; Chinese Chinese Shanghai=yes; Chinese Macao=yes; Tokyo Japan Chinese=no.** [10 marks] ### Answer Multinomial NB assumes token draws are conditionally independent given the class; it combines the class prior and class-specific word probabilities. Repeated Chinese contributes its likelihood three times. ### Given | Training text | Class | | --- | --- | | Chinese Beijing Chinese | yes | | Chinese Chinese Shanghai | yes | | Chinese Macao | yes | | Tokyo Japan Chinese | no | Test: Chinese Chinese Chinese Tokyo Japan ### To find Classify the test using multinomial Naive Bayes. ### Formula: which one and why Assume add-one smoothing unless the question supplies probabilities. Keep repeated words as repeated factors. Use training vocabulary only; ignore out-of-vocabulary words unless an unknown-token model is defined. $$ \mathrm{score}(c)=P(c)\prod_{w\in V}\left(\frac{c(w,c)+1}{N_c+|V|}\right)^{c(w,d)} $$ ### Step 1. Vocabulary and class priors V = {beijing, chinese, japan, macao, shanghai, tokyo}; |V|=6. | Class | Documents | Prior | | --- | --- | --- | | no | 1 | 1/4 | | yes | 3 | 3/4 | ### Step 2. Count every training word within each class | Class | Total tokens | beijing | chinese | japan | macao | shanghai | tokyo | | --- | --- | --- | --- | --- | --- | --- | --- | | no | 3 | 0 | 1 | 1 | 0 | 0 | 1 | | yes | 8 | 1 | 5 | 0 | 1 | 1 | 0 | ### Step 3. Count the test words | Word | Count | | --- | --- | | chinese | 3 | | tokyo | 1 | | japan | 1 | OOV words: none ### Step 4. Calculate likelihoods and multiply with the prior | Class | Prior | Individual factors | Exact score | Decimal score | | --- | --- | --- | --- | --- | | no | 1/4 | chinese: (1+1)/(3+1×6) = 2/9, used 3 time(s); tokyo: (1+1)/(3+1×6) = 2/9, used 1 time(s); japan: (1+1)/(3+1×6) = 2/9, used 1 time(s) | 8/59049 | 0.000135480702467 | | yes | 3/4 | chinese: (5+1)/(8+1×6) = 3/7, used 3 time(s); tokyo: (0+1)/(8+1×6) = 1/14, used 1 time(s); japan: (0+1)/(8+1×6) = 1/14, used 1 time(s) | 81/268912 | 0.000301213779973 | ### Step 5. Compare and optionally normalize Normalized scores: no = 0.310241388, yes = 0.689758612 Division by the same total does not change the winning class. ### Final answer **Predicted class: yes.** **Yaad rakho:** Count → prior → smoothed word chances → repeated factors → compare. --- ## TEE · Q6a · GloVe Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f22--q6a/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-f22/q6a Previous examination · complete worked answers **Question. Explain GloVe word embeddings.** [10 marks] ### Answer **GloVe (Global Vectors) learns word embeddings by fitting weighted relationships in a global word–context co-occurrence matrix.** Word2Vec uses local prediction examples; GloVe uses aggregate counts directly in its objective. $$ X_i=\sum_k X_{ik},\qquad P_{ik}=\frac{X_{ik}}{X_i}. $$ $$ X_{ik} $$ is how often context word $$ k $$ appears around target $$ i $$ . $$ X_i $$ is the total of row $$ i $$ . $$ P_{ik} $$ is the resulting context probability. The ratio $$ P(k\mid i)/P(k\mid j) $$ helps identify how a probe context differs between two words. For targets ice and steam, a probe such as solid should favour ice; gas should favour steam; a probe shared similarly by both gives a ratio near one. These are data tendencies, not universal numerical identities. **Question. For context solid, ice has count 12 out of row total 20; steam has count 2 out of row total 40. Compute P(solid|ice), P(solid|steam) and their ratio.** #### Given X(ice,solid)=12, X(ice)=20; X(steam,solid)=2, X(steam)=40. #### To find Two conditional probabilities and the ice/steam ratio. #### Formula and why it applies $$ P(k\mid i)=X_{ik}/X_i $$ , and $$ R=P(k\mid ice)/P(k\mid steam) $$ . #### Step 1. Ice probability $$ 12/20=0.6 $$ . #### Step 2. Steam probability $$ 2/40=0.05 $$ . #### Step 3. Ratio $$ 0.6/0.05=12 $$ . **Answer.** The probe solid is 12 times as probable in ice's context under these counts. **Question. For probe water, ice and steam both have row total 50. Counts are 10 and 10. Calculate the ratio.** #### Given Both probabilities use a denominator of 50. #### To find The probability ratio. #### Formula and why it applies Use row-normalized counts before taking the ratio. #### Step 1. Ice probability $$ 10/50=0.2 $$ . #### Step 2. Steam probability $$ 10/50=0.2 $$ . #### Step 3. Divide $$ 0.2/0.2=1 $$ . **Answer.** Ratio 1: this probe does not distinguish the targets in this toy data. **Question. For probe gas, counts are 1/100 around ice and 20/100 around steam. Compute the ice-to-steam ratio.** #### Given Conditional counts have the same row denominator. #### To find The ratio and interpretation. #### Formula and why it applies $$ R=P(gas\mid ice)/P(gas\mid steam) $$ . #### Step 1. Ice probability $$ 1/100=0.01 $$ . #### Step 2. Steam probability $$ 20/100=0.2 $$ . #### Step 3. Ratio $$ 0.01/0.2=0.05 $$ . **Answer.** Ratio 0.05, so gas is much more strongly associated with steam. Ratio >1: pehla word favoured. <1: doosra. ≈1: probe dono mein similar. --- ## TEE · Q6b · BoW and TF–IDF movie corpus Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f22--q6b/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-f22/q6b Previous examination · complete worked answers **Question. Find Bag of Words and TF–IDF for: “This movie is very scary and long”; “This movie is not scary and is slow”; “This movie is spooky and good”.** [10 marks] ### Answer ### Given | Document | Original text | | --- | --- | | 1 | This movie is very scary and long | | 2 | This movie is not scary and is slow | | 3 | This movie is spooky and good | Stopwords: none supplied ### To find BoW, IDF and TF–IDF ### Formula: which one and why Use lowercase alphabetic tokens and exactly the stated stoplist. If no stoplist is supplied, retain function words. Adopt raw-count TF and natural-log IDF; the paper does not prescribe another TF/log convention. $$ \mathrm{TFIDF}(w,d)=c(w,d)\ln\frac{N}{\mathrm{df}(w)} $$ A term shared by all N documents has IDF ln(1)=0. Counts and TF–IDF weights are different representations. ### Step 1. Tokenize each document | Document | Tokens | | --- | --- | | 1 | this, movie, is, very, scary, and, long | | 2 | this, movie, is, not, scary, and, is, slow | | 3 | this, movie, is, spooky, and, good | ### Step 2. Fix the vocabulary order and, good, is, long, movie, not, scary, slow, spooky, this, very ### Step 3. Count occurrences (Bag of Words) | Document | and | good | is | long | movie | not | scary | slow | spooky | this | very | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | 1 | 1 | 0 | 1 | 1 | 1 | 0 | 1 | 0 | 0 | 1 | 1 | | 2 | 1 | 0 | 2 | 0 | 1 | 1 | 1 | 1 | 0 | 1 | 0 | | 3 | 1 | 1 | 1 | 0 | 1 | 0 | 0 | 0 | 1 | 1 | 0 | ### Step 4. Calculate document frequency and IDF | Term | df | N / df | ln(N/df) | | --- | --- | --- | --- | | and | 3 | 1 | 0.000000000 | | good | 1 | 3 | 1.098612289 | | is | 3 | 1 | 0.000000000 | | long | 1 | 3 | 1.098612289 | | movie | 3 | 1 | 0.000000000 | | not | 1 | 3 | 1.098612289 | | scary | 2 | 3/2 | 0.405465108 | | slow | 1 | 3 | 1.098612289 | | spooky | 1 | 3 | 1.098612289 | | this | 3 | 1 | 0.000000000 | | very | 1 | 3 | 1.098612289 | ### Step 5. Multiply each TF by its IDF | Document | and | good | is | long | movie | not | scary | slow | spooky | this | very | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | 1 | 0.000000 | 0.000000 | 0.000000 | 1.098612 | 0.000000 | 0.000000 | 0.405465 | 0.000000 | 0.000000 | 0.000000 | 1.098612 | | 2 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 1.098612 | 0.405465 | 1.098612 | 0.000000 | 0.000000 | 0.000000 | | 3 | 0.000000 | 1.098612 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 1.098612 | 0.000000 | 0.000000 | ### Final answer The final table is the TF–IDF matrix in the declared vocabulary order. A different explicitly stated log base rescales all IDFs; length-normalized TF changes each document scale. --- ## TEE · Q7a · Information extraction, chunking and NER Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f22--q7a/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-f22/q7a Previous examination · complete worked answers **Question. Explain information-extraction architecture with a diagram. Explain chunking and named-entity recognition with examples.** [10 marks] ### Answer **Information extraction (IE) converts unstructured language into structured information such as named entities, relations and events.** A structured output has explicit fields or records that can be stored, searched or analysed. IE is broader than NER: finding two names does not yet determine how those entities are related. ### The three levels of extracted information | Level | What it identifies | Example | | --- | --- | --- | | Entity extraction / NER | A span and its entity type | Asha → person; Orbit Labs → organisation | | Relation extraction | A labelled connection between entities | works-for(Asha, Orbit Labs) | | Event extraction | An occurrence with roles and attributes | Joining: employee=Asha, employer=Orbit Labs, date=2025 | An **entity** is a participant such as a person or organisation. A **relation** links participants. An **event** describes something that happened, may happen or did not happen. An **event role** names a participant's function: buyer, seller, employee, employer, location or time. These labels must follow the chosen extraction schema. Raw text → Sentence/token processing → Entity spans + types → Reference resolution → Relations and event roles → Normalize and store records ### An extraction pipeline explained | Step | Operation | Reason | | --- | --- | --- | | Specify schema | Choose fields and allowed relations/events. | A system needs to know what records it is building. | | Preprocess | Segment sentences and tokenize without losing useful structure. | Locate expressions and boundaries. | | Recognise entities | Find spans and assign types. | Names can contain several words. | | Resolve references | Link pronouns/descriptions to the right entities. | Two mentions may denote one participant. | | Extract links/events | Use patterns, syntax or learned models. | Co-occurrence alone does not identify a relationship. | | Handle qualifiers | Preserve negation, time and uncertainty. | Not hired differs from hired. | | Normalize | Standardize dates, names or quantities where justified. | Equivalent forms should be comparable. | | Store/evaluate | Save records and compare them with annotated answers. | Wrong links matter even if entity names are correct. | ### Worked application 1 · turn a sentence into a record **Question. Extract entities and a joining event from “Asha joined Orbit Labs in Mumbai on 3 May 2025.” Use fields person, organisation, location, event and date.** #### Given One affirmative sentence; Orbit Labs is an organisation and Mumbai is a location. #### To find Entity spans/types and one structured joining record. #### Method and why it applies Find the complete entity spans, then assign their roles using the verb joined and its grammatical context. ### Solution #### Step 1. Mark entity spans | Span | Type | | --- | --- | | Asha | Person | | Orbit Labs | Organisation | | Mumbai | Location | | 3 May 2025 | Date | #### Step 2. Identify the event trigger Joined signals a joining event. The subject Asha is the joining person; Orbit Labs is the joined organisation. #### Step 3. Attach contextual fields In Mumbai supplies a location associated with the stated joining event. On 3 May 2025 supplies its date. Do not infer the company's permanent headquarters from this sentence alone. #### Step 4. Normalize the date The unambiguous date can be stored as 2025-05-03 while preserving its original text. #### Step 5. Build the record | Field | Value | | --- | --- | | event | joining | | person | Asha | | organisation | Orbit Labs | | location | Mumbai | | date | 2025-05-03 | #### Step 6. State the information limit The sentence supports a joining event on that date. By itself it does not prove that Asha still works there indefinitely. **Answer.** A joining record for Asha and Orbit Labs at the stated location/date, with complete entity spans preserved. ### Worked application 2 · direction of a relation matters **Question. From “Nova acquired Luma for ₹50 crore in 2025”, extract an acquisition event with buyer, acquired organisation, amount and year.** #### Given Nova and Luma are organisation names. The active verb is acquired; no seller is named. #### To find A correctly directed acquisition record. #### Method and why it applies Use grammatical roles to identify buyer and acquired organisation. Merely returning the unordered pair {Nova,Luma} loses the event direction. ### Solution #### Step 1. Recognise participant entities Nova is the subject organisation; Luma is the object organisation. #### Step 2. Locate the trigger Acquired identifies an acquisition event. #### Step 3. Assign roles | Role | Entity | | --- | --- | | Buyer/acquirer | Nova | | Acquired organisation | Luma | #### Step 4. Extract amount and time Amount text=₹50 crore; year=2025. Currency and magnitude belong together; do not drop crore or replace it by lakh. #### Step 5. Create the record | Field | Value | | --- | --- | | event | acquisition | | buyer | Nova | | acquired\_organisation | Luma | | amount | INR 50 crore | | year | 2025 | | seller | Not stated | #### Step 6. Reject an unsupported inference The sentence does not name the former owner/seller. Luma is the acquired organisation, not automatically the seller. **Answer.** Acquisition(buyer=Nova, acquired organisation=Luma, amount=INR 50 crore, year=2025). Seller remains unspecified. ### Worked application 3 · reference, negation and time **Question. Extract employment-related events from “Ravi joined Vega in 2021. He left the company in 2024. He did not join Orion.” Keep event type, person, organisation, date and polarity.** #### Given He refers to Ravi; the company refers to Vega. The last sentence explicitly negates joining Orion. #### To find A structured event list without claiming an unsupported current employer. #### Method and why it applies Resolve references, identify each event separately and preserve whether it is affirmed or negated. ### Solution #### Step 1. Resolve the repeated mentions | Mention | Referent | | --- | --- | | He in sentence 2 | Ravi | | the company | Vega | | He in sentence 3 | Ravi | #### Step 2. Extract the first event Joining; person=Ravi; organisation=Vega; date=2021; polarity=affirmed. #### Step 3. Extract the second event Leaving; person=Ravi; organisation=Vega; date=2024; polarity=affirmed. #### Step 4. Extract the third statement Joining; person=Ravi; organisation=Orion; date=not stated; polarity=negated. This must not create an affirmative employment relation with Orion. #### Step 5. Write the event table | Event | Person | Organisation | Date | Polarity | | --- | --- | --- | --- | --- | | joining | Ravi | Vega | 2021 | Affirmed | | leaving | Ravi | Vega | 2024 | Affirmed | | joining | Ravi | Orion | Not stated | Negated | #### Step 6. Interpret current status cautiously The text says that Ravi left Vega and did not join Orion. It supplies no other current employer. Do not fill that field by guessing. **Answer.** Three qualified event statements, including a negated Orion joining. No current employer can be established from this text. ### Approaches and their trade-offs | Approach | Mechanism | Advantage | Limitation | | --- | --- | --- | --- | | Dictionary/rule based | Match known names, trigger words and patterns such as person joined organisation. | Transparent and useful in a narrow domain. | New names, paraphrases and complex syntax can escape patterns. | | Supervised statistical | Learn entity/relation/event labels from annotated examples and features. | Combines several cues consistently. | Requires representative labels and careful feature design. | | Neural/contextual | Use contextual representations to classify spans and links or generate a structured record. | Can capture paraphrases and broader context. | May still invent or misassign fields; schema and factual checks remain necessary. | Applications include extracting company events from news, conditions/treatments from clinical text, skills from resumes and attributes from product descriptions. Extraction quality is measured against annotated records: an entity match can require the correct span and type; a relation match additionally requires the correct participants and direction. Precision measures how many extracted records are correct; recall measures how many gold records were found. For a 10-mark answer, begin with the definition and structured output, distinguish NER/relation/event extraction, draw and explain the pipeline, work through a record with roles and qualifiers, compare approaches, then state evaluation and limitations. “Extract keywords” alone is too narrow: keywords do not necessarily represent a factual relation. **Yaad rakho:** NER naam pakadta hai; relation kaun-kisse jodta hai; event kya hua batata hai. Negation aur date mat girao. **Chunking**, also called shallow parsing, groups adjacent tagged words into meaningful non-overlapping units, commonly noun phrases. It usually identifies local phrase boundaries without producing the complete hierarchical structure of a sentence. **Chinking** removes specified tokens from an existing candidate chunk, splitting or trimming it. The/DT small/JJ cat/NN sat/VBD on/IN the/DT mat/NN → NP: [The small cat] → Outside: sat on → NP: [the mat] A POS tagger labels individual words. A chunker groups words into phrases. A full parser shows how phrases relate recursively to form the entire sentence. These are related tasks with different outputs. | Operation | Input | Output | | --- | --- | --- | | POS tagging | The small cat | The/DT small/JJ cat/NN | | NP chunking | Tagged tokens | [NP The small cat] | | Chinking | A broad candidate chunk | Remove verbs/prepositions, keep noun-phrase segments | | Full parsing | Sentence + grammar | A nested tree with sentence, noun phrase and verb phrase nodes | **Rule-based chunking** uses tag patterns. Statistical or learned chunkers infer boundaries from labeled examples. A useful NP rule is an optional determiner, followed by zero or more adjectives, followed by a noun. In NLTK-style notation: `NP: {
?*}`. Here ? means zero or one occurrence; \* means zero or more; + means one or more when used. The braces mean “form a chunk around the matched sequence”. **Question. Apply the NP rule optional DT, zero or more JJ, then one NN to “The/DT quick/JJ brown/JJ fox/NN jumps/VBZ over/IN the/DT lazy/JJ dog/NN”.** #### Given Tagged input is supplied. Rule: DT? JJ\* NN. #### To find All noun-phrase chunks. #### Formula and why it applies `NP → DT? JJ* NN` The rule groups a noun with its preceding determiner and adjectives; a verb or preposition does not match this NP pattern. #### Solution ##### Step 1. Start at The The matches optional DT; quick and brown both match JJ\*; fox matches NN. ##### Step 2. Form the first NP **[NP The quick brown fox]** ##### Step 3. Skip nonmatching tokens jumps/VBZ and over/IN do not satisfy the NP pattern. ##### Step 4. Match the second NP the is DT, lazy is JJ, dog is NN: **[NP the lazy dog]**. **Final answer.** [NP The quick brown fox] jumps over [NP the lazy dog]. **Question. Using NP → DT? JJ\* NN, identify the chunks in “bright/JJ stars/NN shine/VBP and/CC a/DT moon/NN glows/VBZ”. Use the supplied tags exactly.** #### Given Input tags label stars as NN for this exercise. Rule requires optional DT, zero or more JJ, then NN. #### To find The matched NP spans. #### Formula and why it applies `NP → DT? JJ* NN` The optional determiner may be absent and the adjective repetition may be empty. #### Solution ##### Step 1. Match bright stars No determiner is present, which DT? permits. bright matches JJ; stars matches NN. ##### Step 2. Inspect shine and and VBP and CC fall outside the NP rule. ##### Step 3. Match a moon a matches DT; no adjective is present; moon matches NN. **Final answer.** Chunks are [NP bright stars] and [NP a moon]. With ordinary Penn tagging, stars is usually NNS; broaden the noun pattern to include NNS when that tag is used. **Question. Begin with one candidate chunk containing “The/DT cat/NN sat/VBD on/IN the/DT mat/NN”. Chink out verbs and prepositions. Show the remaining chunks.** #### Given Initial candidate: [The cat sat on the mat]. Remove tags VBD and IN. #### To find The chunk segments after exclusion. #### Formula and why it applies Chinking removes matched tokens from the existing chunk. Removing the middle verb/preposition region divides the original candidate into two remaining contiguous spans. #### Solution ##### Step 1. Locate the tokens to remove sat is VBD and on is IN. ##### Step 2. Remove those tokens from chunk membership The remaining token groups are “The cat” and “the mat”. sat and on still exist in the sentence; they are simply outside these chunks. ##### Step 3. Write the final chunked sentence **[The cat]** sat on **[the mat]**. **Final answer.** Two chunks remain: The cat; the mat. **Yaad rakho:** Chunking = andar group karo; chinking = unwanted tokens ko group se bahar karo. **Named Entity Recognition (NER)** identifies text spans referring to entities and assigns their types, such as PERSON, ORGANIZATION and LOCATION. Many practical schemas also include DATE, TIME, MONEY or domain-specific types. The exact allowed labels depend on the application's annotation scheme. NER has two parts: **boundary detection** finds where the entity starts and ends; **classification** assigns the appropriate type. A multiword name should normally be one entity span: “New York” is a location span, not two unrelated location decisions. Text: Mary joined Google in New York on Monday. → Mary → PERSON → Google → ORGANIZATION → New York → LOCATION → Monday → DATE Not every noun is a named entity. In “a developer works at Google”, developer is an ordinary job description while Google is a named organization. A specialized HR system may define a JOB\_TITLE category, but it must state that schema explicitly. NER identifies mentions; entity linking is the separate task of connecting a mention to a unique database identity. | Task | Question it answers | Example | | --- | --- | --- | | POS | What grammatical role does this token have? | Google/NNP | | Chunking | Which adjacent words form a phrase? | [NP a skilled developer] | | NER | Which span names an entity, and of what type? | [Google] ORGANIZATION | | Entity linking | Which real entity does this mention denote? | Connect Paris to the intended city or person record | **Applications.** NER structures search results and documents, extracts organizations/people from news, routes support tickets by product or organization, supports recommendation, and extracts specialized entities from healthcare or legal text. Context is essential: “Apple released a laptop” names an organization, while “she ate an apple” describes fruit. **Yaad rakho:** NER mein do kaam: naam kahan se kahan tak, aur kis category ka. --- ## TEE · Q7b · Distributional hypothesis and cosine Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f22--q7b/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-f22/q7b Previous examination · complete worked answers **Question. Justify “You shall know a word by the company it keeps.” Calculate cosine similarity of w₁=[0.2,0.1,0.3,0.4] and w₂=[0.3,0,0.2,0.5].** [10 marks] ### Answer Words occurring in similar contexts often have related meanings; vectors encode their context distributions. This is a statistical tendency, not proof that every similar-context pair is synonymous—antonyms can share contexts too. ### Given u=[0.2, 0.1, 0.3, 0.4]; v=[0.3, 0, 0.2, 0.5] ### To find Cosine similarity in the given coordinate order. ### Formula: which one and why $$ \cos(u,v)=\frac{u\cdot v}{\|u\|\|v\|} $$ ### Step 1. Multiply matching coordinates 0.2×0.3 + 0.1×0 + 0.3×0.2 + 0.4×0.5 = 0.32 ### Step 2. Calculate both squared lengths u: 0.2² + 0.1² + 0.3² + 0.4² = 0.30000000000000004; v: 0.3² + 0² + 0.2² + 0.5² = 0.38 ### Step 3. Take square roots and divide 0.32 / (√0.30000000000000004 × √0.38) = 0.947758204 ### Final answer **Cosine=0.947758204.** This measures direction, not a probability. --- ## TEE · Write the whole paper under exam conditions Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/mocks/timed-final-2025--instructions/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/mocks/timed-final-2025/instructions TEE · actual previous paper · 100 marks · 180 minutes Use this actual previous paper as a three-hour, 100-mark practice paper. Q1 is compulsory (20 marks); choose any four full questions from Q2–Q7 (20 marks each). Each original question and its complete answer follow. Read the question, write your answer in your notebook, then compare the working below it. Budget roughly 36 minutes per attempted 20-mark question, including its subparts. Start each full question on a fresh page, as the supplied papers instruct. This is an old paper used for timed practice, not a forecast of the next TEE. --- ## TEE · Q1A · NLU and NLG Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/mocks/timed-final-2025--final-2025-q1a/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/mocks/timed-final-2025/final-2025-q1a TEE · actual previous paper · 100 marks · 180 minutes Question. Differentiate between Natural Language Understanding (NLU) and Natural Language Generation (NLG) in NLP with suitable examples. [5 marks] ### Answer **Natural Language Understanding (NLU)** interprets language to identify its meaning, intent and relevant information. **Natural Language Generation (NLG)** produces language that expresses information in a useful form. | Basis | NLU | NLG | | --- | --- | --- | | Direction | Language → meaning or structured information | Meaning or structured information → language | | Typical input | A user utterance, document or sentence | Facts, a database record, a prompt or a representation | | Typical output | Intent, entities, sentiment or a semantic representation | A sentence, explanation, summary or dialogue response | | Main difficulty | Resolving ambiguity and context | Choosing accurate content and producing coherent, fluent wording | ### Example: a weather assistant User: “Will it rain in Mumbai tomorrow?” NLU recognises a weather-query intent, the location Mumbai and the date tomorrow. After a weather service supplies the forecast, NLG produces “Rain is likely in Mumbai tomorrow; carry an umbrella.” User language → NLU: identify request → Retrieve forecast → NLG: express answer The two functions often work together in a chatbot. Understanding the request and wording the reply are separate tasks; fluent generation alone does not prove correct understanding. **Yaad rakho:** NLU mein meaning nikalo; NLG mein meaning ko sentence banao. --- ## TEE · Q1B · Chunking and chinking Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/mocks/timed-final-2025--final-2025-q1b/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/mocks/timed-final-2025/final-2025-q1b TEE · actual previous paper · 100 marks · 180 minutes Question. Explain the concepts of chunking and chinking in Natural Language Processing (NLP). Discuss how Part-of-Speech (POS) tagging supports these processes, and illustrate your explanation with a suitable example. [5 marks] ### Answer **Chunking** groups adjacent tagged words into shallow phrases, such as noun phrases, without building a complete sentence parse. **Chinking** removes a specified sequence of tags or words from a chunk, splitting a larger candidate chunk when required. ### POS tags provide the pattern A POS tag describes a grammatical category: DT = determiner, JJ = adjective, NN = singular noun, VBD = past-tense verb and IN = preposition. A noun-phrase rule can be written as `NP: {
?*+}`: an optional determiner, zero or more adjectives, followed by one or more nouns. ### Worked example | Word | The | small | cat | sat | on | the | mat | | --- | --- | --- | --- | --- | --- | --- | --- | | Tag | DT | JJ | NN | VBD | IN | DT | NN | Applying the NP rule gives **[The small cat] sat on [the mat]**. The first chunk contains DT–JJ–NN; the second contains DT–NN. “sat on” is not included because it does not match the noun-phrase rule. For chinking, begin with one broad chunk: **[The small cat sat on the mat]**. Remove the VBD and IN sequence “sat on”, for example with `}+{`. The result is **[The small cat] sat on [the mat]**. POS tagging therefore supplies grammatical labels; chunking uses labels to select phrases, while chinking uses labels to exclude unwanted material. Incorrect tags can lead to incorrect chunks. **Yaad rakho:** Chunking = phrase ko jodo; chinking = beech ka unwanted part nikalo. --- ## TEE · Q1C · Hyponymy, hypernymy and WSD Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/mocks/timed-final-2025--final-2025-q1c/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/mocks/timed-final-2025/final-2025-q1c TEE · actual previous paper · 100 marks · 180 minutes Question. Define hyponymy and hypernymy with suitable linguistic examples. How can hypernymy be applied in Word Sense Disambiguation (WSD) to resolve the correct meaning of polysemous words? Provide an example? [5 marks] ### Answer **Hyponymy** is the “is-a-kind-of” relation from a specific concept to a broader concept. **Hypernymy** is the inverse relation from the broader concept to the specific one. “Rose” is a hyponym of “flower”; “flower” is a hypernym of “rose”. rose / tulip → flower → plant The arrow points towards a broader category. Hyponymy is not a part–whole relation: a wheel is part of a car, but it is not a kind of car. ### Using the hierarchy for WSD WSD selects the intended sense of a word from its context. Obtain candidate senses from a lexical resource such as WordNet, inspect their hypernym chains, and favour the chain most compatible with nearby words. 1. Sentence: “The mouse clicked the icon.” 2. Candidate 1: mouse → rodent → mammal → animal. Candidate 2: mouse → pointing device → computer peripheral. 3. The context words “clicked” and “icon” belong to the computer-use domain. They support the device hierarchy, rather than the animal hierarchy. 4. Select the computer-pointing-device sense. In “The mouse ate cheese,” the animal hierarchy is more compatible. Hypernym evidence supplies semantic generalisation; it is a useful signal rather than a guarantee, because short or unusual contexts may still be ambiguous. **Yaad rakho:** Hyponym specific hai; hypernym uska bada category hai. --- ## TEE · Q1D · Pragmatics and discourse Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/mocks/timed-final-2025--final-2025-q1d/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/mocks/timed-final-2025/final-2025-q1d TEE · actual previous paper · 100 marks · 180 minutes Question. Compare pragmatic and discourse analysis with the help of appropriate examples. [5 marks] ### Answer **Pragmatic analysis** interprets an utterance using the situation, speaker intention, shared knowledge and conversational expectations. **Discourse analysis** studies how multiple utterances form a connected text or conversation. | Basis | Pragmatic analysis | Discourse analysis | | --- | --- | --- | | Focus | Intended meaning in context | Relationships and coherence across utterances | | Useful information | Speaker, listener, place, time and common knowledge | Previous sentences, entities, topics and rhetorical relations | | Typical tasks | Interpret requests, implicatures and deixis | Resolve references and analyse cohesion, coherence and discourse structure | | Example | “It is cold here” may be a request to close a window | “Riya bought a book. She read it.” links She to Riya and it to book | In “Can you pass the salt?”, literal syntax asks about ability; pragmatic interpretation usually identifies a polite request. In “The road flooded. Therefore, the school closed,” discourse analysis identifies a cause–result link between the sentences. The levels interact: resolving “she” across a conversation may require both earlier text and real-world context. They are complementary rather than mutually exclusive. **Yaad rakho:** Pragmatics: bolne ka intention. Discourse: sentences ka connection. --- ## TEE · Q2A · Morpheme, lexeme and NER Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/mocks/timed-final-2025--final-2025-q2a/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/mocks/timed-final-2025/final-2025-q2a TEE · actual previous paper · 100 marks · 180 minutes Question. i. Differentiate between Morpheme and Lexeme with examples. ii. Discuss the significance of NER in three different NLP applications. [10 marks] ### Answer ### i. Morpheme and lexeme A **morpheme** is the smallest linguistic unit that contributes meaning or grammatical function. A **lexeme** is an abstract vocabulary item representing a family of inflected word forms. | Basis | Morpheme | Lexeme | | --- | --- | --- | | Unit | A meaningful component of a word | An abstract dictionary vocabulary item | | Examples | un-, happy, -ness; cat and plural -s | RUN covers run, runs, ran and running | | Independence | Free morphemes can stand alone; bound morphemes cannot | A lexeme is realised through one or more word forms | | NLP relevance | Morphological segmentation, stemming and analysis | Lemmatization and lexical meaning | “Cats” contains two morphemes: cat + plural -s. “cat” and “cats” are forms of the lexeme CAT. “Unhappiness” can be segmented into un + happy + ness, although its spelling changes at the boundary. A morpheme and a syllable are different: syllables concern pronunciation, while morphemes concern meaning or grammar. ### ii. Named Entity Recognition in applications **NER** locates spans that name entities and assigns types such as PERSON, ORGANIZATION, LOCATION, DATE or domain-specific categories. | Application | Entities and example | Why NER matters | | --- | --- | --- | | News information extraction | “Asha joined Infosys in Pune.” → Asha/PERSON, Infosys/ORG, Pune/LOCATION | Creates searchable people–organisation–location records and supports later relation extraction | | Healthcare document processing | Find disease, drug and dosage spans in a clinical note | Supports record indexing and medication extraction; specialist labels and validation are required | | Customer support and commerce | Extract product names, dates and order identifiers from “My Pixel arrived damaged on Monday.” | Routes complaints, links the correct product and fills structured support fields | NER answers “Which entity spans occur here, and what type are they?” It does not by itself establish every relation between those entities or resolve all names to unique real-world identities. **Yaad rakho:** Morpheme = word ka meaningful piece; lexeme = word family. NER = naam dhoondho aur type do. --- ## TEE · Q2B · Laplace-smoothed bigram probability Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/mocks/timed-final-2025--final-2025-q2b/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/mocks/timed-final-2025/final-2025-q2b TEE · actual previous paper · 100 marks · 180 minutes Question. Consider the following training corpus: 1. artists paint murals in city 2. musicians play songs in city 3. painters paint murals on walls 4. signs are near city Using a bigram language model with Laplace smoothing, evaluate the probability of the test sentence: “artists paint murals near city”. Assume the and for each sentence. [10 marks] ### Answer ### Given - artists paint murals in city - musicians play songs in city - painters paint murals on walls - signs are near city Test: artists paint murals near city. Add one start token and one end token to every training and test sentence. ### To find The Laplace-smoothed bigram probability of the complete test sentence, including its end token. ### Formula: which one and why $$ P_{+1}(w_i\mid w_{i-1})=\frac{C(w_{i-1},w_i)+1}{C(w_{i-1})+V} $$ C(previous,current) counts that adjacent pair in the training corpus. C(previous) counts its occurrences as a history. V is the vocabulary size. Add-one smoothing is necessary because an unseen pair must not force the whole sentence probability to zero. $$ P(w_1,\ldots,w_n,\langle/s\rangle\mid\langle s\rangle)=\prod_{i=1}^{n+1}P(w_i\mid w_{i-1}) $$ ### Solution ### Step 1. Fix the token and vocabulary convention Lowercase words and remove the final punctuation. The supplied course key counts both boundary tokens in V. Following that convention: are, artists, city, in, murals, musicians, near, on, paint, painters, play, signs, songs, walls, , gives **V = 16**. The start marker supplies history; it is not an extra probability factor before the first word. ### Step 2. List every required transition The complete test chain is → artists → paint → murals → near → city → . ### Step 3. Count first, then add one | Required bigram | History count | Pair count | Smoothed probability | | --- | --- | --- | --- | | → artists | 4 | 1 | (1+1)/(4+16) = 1/10 | | artists → paint | 1 | 1 | (1+1)/(1+16) = 2/17 | | paint → murals | 2 | 2 | (2+1)/(2+16) = 1/6 | | murals → near | 2 | 0 | (0+1)/(2+16) = 1/18 | | near → city | 1 | 1 | (1+1)/(1+16) = 2/17 | | city → | 3 | 3 | (3+1)/(3+16) = 4/19 | Count only adjacent words, never words that merely appear in the same sentence. The unseen pairs have count zero, so their numerator becomes one. ### Step 4. Multiply all six probabilities $$ P=\frac{2}{20}\times\frac{2}{17}\times\frac{3}{18}\times\frac{1}{18}\times\frac{2}{17}\times\frac{4}{19} $$ Start with subtotal 1. At each row, multiply the previous subtotal by exactly one next factor. Simplify the fraction after multiplication, but do not round it. | Factor being included | Previous subtotal × next factor = new subtotal | | --- | --- | | 1. → artists | $$ 1\times\frac{1}{10}=\frac{1}{10} $$ | | 2. artists → paint | $$ \frac{1}{10}\times\frac{2}{17}=\frac{1}{85} $$ | | 3. paint → murals | $$ \frac{1}{85}\times\frac{1}{6}=\frac{1}{510} $$ | | 4. murals → near | $$ \frac{1}{510}\times\frac{1}{18}=\frac{1}{9180} $$ | | 5. near → city | $$ \frac{1}{9180}\times\frac{2}{17}=\frac{1}{78030} $$ | | 6. city → | $$ \frac{1}{78030}\times\frac{4}{19}=\frac{2}{741285} $$ | The last subtotal is the complete product. Only now convert the fraction to a decimal. $$ P=\frac{2}{741285}\approx 2.69801763\times10^{-6} $$ ### Step 5. Interpret the result This is a joint sentence probability, so a small value is normal. Do not add transition probabilities; each word must occur in sequence, so the chain multiplies them. ### Final answer **P ≈ 2.69801763 × 10-6** with the stated course vocabulary convention. **Yaad rakho:** Pair count mein 1 jodo, previous-word count mein V; phir saare fractions multiply karo. **Convention note:** Some implementations exclude from the predicted vocabulary. Their V is one smaller and their numerical probability differs. In an exam, state V explicitly and follow the convention specified in the question or class solution. --- ## TEE · Q3A · Ambiguity and bottom-up parsing Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/mocks/timed-final-2025--final-2025-q3a/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/mocks/timed-final-2025/final-2025-q3a TEE · actual previous paper · 100 marks · 180 minutes Question. i. Explain the various types of ambiguities encountered in Natural Language Processing (NLP). Illustrate each type with suitable examples. ii. Compare top-down and bottom-up parsing and apply bottom-up syntax parsing approach to the sentence “A cat chased the mouse” and draw a parse tree using the grammar rules: S → NP VP; NP → Pronoun | Proper-Noun | Det Nominal; Nominal → Noun | Nominal Noun | Nominal PP; VP → Verb | Verb NP | VP PP; PP → Prep NP. Lexicon: Det → {A, the}; Noun → {cat, mouse}; Verb → {chased}. [10 marks] ### Answer ### i. Types of ambiguity Ambiguity exists when the same language expression permits more than one interpretation. The correct interpretation usually requires grammatical, semantic or contextual evidence. | Type | Meaning | Example and readings | | --- | --- | --- | | Lexical | One word has multiple meanings | “She visited the bank”: financial institution or riverbank | | Syntactic | The sentence permits different structural attachments | “I saw the man with a telescope”: I used the telescope, or the man had it | | Semantic / scope | A structured sentence permits different interpretations | “Every student read a book”: one shared book, or possibly a different book per student | | Referential | A referring expression has more than one candidate referent | “Asha told Riya that she won”: she could refer to Asha or Riya | | Pragmatic | Intention depends on the situation | “Can you open the door?” can ask about ability or request an action | ### ii. Parsing approaches | Aspect | Top-down | Bottom-up | | --- | --- | --- | | Start | The start symbol S | The input words | | Operation | Expand a nonterminal using grammar productions | Combine words and constituents using productions in reverse | | Success | Leaves match the complete input | All input is reduced to S | | Possible difficulty | Naive recursive descent can loop on left recursion or need backtracking | A naive shift-reduce parser may face shift/reduce conflicts | ### Given The five tokens are A / cat / chased / the / mouse. The supplied grammar permits determiner + nominal noun phrases and a verb + noun phrase verb phrase. ### To find A bottom-up sequence of reductions and a parse tree whose root is S and whose leaves preserve the input order. ### Rule: which one and why A production A → B C permits B C to be reduced to A. Bottom-up parsing begins with the words; lexical rules assign their categories, and phrase rules combine those categories. ### Step 1. Assign lexical categories A → Det; cat → Noun; chased → Verb; the → Det; mouse → Noun. ### Step 2. Build the first noun phrase cat is Noun. Reduce Noun to Nominal using Nominal → Noun. Then reduce Det Nominal, representing “A cat”, to NP. ### Step 3. Build the object noun phrase mouse is Noun. Reduce it to Nominal. Reduce “the” + “mouse”, now Det Nominal, to NP. ### Step 4. Build the verb phrase “chased” is Verb and “the mouse” is NP. Reduce Verb NP to VP. ### Step 5. Build the sentence The remaining constituents are NP (“A cat”) and VP (“chased the mouse”). Apply S → NP VP in reverse to obtain S. Every input token is used exactly once. SNPVPDetNominalVerbNPNounDetNominalNounAcatchasedthemouse Bottom-up parse tree for A cat chased the mouse. **Final answer:** [S [NP [Det A] [Nominal [Noun cat]]] [VP [Verb chased] [NP [Det the] [Nominal [Noun mouse]]]]]. **Yaad rakho:** Bottom-up: words se phrases, phrases se S. Top-down: S se words. --- ## TEE · Q3B · Co-occurrence matrix and cosine Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/mocks/timed-final-2025--final-2025-q3b/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/mocks/timed-final-2025/final-2025-q3b TEE · actual previous paper · 100 marks · 180 minutes Question. Consider the given paragraph: The teacher wrote on the board. The students listened to the teacher. The board was full of notes. The student asked a question. The teacher answered from the book. Target words: teacher, student, board, book. Context words: wrote, listened, notes, question. i. Construct a co-occurrence matrix for the target and context words. Take the window size as one whole sentence. ii. Evaluate the similarity between the vectors of the target words “teacher-student” and “teacher-board” using the cosine similarity. iii. Plot the target word vectors by considering only two dimensions: “wrote” and “notes”. [10 marks] ### Answer ### Given Five sentences, four target rows and four context columns. Treat students and student as the same target lemma, and count each target–context pair once when both occur in a sentence. ### To find The 4 × 4 sentence-window co-occurrence matrix, two cosine similarities and the requested 2-D plot. ### Formula: which one and why $$ M_{ij}=\sum_{s=1}^{5}\mathbf{1}[\text{target}_i\text{ and context}_j\text{ occur in sentence }s] $$ The indicator is 1 when both words appear in that sentence and 0 otherwise. Each matrix row is the target word’s vector: its list of four context counts. $$ \cos(\mathbf a,\mathbf b)=\frac{\mathbf a\cdot\mathbf b}{\|\mathbf a\|\|\mathbf b\|} $$ The dot product multiplies corresponding coordinates and adds them. The length is the square root of the sum of squared coordinates. Cosine compares directions while allowing different vector lengths. ### Solution ### Step 1. Record the contribution of each sentence | Sentence | Targets present | Relevant context | Counts added | | --- | --- | --- | --- | | 1 | teacher, board | wrote | teacher–wrote +1; board–wrote +1 | | 2 | student, teacher | listened | student–listened +1; teacher–listened +1 | | 3 | board | notes | board–notes +1 | | 4 | student | question | student–question +1 | | 5 | teacher, book | none | No requested context count | ### Step 2. Construct every requested column | Target | wrote | listened | notes | question | | --- | --- | --- | --- | --- | | teacher | 1 | 1 | 0 | 0 | | student | 0 | 1 | 0 | 1 | | board | 1 | 0 | 1 | 0 | | book | 0 | 0 | 0 | 0 | ### Step 3. Calculate teacher–student $$ \mathbf t\cdot\mathbf s=(1)(0)+(1)(1)+(0)(0)+(0)(1)=1 $$ $$ \|\mathbf t\|=\sqrt{1^2+1^2+0^2+0^2}=\sqrt2 $$ $$ \|\mathbf s\|=\sqrt{0^2+1^2+0^2+1^2}=\sqrt2 $$ $$ \cos(\mathbf t,\mathbf s)=\frac1{\sqrt2\sqrt2}=\frac12=0.5 $$ ### Step 4. Calculate teacher–board $$ \mathbf t\cdot\mathbf b=(1)(1)+(1)(0)+(0)(1)+(0)(0)=1 $$ $$ \|\mathbf b\|=\sqrt{1^2+0^2+1^2+0^2}=\sqrt2 $$ $$ \cos(\mathbf t,\mathbf b)=\frac1{\sqrt2\sqrt2}=0.5 $$ ### Step 5. Select only the two requested plot coordinates | Word | x = wrote | y = notes | | --- | --- | --- | | teacher | 1 | 0 | | student | 0 | 0 | | board | 1 | 1 | | book | 0 | 0 | 001122teacherstudent, bookboardwrotenotes The plot uses only the requested two coordinates; cosine uses all context columns. ### Final answer Both requested cosine similarities are **0.5**. Student and book overlap at the origin of the 2-D projection, but their complete four-dimensional vectors differ. The book row is zero, so a cosine involving book would be undefined without an explicit convention. **Yaad rakho:** Har sentence mein target aur context saath? Ek count. Cosine ke liye poori row lo, plot ke liye sirf do columns. --- ## TEE · Q4A · Three preprocessing techniques Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/mocks/timed-final-2025--final-2025-q4a/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/mocks/timed-final-2025/final-2025-q4a TEE · actual previous paper · 100 marks · 180 minutes Question. Explain any three basic text preprocessing techniques used in Natural Language Processing (NLP). Discuss the purpose of each technique and illustrate them with suitable examples. [10 marks] ### Answer Text preprocessing transforms raw text into a representation suitable for a chosen NLP task. The best steps depend on the task; removing information indiscriminately can reduce performance. ### 1. Tokenization Tokenization divides text into units called tokens, such as words, subwords or sentences. These units are required for counting words and building many model inputs. “The cats are sleeping.” can become [The, cats, are, sleeping, .] under a punctuation-preserving word tokenizer. A different tokenizer may split contractions or punctuation differently, so the convention must be consistent during training and prediction. ### 2. Lowercasing Lowercasing maps uppercase letters to lowercase so that variants such as “Book”, “BOOK” and “book” can share one feature. This reduces vocabulary duplication in a case-insensitive search system. However, “US” and “us” are different, and capitals help NER recognise names. Therefore lowercasing should be chosen according to the application. ### 3. Lemmatization Lemmatization maps an inflected word to its dictionary base form, often using its POS and context. Examples: cats → cat; running/VBG → run; better/JJR → good. It allows a search for “run” to match relevant inflections. A stemmer instead uses simpler truncation rules and can produce a non-word stem; lemmatization is linguistically informed and can need more processing. ### Combined example | Stage | Representation | | --- | --- | | Raw | The cats are running. | | Tokenized | The | cats | are | running | . | | Lowercased | the | cats | are | running | . | | Lemmatized with correct POS | the | cat | be | run | . | These steps do not automatically remove stop words or punctuation. Those are separate choices. For sentiment analysis, preserve negation such as “not”; for syntax parsing, function words and word order are often essential. **Yaad rakho:** Preprocess task dekh ke karo: har word ko blindly delete mat karo. --- ## TEE · Q4B · HMM emission, transition and sequence Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/mocks/timed-final-2025--final-2025-q4b/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/mocks/timed-final-2025/final-2025-q4b TEE · actual previous paper · 100 marks · 180 minutes Question. Consider the following POS tagged corpus. Apply Hidden Markov Model (HMM) approach to: 1. Cat/Noun chase/Verb mice/Noun 2. The/Det dog/Noun runs/Verb 3. A/Det cat/Noun eats/Verb cheese/Noun i. Construct the emission probability matrix. ii. Construct the transmission probability matrix. iii. Calculate the probability of sequence of events: The/Det cat/Noun eats/Verb mice/Noun. Note: Include start and end characters for all the sentences. [10 marks] ### Answer ### Given | Sentence | Tagged sequence | | --- | --- | | 1 | cat/Noun chase/Verb mice/Noun | | 2 | the/Det dog/Noun runs/Verb | | 3 | a/Det cat/Noun eats/Verb cheese/Noun | Test words: the cat eats mice. Given tag sequence: Det → Noun → Verb → Noun. Include Start and End for all sentences. Lowercase the corpus so sentence-initial Cat and cat are one word. ### To find Emission probabilities, transition probabilities and the joint probability of the specified tag-and-word sequence. ### Formula: which one and why $$ P(w\mid t)=\frac{C(t,w)}{C(t)} $$ An emission probability asks: among tokens carrying this tag t, what fraction are the word w? $$ P(t_j\mid t_i)=\frac{C(t_i,t_j)}{\sum_k C(t_i,t_k)} $$ A transition probability asks: among transitions leaving tag tᵢ, what fraction go to tⱼ? Include End in the outgoing counts. The paper says “transmission”; the standard HMM term is transition. $$ P(\mathbf w,\mathbf t)=P(t_1\mid\mathrm{Start})\prod_{i=1}^{n}P(w_i\mid t_i)\prod_{i=2}^{n}P(t_i\mid t_{i-1})P(\mathrm{End}\mid t_n) $$ ### Solution ### Step 1. Count tokens for each tag Det has 2 tokens; Noun has 5 tokens; Verb has 3 tokens. The repeated noun cat occurs twice with Noun; every other displayed noun occurs once. The three verb types each occur once. ### Step 2. Build the emission matrix | Tag / word | a | cat | chase | cheese | dog | eats | mice | runs | the | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | Det | 1/2 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1/2 | | Noun | 0 | 2/5 | 0 | 1/5 | 1/5 | 0 | 1/5 | 0 | 0 | | Verb | 0 | 0 | 1/3 | 0 | 0 | 1/3 | 0 | 1/3 | 0 | Each row sums to one. A zero means no occurrence under that tag in this unsmoothed training corpus; no add-one smoothing was requested. ### Step 3. Write the three tag chains Start → Noun → Verb → Noun → End Start → Det → Noun → Verb → End Start → Det → Noun → Verb → Noun → End ### Step 4. Count outgoing transitions | From | To Det | To Noun | To Verb | To End | Outgoing total | | --- | --- | --- | --- | --- | --- | | Start | 2 | 1 | 0 | 0 | 3 | | Det | 0 | 2 | 0 | 0 | 2 | | Noun | 0 | 0 | 3 | 2 | 5 | | Verb | 0 | 2 | 0 | 1 | 3 | ### Step 5. Divide each row by its outgoing total | From / To | Det | Noun | Verb | End | | --- | --- | --- | --- | --- | | Start | 2/3 | 1/3 | 0 | 0 | | Det | 0 | 1 | 0 | 0 | | Noun | 0 | 0 | 3/5 | 2/5 | | Verb | 0 | 2/3 | 0 | 1/3 | End is terminal, so it has no estimated outgoing row. The four nonterminal rows each sum to one. ### Step 6. Choose the factors for the given sequence | Event in order | Probability | | --- | --- | | Start → Det | 2/3 | | Det emits the | 1/2 | | Det → Noun | 1 | | Noun emits cat | 2/5 | | Noun → Verb | 3/5 | | Verb emits eats | 1/3 | | Verb → Noun | 2/3 | | Noun emits mice | 1/5 | | Noun → End | 2/5 | ### Step 7. Multiply, including the end transition $$ P=\frac23\times\frac12\times1\times\frac25\times\frac35\times\frac13\times\frac23\times\frac15\times\frac25 $$ Start with subtotal 1. At each row, multiply the previous subtotal by exactly one next factor. Simplify the fraction after multiplication, but do not round it. | Factor being included | Previous subtotal × next factor = new subtotal | | --- | --- | | 1. Start → Det | $$ 1\times\frac{2}{3}=\frac{2}{3} $$ | | 2. Det emits the | $$ \frac{2}{3}\times\frac{1}{2}=\frac{1}{3} $$ | | 3. Det → Noun | $$ \frac{1}{3}\times1=\frac{1}{3} $$ | | 4. Noun emits cat | $$ \frac{1}{3}\times\frac{2}{5}=\frac{2}{15} $$ | | 5. Noun → Verb | $$ \frac{2}{15}\times\frac{3}{5}=\frac{2}{25} $$ | | 6. Verb emits eats | $$ \frac{2}{25}\times\frac{1}{3}=\frac{2}{75} $$ | | 7. Verb → Noun | $$ \frac{2}{75}\times\frac{2}{3}=\frac{4}{225} $$ | | 8. Noun emits mice | $$ \frac{4}{225}\times\frac{1}{5}=\frac{4}{1125} $$ | | 9. Noun → End | $$ \frac{4}{1125}\times\frac{2}{5}=\frac{8}{5625} $$ | The last subtotal is the complete product. Only now convert the fraction to a decimal. $$ P=\frac8{5625}\approx0.0014222222 $$ ### Final answer **Joint sequence probability = 8/5625 ≈ 0.0014222222.** This is the probability of the specified words and tags together. It is not the probability of the word sequence summed over every possible tag path. **Yaad rakho:** Tag se tag = transition; tag se word = emission. Dono alternate multiply, end ko mat bhoolo. **Boundary reminder:** The initial factor is P(Det | Start) = 2/3, based on three sentence starts, and the final factor is P(End | Noun) = 2/5. The overall fraction of Det tokens, 2/10, is not the start probability. Using it and omitting End, as in the handwritten key’s last line, computes a different quantity. --- ## TEE · Q5A · TF–IDF matrix Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/mocks/timed-final-2025--final-2025-q5a/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/mocks/timed-final-2025/final-2025-q5a TEE · actual previous paper · 100 marks · 180 minutes Question. For the following two sentences: Sentence A: “The student read a science book in the library” Sentence B: “The teacher read a history book in the classroom” 1. Construct the TF-IDF matrix for these sentences after removing the stop words {The, a, in}. 2. Identify and comment on the important terms in each sentence based on their TF-IDF scores. [10 marks] ### Answer ### Given Two documents, N = 2. Remove the listed stop words case-insensitively, so both “The” and “the” are removed. Use raw count TF and natural-log IDF, the convention used in the course key. ### To find The term-by-document TF–IDF matrix and the terms that distinguish each sentence. ### Formula: which one and why $$ \mathrm{TF}(t,d)=C(t,d),\qquad \mathrm{IDF}(t)=\ln\frac{N}{\mathrm{df}(t)} $$ $$ \mathrm{TFIDF}(t,d)=\mathrm{TF}(t,d)\,\mathrm{IDF}(t) $$ C(t,d) is the number of occurrences of term t in document d. df(t) is the number of documents containing t, not its total token count. N is the number of documents. IDF downweights terms shared by many documents. ### Solution ### Step 1. Clean and list the tokens A: student, read, science, book, library. B: teacher, read, history, book, classroom. Each retained token appears once in its sentence. ### Step 2. Fix one common vocabulary order student, read, science, book, library, teacher, history, classroom. Each row must refer to the same term in both documents. ### Step 3. Count document frequency and calculate IDF | Term type | df | IDF calculation | | --- | --- | --- | | read, book | 2 | ln(2/2) = ln(1) = 0 | | Every other term | 1 | ln(2/1) = ln(2) ≈ 0.693147 | A term absent from a document has TF = 0. A present term has TF = 1 here. ### Step 4. Multiply TF by IDF | Term | TF in A | TF in B | IDF | TF–IDF A | TF–IDF B | | --- | --- | --- | --- | --- | --- | | student | 1 | 0 | 0.693147 | 0.693147 | 0 | | read | 1 | 1 | 0 | 0 | 0 | | science | 1 | 0 | 0.693147 | 0.693147 | 0 | | book | 1 | 1 | 0 | 0 | 0 | | library | 1 | 0 | 0.693147 | 0.693147 | 0 | | teacher | 0 | 1 | 0.693147 | 0 | 0.693147 | | history | 0 | 1 | 0.693147 | 0 | 0.693147 | | classroom | 0 | 1 | 0.693147 | 0 | 0.693147 | ### Step 5. Interpret the weights A is distinguished by student, science and library. B is distinguished by teacher, history and classroom. “read” and “book” have zero IDF in this tiny unsmoothed two-document collection because they appear in every document; that does not mean those words are meaningless in English. ### Final answer All six document-specific terms receive **0.693147** in their own document and 0 in the other. Shared read and book receive 0 in both. **Yaad rakho:** TF: is document mein kitni baar? IDF: kitne documents mein common? Rare-across-documents term ko zyada weight. If normalized TF is explicitly requested, divide each present-token count by five; each nonzero weight becomes 0.693147/5 = 0.1386294. Do not silently switch conventions. --- ## TEE · Q5B · Naïve Bayes word-sense classification Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/mocks/timed-final-2025--final-2025-q5b/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/mocks/timed-final-2025/final-2025-q5b TEE · actual previous paper · 100 marks · 180 minutes Question. Consider the sentence: “The bat hit the ball hard on the cricket field.” Two possible senses of “bat”: Sense 1: Bat (Sports equipment – cricket/baseball); Sense 2: Bat (Nocturnal flying mammal). | Sentence ID | Context words | Sense | | --- | --- | --- | | 1 | ball, cricket, score, stadium | Sports | | 2 | night, cave, wings, fly | Animal | | 3 | player, run, match, bat | Sports | | 4 | nocturnal, vampire, insect | Animal | | 5 | team, field, bowl, bat | Sports | | 6 | dark, flying, cave, night | Animal | a) Using the Naïve Bayes algorithm, classify the sense of “bat” in the given sentence. Apply Laplace Smoothing wherever applicable. b) Discuss the significance of Laplace Smoothing in Naïve Bayes Classification. [10 marks] ### Answer ### Given There are three Sports examples and three Animal examples. Treat training context lists as bag-of-words token counts. Exclude the ambiguous target “bat” from the test context. Use the fixed training vocabulary; ignore out-of-vocabulary test words hit and hard. The usable test features are ball, cricket and field. ### To find The posterior-score comparison for Sports and Animal, and the role of smoothing. ### Formula: which one and why $$ \mathrm{score}(s)=P(s)\prod_{w\in\mathrm{context}}P(w\mid s) $$ $$ P(w\mid s)=\frac{C(w,s)+1}{N_s+V} $$ P(s) is the class prior, C(w,s) counts occurrences of context word w in class s, Nₛ is the total number of context tokens in that class and V is the shared vocabulary size. Conditional independence lets us multiply word probabilities. The common denominator P(context) is unnecessary when only comparing classes. ### Solution ### Step 1. Calculate the priors $$ P(S)=\frac36=0.5,\qquad P(A)=\frac36=0.5 $$ ### Step 2. Count class tokens and vocabulary Sports has 4 + 4 + 4 = 12 tokens. Animal has 4 + 3 + 4 = 11 tokens. The shared training vocabulary has 20 distinct words: ball, cricket, score, stadium, player, run, match, bat, team, field, bowl, night, cave, wings, fly, nocturnal, vampire, insect, dark, flying. “bat”, “night” and “cave” repeat but count only once each towards V. ### Step 3. Look up each usable context word | Feature | Sports count | Animal count | P(word | Sports) | P(word | Animal) | | --- | --- | --- | --- | --- | | ball | 1 | 0 | (1+1)/(12+20) = 2/32 | (0+1)/(11+20) = 1/31 | | cricket | 1 | 0 | 2/32 | 1/31 | | field | 1 | 0 | 2/32 | 1/31 | ### Step 4. Multiply the Sports score $$ \mathrm{score}(S)=0.5\times\frac2{32}\times\frac2{32}\times\frac2{32}=0.0001220703125 $$ ### Step 5. Multiply the Animal score $$ \mathrm{score}(A)=0.5\times\frac1{31}\times\frac1{31}\times\frac1{31}\approx0.00001678359236 $$ ### Step 6. Compare the scores 0.0001220703125 is greater than 0.00001678359236. Therefore select Sports. Optional normalization gives P(Sports | retained features) ≈ 0.87913; normalization is not needed for the decision. ### Final answer The intended sense is **bat as sports equipment**. Ball, cricket and field favour that sense. **Yaad rakho:** Pehle class prior, phir har context-word likelihood multiply; bigger score wali sense choose karo. ### Why Laplace smoothing matters Without smoothing, all three test features have zero count in Animal. One zero factor makes the entire class score zero, even if other evidence could support that class. Add-one smoothing assigns nonzero probability to every word in the defined vocabulary and redistributes probability mass. It does not mean every unseen word outside the vocabulary can be inserted without changing the vocabulary model. ### If following the supplied paper key exactly The key also multiplies probabilities for hit and hard using the same denominators 32 and 31. That convention produces Sports = 1.19209290 × 10⁻⁷ and Animal ≈ 1.74647163 × 10⁻⁸, again selecting Sports. Its treatment of out-of-vocabulary words is not a normalized fixed-vocabulary model. State your feature convention clearly; the complete calculation above uses a consistent fixed training vocabulary. --- ## TEE · Q6A · Coreference, pronouns and Word2Vec Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/mocks/timed-final-2025--final-2025-q6a/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/mocks/timed-final-2025/final-2025-q6a TEE · actual previous paper · 100 marks · 180 minutes Question. i. Differentiate between co-reference resolution and pronoun resolution with suitable examples. ii. Explain the working principle of the Continuous Bag of Words (CBOW) model and Skipgram model in Word2Vec. Illustrate how it predicts the target word from its surrounding context. [10 marks] ### Answer ### i. Coreference and pronoun resolution **Coreference resolution** groups expressions that refer to the same entity, including names, descriptions and pronouns. **Pronoun resolution** identifies the referent of a pronoun; it is narrower in scope. | Expression / task | Example | | --- | --- | | Coreference cluster | “Dr Rao arrived. The surgeon opened her bag.” → {Dr Rao, the surgeon, her} | | Pronoun resolution | For the same text, resolve “her” to Dr Rao. | | Difference | Resolving the description “the surgeon” is coreference work even when no pronoun is involved. | ### ii. Word2Vec Word2Vec learns dense word embeddings by predicting words from local context. A dense embedding is a short learned list of real numbers for a word; words used in similar contexts can obtain related vectors. CBOW and Skip-gram use opposite prediction directions. | Feature | CBOW | Skip-gram | | --- | --- | --- | | Input | Surrounding context words | One centre word | | Prediction target | The centre word | Surrounding context words | | Training example | [the, cat, on, the] → sat | sat → the; sat → cat; sat → on; sat → the | | Representation | Combine context embeddings, often by averaging | Use the centre-word embedding for each context prediction | Example sentence: “The cat sat on the mat.” With a radius-two window around sat, the context is [The, cat, on, the]. In CBOW, obtain the four context embeddings, average or sum them, calculate scores for vocabulary words, and train the model to assign high probability to sat. In Skip-gram, use sat as input and create four centre–context training pairs. CBOW: nearby words → combine embeddings → predict centre word Skip-gram: centre word → its embedding → predict nearby words Training adjusts embedding parameters to improve prediction; the final word vectors are the learned representation. Thus the question’s phrase “predicts the target word from surrounding context” describes CBOW specifically; Skip-gram reverses that direction. **Yaad rakho:** CBOW: neighbours se centre. Skip-gram: centre se neighbours. --- ## TEE · Q6B · Hobbs: Ravi and Suresh Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/mocks/timed-final-2025--final-2025-q6b/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/mocks/timed-final-2025/final-2025-q6b TEE · actual previous paper · 100 marks · 180 minutes Question. i. Explain Hobb’s Algorithm. Discuss the steps with help of example and state its limitation. ii. Sentence: Ravi met Suresh at the station after he arrived from Delhi. Using Hobbs’ algorithm, determine the most likely antecedent of the pronoun “he”. Show the steps of traversal according to the algorithm. [10 marks] ### Answer ### i. Method **Hobbs’ algorithm** is a syntax-based pronoun-resolution procedure that searches constituency parse trees in a specified order. It proposes noun phrases as antecedents and filters incompatible candidates, for example by number and gender agreement. **Read the tree labels first:** NP means noun phrase; S means sentence/clause; VP means verb phrase. X is the current NP or S search node, and p is the upward path used to reach it. Breadth-first means shallower nodes before deeper nodes; left-to-right fixes the order at the same depth. An N-bar (N′) node is the noun-centred part of an NP in a parse using that label. ### Complete ordered procedure 1. **Locate the pronoun.** Start at the NP immediately dominating the pronoun: the smallest NP containing it. 2. **Make the first climb.** Move upward to the first NP or S above the starting NP. Call it X and record the path p. 3. **Search initially to the left.** Traverse branches below X to the left of p, breadth-first and left-to-right. Propose an NP only if another NP or S lies between that candidate and X. Check agreement and other applicable constraints for each proposed candidate; accept the first eligible compatible one. 4. **Check the highest sentence.** If X is the highest S and no candidate has been accepted, search previous sentence trees, most recent first. In each, traverse breadth-first and left-to-right and propose NPs. Accept the first eligible compatible candidate; if the available context contains none, report no resolved antecedent. 5. **Otherwise climb again.** Move from X to the next NP or S above it. Make that node the new X and record the new upward path p. 6. **Test a higher NP.** If the new X is an NP, propose X only when p did not pass through its nominal N-bar child. Apply the same compatibility checks. Use the labels in the supplied parse; do not invent a missing N-bar branch. 7. **Search left again.** Traverse the new X’s branches to the left of p, breadth-first and left-to-right, proposing NPs. The extra intervening-NP/S requirement in Step 3 belongs to the initial search. 8. **Search permitted right branches.** If X is an S, traverse branches to the right of p, breadth-first and left-to-right, but do not descend below any NP or S encountered. Propose the encountered eligible NPs and test their compatibility. 9. **Repeat or finish.** Return to Step 4 until a compatible candidate is accepted or the available sentence trees are exhausted. Record the candidate order and why each candidate was accepted or rejected. **Why the initial restriction matters:** in “John saw him”, the subject NP John is directly under the clause S, with no additional NP or S between them. Step 3 therefore does not propose John as the ordinary local antecedent of him. A reflexive such as himself is a different grammatical case. The worked question below shows the specific parse and traversal; selecting an antecedent is not simply choosing the nearest name. ### ii. Apply the method **Given:** the pronoun he occurs as the subject of the subordinate clause “after he arrived from Delhi”. Assume that this clause is attached to the main verb phrase. The relevant main-clause NPs are Ravi, Suresh and the station. **To find:** the first eligible singular-male antecedent in Hobbs’ syntactic search order. ``` S (main) ├── NP: Ravi ← first higher-left candidate └── VP ├── VP: met [NP Suresh] at [NP the station] └── SBAR: after └── S (embedded) ← first X ├── NP: he ← start └── VP: arrived from Delhi ``` ### Step 1. Start at the pronoun NP Locate NP “he” inside the after-clause. Move to that clause’s S node. No earlier noun phrase in this small clause provides a candidate. ### Step 2. Move to the higher sentence The embedded S is not the highest S, so Step 5 climbs through SBAR and VP to the main S. Step 7 now searches to the left of that new path. This is the later left search, so the extra initial-search restriction from Step 3 is not imposed again. ### Step 3. Examine candidates in structural order The main subject NP Ravi is encountered before the deeper object NP Suresh in the relevant breadth-first search. Ravi is singular and compatible with “he”. “the station” would fail the ordinary person constraint. ### Step 4. Choose the first compatible candidate Select Ravi under the assumed parse and agreement settings. Suresh remains a possible interpretation in ordinary language, but the syntactic preference chooses Ravi first. ### Limitations - The result depends on the accuracy and attachment choices of the parse tree. - Syntactic ordering alone does not model all semantic plausibility, discourse focus or world knowledge. - Agreement can eliminate candidates but cannot distinguish two compatible male singular people. - Long-distance references and sentences with multiple possible parses can remain ambiguous. **Final answer:** Hobbs’ syntactic preference is **Ravi** for this analysis. This is an algorithmic selection, not proof of the speaker’s intended referent. **Yaad rakho:** Nearest naam blindly nahi: tree ki search order aur agreement follow karo. --- ## TEE · Q7A · Text classification pipeline Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/mocks/timed-final-2025--final-2025-q7a/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/mocks/timed-final-2025/final-2025-q7a TEE · actual previous paper · 100 marks · 180 minutes Question. Explain the working of text classification in Natural Language Processing (NLP) and discuss the key steps involved in building a text classification system in context of real-world applications such as spam detection or news categorization. [10 marks] ### Answer **Text classification** assigns one or more predefined labels to a document or message based on its content. In spam detection the labels may be spam and not-spam; in news categorization they may be sports, business and politics. Labelled text → Prepare and split data → Represent text → Train classifier → Evaluate → Classify new text 1. **Define the labels and task.** Decide whether each text gets one label or multiple labels. Establish what counts as spam so annotators use a consistent rule. 2. **Collect and label representative examples.** Gather messages from the intended application and assign reliable labels. Include enough examples of each class and protect private information. 3. **Split training, validation and test data.** Use training data to learn, validation data to choose settings and an untouched test set for final evaluation. Keep duplicate or closely related texts from leaking between splits. 4. **Preprocess consistently.** Tokenize and normalize only as appropriate. In spam detection, URLs, currency symbols and capitalisation may contain useful information. 5. **Create features.** A classical baseline uses BoW or TF–IDF. A neural system may use embeddings or a pretrained contextual model. Fit the vocabulary and IDF on training data only. 6. **Train a classifier.** For example, multinomial Naïve Bayes estimates class priors and word likelihoods; logistic regression learns a decision boundary. Training uses the labelled examples. 7. **Evaluate.** Inspect a confusion matrix, precision, recall and F1 as well as accuracy. False spam flags may hide legitimate mail, so the costs of errors matter. 8. **Deploy and monitor.** Apply the same preprocessing and feature mapping to new texts. Monitor changing vocabulary and error patterns, and retrain when suitable new labelled data becomes available. ### Example prediction A message such as “Claim your free prize now” is transformed into its feature representation. The classifier compares the evidence for the two labels and outputs spam if its decision score crosses the chosen threshold. The label comes from learned patterns; the mere presence of one word should not be treated as a universally correct rule. **Yaad rakho:** Labelled examples → features → model → evaluation → new text ka label. --- ## TEE · Q7B · Corpora and Jaccard similarity Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/mocks/timed-final-2025--final-2025-q7b/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/mocks/timed-final-2025/final-2025-q7b TEE · actual previous paper · 100 marks · 180 minutes Question. i. Differentiate between a flat text corpus and an annotated corpus, and discuss how each can be applied in various NLP tasks with suitable examples. ii. Explain a real-world scenario where the Jaccard Similarity measure is more appropriate than Euclidean or Manhattan distance, and justify why Jaccard is the best choice in that case. [10 marks] ### Answer ### i. Flat and annotated corpora A **corpus** is a collection of language data. A flat text corpus stores text without the linguistic labels needed by a particular task. An annotated corpus adds labels or structure such as POS tags, entity spans, parse trees or sentence sentiment. | Basis | Flat text corpus | Annotated corpus | | --- | --- | --- | | Example | A collection of plain news articles | The same articles with PERSON/ORG spans marked | | Information | The original text and possibly simple metadata | Text plus explicit task-related linguistic labels | | Applications | Word frequency, collocations and unsupervised language-model training | Supervised POS tagging, NER or parser training and evaluation | | Preparation | Relatively inexpensive to gather, but still needs cleaning | Requires annotation guidelines, labour and quality checks | “Flat” describes the lack of the annotations being discussed, not an absence of all file organization. Both corpus types may still have document boundaries and metadata. ### ii. Jaccard in a set-based application Consider recommending articles using their topic-tag sets. Article A has {AI, NLP, ethics}; article B has {AI, NLP, healthcare}. We care about overlap among present tags, not how many irrelevant tags are absent from both. ### Given A = {AI, NLP, ethics}; B = {AI, NLP, healthcare}. ### To find The fraction of distinct present tags shared by the articles. ### Formula and why $$ J(A,B)=\frac{|A\cap B|}{|A\cup B|} $$ The intersection counts common items; the union counts all distinct items present in either set. ### Step 1. Find common tags A ∩ B = {AI, NLP}, so the intersection size is 2. ### Step 2. Find all distinct present tags A ∪ B = {AI, NLP, ethics, healthcare}, so the union size is 4. ### Step 3. Divide $$ J(A,B)=\frac24=0.5 $$ Jaccard directly measures proportional overlap and ignores joint absences. Euclidean or Manhattan distance on binary tag vectors counts mismatches but does not by itself normalize by the union size. If frequency or magnitude were meaningful, those distance measures could be more appropriate. Here, membership-only tags and sparse sets make Jaccard a suitable choice. **Yaad rakho:** Jaccard: common present items / total distinct present items. --- ## TEE · Write the whole paper under exam conditions Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/mocks/timed-reexam-2026--instructions/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/mocks/timed-reexam-2026/instructions TEE · actual previous paper · 100 marks · 180 minutes Use this actual previous paper as a three-hour, 100-mark practice paper. Q1 is compulsory (20 marks); choose any four full questions from Q2–Q7 (20 marks each). Each original question and its complete answer follow. Read the question, write your answer in your notebook, then compare the working below it. Budget roughly 36 minutes per attempted 20-mark question, including its subparts. Start each full question on a fresh page, as the supplied papers instruct. This is an old paper used for timed practice, not a forecast of the next TEE. --- ## TEE · Q1A · Generic NLP pipeline Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/mocks/timed-reexam-2026--reexam-2026-q1a/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/mocks/timed-reexam-2026/reexam-2026-q1a TEE · actual previous paper · 100 marks · 180 minutes Question. Draw block diagram of generic NLP pipeline and explain the task associated with each block. [5 marks] ### Answer An NLP pipeline is an ordered workflow that transforms language data into a task-specific result. The exact components depend on the application; a generic supervised pipeline is shown below. Collect text → Clean and preprocess → Represent text → Train / apply model → Evaluate and deliver output | Block | Task | Example for sentiment analysis | | --- | --- | --- | | Data acquisition | Collect representative text and any required labels | Reviews labelled positive or negative | | Cleaning and preprocessing | Handle noise, tokenize and normalize appropriately | Remove HTML while preserving “not” | | Text representation | Convert tokens into usable model features | TF–IDF values or contextual embeddings | | Modeling | Learn patterns from training data and predict new labels | Train a classifier on labelled review features | | Evaluation | Measure performance on unseen held-out data and inspect errors | Precision, recall, F1 and confusion matrix | | Deployment and feedback | Deliver predictions and monitor changing data | Show review sentiment and monitor new product vocabulary | Training/validation/test separation must precede fitting the learned preprocessing and feature statistics. New inputs must use the same feature mapping as training. Not every task uses every linguistic level explicitly: a modern end-to-end model may combine several operations internally. **Yaad rakho:** Text lo → prepare karo → numbers banao → model lagao → result check aur use karo. --- ## TEE · Q1B · Content and function words Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/mocks/timed-reexam-2026--reexam-2026-q1b/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/mocks/timed-reexam-2026/reexam-2026-q1b TEE · actual previous paper · 100 marks · 180 minutes Question. i. Define function words and content words. ii. Which category (function words and content words) belong to open class and closed class? Justify your answer. iii. For the sentence “The small cat is sleeping under the tree”, identify the function words and content words. [5 marks] ### Answer **Content words** carry much of a sentence’s lexical meaning, usually including nouns, main verbs, adjectives and many adverbs. **Function words** primarily express grammatical relationships, including determiners, prepositions, conjunctions, pronouns and auxiliary verbs. | Category | Typical class | Reason | | --- | --- | --- | | Content words | Open class | New nouns, verbs and adjectives enter the language readily, such as new technology names | | Function words | Closed class | The grammatical inventory is relatively stable; new determiners or pronouns are much less frequently added | | Word | Role in this sentence | Category | | --- | --- | --- | | The | Determiner | Function | | small | Adjective | Content | | cat | Noun | Content | | is | Auxiliary supporting progressive tense | Function | | sleeping | Main verb | Content | | under | Preposition | Function | | the | Determiner | Function | | tree | Noun | Content | **Final answer:** content words = small, cat, sleeping, tree; function words = The, is, under, the. “is” is auxiliary here; a word’s role must be judged in context. “Closed” means relatively resistant to new members, not that change is logically impossible. **Yaad rakho:** Content words scene batate hain; function words grammar ka connection banate hain. --- ## TEE · Q1C · Word2Vec and GloVe Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/mocks/timed-reexam-2026--reexam-2026-q1c/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/mocks/timed-reexam-2026/reexam-2026-q1c TEE · actual previous paper · 100 marks · 180 minutes Question. Differentiate between word2vec and GloVe in terms of their learning strategies. Which type of corpus would be more suitable for each method, and why? [5 marks] ### Answer Both Word2Vec and GloVe learn dense word vectors from distributional information: words occurring in related contexts tend to receive related representations. They differ mainly in the training objective and how corpus co-occurrence evidence is used. | Basis | Word2Vec | GloVe | | --- | --- | --- | | Learning strategy | Predictive: CBOW predicts a centre word; Skip-gram predicts nearby words | Count-based objective fitted to global word–context co-occurrence statistics | | Training evidence | Local context windows converted into prediction examples | A corpus-wide co-occurrence matrix and relationships among counts | | Parameters learned | Word/context embeddings that make context predictions accurate | Word/context embeddings and biases that fit weighted log co-occurrence counts | | Corpus considerations | Large token sequences with useful local context, including domain-specific text | A sufficiently large corpus from which reliable global co-occurrence statistics can be collected | For a domain such as medicine, either method can benefit from a representative medical corpus. Word2Vec is convenient when generating local training windows directly from a stream of tokens. GloVe is natural when a global sparse co-occurrence matrix can be constructed and stored. Corpus size, domain match, preprocessing and available compute matter; there is no rule that Word2Vec requires informal text while GloVe requires formal text. Both standard forms produce one static vector per vocabulary word, so multiple senses are blended. They do not automatically provide a different vector for the same word in each sentence. **Yaad rakho:** Word2Vec predicts local neighbours; GloVe fits global co-occurrence counts. --- ## TEE · Q1D · Anaphora, cataphora and coreference Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/mocks/timed-reexam-2026--reexam-2026-q1d/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/mocks/timed-reexam-2026/reexam-2026-q1d TEE · actual previous paper · 100 marks · 180 minutes Question. Differentiate between cataphora and anaphora in terms of direction of reference. How coreference resolution is different from anaphora reference? [5 marks] ### Answer **Anaphora** points backward to an expression or information introduced earlier. **Cataphora** points forward to an expression introduced later. | Relation | Example | Direction | | --- | --- | --- | | Anaphora | “Riya arrived. She smiled.” She refers to the earlier Riya. | Backward | | Cataphora | “Before she spoke, Riya checked the microphone.” She refers to the later Riya. | Forward | **Coreference** means two or more expressions refer to the same entity. Coreference resolution groups such expressions, regardless of whether they are names, descriptions or pronouns and regardless of which is mentioned first. For “Dr Sen arrived. The surgeon smiled. She waved,” {Dr Sen, the surgeon, She} is a coreference cluster. The latter two mentions are also backward-looking references. Coreference is an identity relation; anaphora describes a dependency on prior discourse and is not always identity. For example, “I bought a house. The roof leaks” contains a bridging reference: the roof depends on the house for interpretation, but roof and house are not the same entity. **Yaad rakho:** Ana = pehle ki taraf; cata = aage ki taraf; coreference = same entity. --- ## TEE · Q2A · NLP tasks and syntax versus semantics Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/mocks/timed-reexam-2026--reexam-2026-q2a/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/mocks/timed-reexam-2026/reexam-2026-q2a TEE · actual previous paper · 100 marks · 180 minutes Question. i. Discuss five different tasks associated with Natural Language Processing. ii. Compare and contrast syntactic analysis and semantic analysis in NLP. Support your explanation with examples. [10 marks] ### Answer ### i. Five NLP tasks | Task | What it does | Example | | --- | --- | --- | | Machine translation | Expresses source-language content in a target language | English “Good morning” → Hindi greeting | | Sentiment analysis | Identifies the opinion or attitude expressed | “The service was excellent” → positive sentiment | | Named Entity Recognition | Finds and types entity spans | “Asha joined Google” → PERSON and ORGANIZATION | | Question answering | Returns an answer supported by a question’s relevant information | From a passage, identify who discovered a described result | | Text classification | Assigns predefined category labels | Classify an email as spam or legitimate | ### ii. Syntactic and semantic analysis | Basis | Syntactic analysis | Semantic analysis | | --- | --- | --- | | Main question | How are words grammatically organised? | What do the words and sentence mean? | | Typical output | POS tags, constituency or dependency structure | Word senses, semantic relations or a meaning representation | | Knowledge used | Grammar rules and structural patterns | Lexical meaning, context and semantic constraints | | Example | Identify “The boy” as subject NP and “ate an apple” as VP | Interpret the boy as eater and apple as the thing eaten | “Colourless green ideas sleep furiously” can have a plausible English grammatical structure while remaining semantically unusual. Syntax alone therefore does not guarantee ordinary meaning. Conversely, “boy eat apple yesterday” conveys a possible event despite grammatical errors. The analyses support each other. In “I saw the man with a telescope,” syntactic PP attachment determines which meaning is available, and semantic knowledge can help choose a plausible attachment. A full NLP system often needs both. **Yaad rakho:** Syntax = sentence ka structure; semantics = us structure ka meaning. --- ## TEE · Q2B · Laplace smoothing on a second corpus Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/mocks/timed-reexam-2026--reexam-2026-q2b/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/mocks/timed-reexam-2026/reexam-2026-q2b TEE · actual previous paper · 100 marks · 180 minutes Question. Consider the following training corpus: 1. cats chase mice in the garden 2. dogs bark loudly at night 3. birds sing in the morning 4. children play games in the park Using a bigram language model with Laplace smoothing, evaluate the probability of the test sentence: “cats bark in the park.” Assume and as start and end of the sentence. [10 marks] ### Answer ### Given - cats chase mice in the garden - dogs bark loudly at night - birds sing in the morning - children play games in the park Test: cats bark in the park. Add one start token and one end token to every training and test sentence. ### To find The Laplace-smoothed bigram probability of the complete test sentence, including its end token. ### Formula: which one and why $$ P_{+1}(w_i\mid w_{i-1})=\frac{C(w_{i-1},w_i)+1}{C(w_{i-1})+V} $$ C(previous,current) counts that adjacent pair in the training corpus. C(previous) counts its occurrences as a history. V is the vocabulary size. Add-one smoothing is necessary because an unseen pair must not force the whole sentence probability to zero. $$ P(w_1,\ldots,w_n,\langle/s\rangle\mid\langle s\rangle)=\prod_{i=1}^{n+1}P(w_i\mid w_{i-1}) $$ ### Solution ### Step 1. Fix the token and vocabulary convention Lowercase words and remove the final punctuation. The supplied course key counts both boundary tokens in V. Following that convention: at, bark, birds, cats, chase, children, dogs, games, garden, in, loudly, mice, morning, night, park, play, sing, the, , gives **V = 20**. The start marker supplies history; it is not an extra probability factor before the first word. ### Step 2. List every required transition The complete test chain is → cats → bark → in → the → park → . ### Step 3. Count first, then add one | Required bigram | History count | Pair count | Smoothed probability | | --- | --- | --- | --- | | → cats | 4 | 1 | (1+1)/(4+20) = 1/12 | | cats → bark | 1 | 0 | (0+1)/(1+20) = 1/21 | | bark → in | 1 | 0 | (0+1)/(1+20) = 1/21 | | in → the | 3 | 3 | (3+1)/(3+20) = 4/23 | | the → park | 3 | 1 | (1+1)/(3+20) = 2/23 | | park → | 1 | 1 | (1+1)/(1+20) = 2/21 | Count only adjacent words, never words that merely appear in the same sentence. The unseen pairs have count zero, so their numerator becomes one. ### Step 4. Multiply all six probabilities $$ P=\frac{2}{24}\times\frac{1}{21}\times\frac{1}{21}\times\frac{4}{23}\times\frac{2}{23}\times\frac{2}{21} $$ Start with subtotal 1. At each row, multiply the previous subtotal by exactly one next factor. Simplify the fraction after multiplication, but do not round it. | Factor being included | Previous subtotal × next factor = new subtotal | | --- | --- | | 1. → cats | $$ 1\times\frac{1}{12}=\frac{1}{12} $$ | | 2. cats → bark | $$ \frac{1}{12}\times\frac{1}{21}=\frac{1}{252} $$ | | 3. bark → in | $$ \frac{1}{252}\times\frac{1}{21}=\frac{1}{5292} $$ | | 4. in → the | $$ \frac{1}{5292}\times\frac{4}{23}=\frac{1}{30429} $$ | | 5. the → park | $$ \frac{1}{30429}\times\frac{2}{23}=\frac{2}{699867} $$ | | 6. park → | $$ \frac{2}{699867}\times\frac{2}{21}=\frac{4}{14697207} $$ | The last subtotal is the complete product. Only now convert the fraction to a decimal. $$ P=\frac{4}{14697207}\approx 2.72160554\times10^{-7} $$ ### Step 5. Interpret the result This is a joint sentence probability, so a small value is normal. Do not add transition probabilities; each word must occur in sequence, so the chain multiplies them. ### Final answer **P ≈ 2.72160554 × 10-7** with the stated course vocabulary convention. **Yaad rakho:** Pair count mein 1 jodo, previous-word count mein V; phir saare fractions multiply karo. **Convention note:** Some implementations exclude from the predicted vocabulary. Their V is one smaller and their numerical probability differs. In an exam, state V explicitly and follow the convention specified in the question or class solution. --- ## TEE · Q3A · Top-down parsing Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/mocks/timed-reexam-2026--reexam-2026-q3a/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/mocks/timed-reexam-2026/reexam-2026-q3a TEE · actual previous paper · 100 marks · 180 minutes Question. In the context of natural language processing, explain the concept of top-down and bottom-up parsing approaches. Apply top-down syntax parsing approach to the sentence “The tall boy saw a girl in the park.” and draw a parse tree using the CFG grammar rules: S → NP VP; NP → Det Adj N | Det N; VP → V NP | VP PP; PP → P NP. Lexicon: Det → {the | a}; Adj → {tall}; N → {boy | girl | park}; V → {saw}; P → {in}. [10 marks] ### Answer | Aspect | Top-down | Bottom-up | | --- | --- | --- | | Start | The start symbol S | The input words | | Operation | Expand a nonterminal using grammar productions | Combine words and constituents using productions in reverse | | Success | Leaves match the complete input | All input is reduced to S | | Possible difficulty | Naive recursive descent can loop on left recursion or need backtracking | A naive shift-reduce parser may face shift/reduce conflicts | ### Given The sentence contains the tokens the / tall / boy / saw / a / girl / in / the / park. Lowercase the initial “The” to match the lexicon. The full stop is punctuation outside the listed grammar. ### To find A derivation beginning at S and a parse tree matching the whole sentence. ### Rule: which one and why A context-free production replaces one nonterminal with its right-hand sequence. A top-down derivation begins at S and expands the desired structure. The supplied grammar attaches the PP through VP → VP PP; it does not provide NP → NP PP. ### Step 1. Expand the sentence S ⇒ NP VP. Choose NP → Det Adj N for “the tall boy”. ### Step 2. Match the subject words Det ⇒ the, Adj ⇒ tall, N ⇒ boy. The remaining material must be generated by VP. ### Step 3. Allow a verb phrase followed by a PP Expand the outer VP using VP → VP PP. Then expand the inner VP using VP → V NP. This finite derivation selects the recursive rule once. ### Step 4. Match the verb and object V ⇒ saw. Choose NP → Det N, then Det ⇒ a and N ⇒ girl. ### Step 5. Expand the prepositional phrase PP ⇒ P NP. P ⇒ in. NP ⇒ Det N, with Det ⇒ the and N ⇒ park. ### Step 6. Check the complete leaf order The leaves read “the tall boy saw a girl in the park”, exactly the normalized input. The PP “in the park” attaches to the VP under this supplied grammar. SNPVPDetAdjNVPPPVNPPNPDetNDetNThetallboysawagirlinthepark Top-down parse tree showing the PP attached to VP. ### Complete derivation S ⇒ NP VP ⇒ Det Adj N VP ⇒ the tall boy VP ⇒ the tall boy VP PP ⇒ the tall boy V NP PP ⇒ the tall boy saw Det N PP ⇒ the tall boy saw a girl P NP ⇒ the tall boy saw a girl in Det N ⇒ the tall boy saw a girl in the park. **Implementation caution:** VP → VP PP is left-recursive. A naive recursive-descent implementation may loop; it must transform or otherwise handle the grammar. A valid finite top-down derivation, as shown above, still exists. **Yaad rakho:** Rule jo diya hai wahi use karo: yahan “in the park” VP ka part hai. --- ## TEE · Q3B · Lesk word-sense disambiguation Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/mocks/timed-reexam-2026--reexam-2026-q3b/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/mocks/timed-reexam-2026/reexam-2026-q3b TEE · actual previous paper · 100 marks · 180 minutes Question. Consider the sentence: “The bank can ensure your deposits are safe.” The word “bank” has the following two possible senses (WordNet-style glosses): Bank₁: Sloping land especially beside a body of water. Bank₂: A financial institution that accepts deposits and channels the money into lending activities. i. Apply the LESK algorithm to determine the correct sense of the word “bank” in the given sentence. Show the steps clearly. ii. Briefly discuss limitations of the LESK algorithm in handling ambiguous words in real-world contexts. [10 marks] ### Answer ### Given Context sentence and the two supplied dictionary glosses. Use a simplified Lesk method: lowercase, remove function words, exclude the target bank and compare distinct content-word overlap. Do not add synonyms that the question does not supply. ### To find Which candidate gloss has the greatest overlap with the local context. ### Formula: which one and why $$ \mathrm{score}(s)=|C\cap G_s|,\qquad \hat s=\arg\max_s\mathrm{score}(s) $$ C is the set of context content words. Gₛ is the content-word set of the gloss for sense s. The intersection contains words appearing in both; its size is the score. This is an overlap count, not a probability. ### Solution ### Step 1. Prepare the context Remove The, can, your, are and the target bank. A reasonable content set is C = {ensure, deposits, safe}. If “can” is retained under another fixed stop-list it matches neither gloss and does not change the result. ### Step 2. Prepare Bank₁’s gloss G₁ = {sloping, land, especially, beside, body, water}. There is no shared word with {ensure, deposits, safe}. ### Step 3. Calculate Bank₁’s score $$ C\cap G_1=\varnothing,\qquad \mathrm{score}(1)=0 $$ ### Step 4. Prepare Bank₂’s gloss and count overlap G₂ = {financial, institution, accepts, deposits, channels, money, lending, activities}. The shared word is deposits. ### Step 5. Calculate Bank₂’s score and select $$ C\cap G_2=\{\mathrm{deposits}\},\qquad \mathrm{score}(2)=1 $$ 1 > 0, so choose the financial-institution sense. ### Final answer **Bank₂: financial institution.** The explicit overlap “deposits” supports this choice. **Yaad rakho:** Lesk: context aur har meaning ki definition mein common words count karo; highest overlap choose karo. ### ii. Limitations - Short contexts or short glosses may produce no overlaps or ties. - Exact word matching misses related words and synonyms, such as deposit versus savings, unless an extension is defined. - Irrelevant frequent words can create misleading overlaps if preprocessing is poor. - Dictionary glosses may omit a domain-specific sense or fail to express the context’s wording. - Simplified Lesk uses limited local information and may miss discourse or real-world knowledge. Lemmatization, richer glosses and semantic similarity can improve variants, but the exact variant and tie-breaking rule should be stated. --- ## TEE · Q4A · Bag of Words and three cosine similarities Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/mocks/timed-reexam-2026--reexam-2026-q4a/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/mocks/timed-reexam-2026/reexam-2026-q4a TEE · actual previous paper · 100 marks · 180 minutes Question. Consider the following three documents: Document A: “The dog chased the cat.” Document B: “The cat chased the mouse.” Document C: “The dog ran after the cat.” Using the Bag of Words (BoW) model, represent each document as a vector after removing the stop words {The, by, was}. i. Construct the term-document matrix. ii. Compute the cosine similarity between each pair of documents (A–B, A–C, and B–C). iii. Interpret and comment on the results obtained in terms of document similarity. [10 marks] ### Answer ### Given Three documents. Lowercase and remove punctuation. Remove the specified stop words case-insensitively; “the” is removed everywhere. Keep “after” because it is not in the supplied stop list. Use raw word counts. ### To find A common-vocabulary count matrix, three cosine values and their interpretation. ### Formula: which one and why $$ \mathrm{BoW}_j(d)=C(v_j,d) $$ vⱼ is vocabulary term j. The coordinate stores its count in document d; order in the sentence is not retained. $$ \cos(\mathbf a,\mathbf b)=\frac{\sum_j a_jb_j}{\sqrt{\sum_j a_j^2}\sqrt{\sum_j b_j^2}} $$ ### Solution ### Step 1. Clean each document A: dog chased cat. B: cat chased mouse. C: dog ran after cat. ### Step 2. Fix vocabulary and matrix Vocabulary: dog, chased, cat, mouse, ran, after. | Term | A | B | C | | --- | --- | --- | --- | | dog | 1 | 0 | 1 | | chased | 1 | 1 | 0 | | cat | 1 | 1 | 1 | | mouse | 0 | 1 | 0 | | ran | 0 | 0 | 1 | | after | 0 | 0 | 1 | A = [1,1,1,0,0,0]; B = [0,1,1,1,0,0]; C = [1,0,1,0,1,1]. ### Step 3. Calculate vector lengths $$ \|A\|=\sqrt{1^2+1^2+1^2}=\sqrt3 $$ $$ \|B\|=\sqrt{1^2+1^2+1^2}=\sqrt3 $$ $$ \|C\|=\sqrt{1^2+1^2+1^2+1^2}=2 $$ ### Step 4. Calculate A–B $$ A\cdot B=(1)(0)+(1)(1)+(1)(1)+(0)(1)+(0)(0)+(0)(0)=2 $$ $$ \cos(A,B)=\frac2{\sqrt3\sqrt3}=\frac23\approx0.666667 $$ ### Step 5. Calculate A–C $$ A\cdot C=(1)(1)+(1)(0)+(1)(1)+(0)(0)+(0)(1)+(0)(1)=2 $$ $$ \cos(A,C)=\frac2{\sqrt3\cdot2}=\frac1{\sqrt3}\approx0.577350 $$ ### Step 6. Calculate B–C $$ B\cdot C=(0)(1)+(1)(0)+(1)(1)+(1)(0)+(0)(1)+(0)(1)=1 $$ $$ \cos(B,C)=\frac1{\sqrt3\cdot2}\approx0.288675 $$ ### Final answer **A–B = 0.666667; A–C = 0.577350; B–C = 0.288675.** A and B have the highest lexical overlap under this representation. B and C share only cat. BoW similarity reflects token overlap, not full semantic equivalence or word order. **Yaad rakho:** Same vocabulary order rakho. Corresponding counts multiply-add; phir dono lengths se divide. --- ## TEE · Q4B · A second HMM sequence Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/mocks/timed-reexam-2026--reexam-2026-q4b/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/mocks/timed-reexam-2026/reexam-2026-q4b TEE · actual previous paper · 100 marks · 180 minutes Question. Consider the following POS tagged corpus. Apply Hidden Markov Model (HMM) approach to: 1. Dog/Noun eats/Verb bread/Noun 2. The/Det cat/Noun sleeps/Verb 3. A/Det dog/Noun eats/Verb fish/Noun i. Construct the emission probability matrix. ii. Construct the transmission probability matrix. iii. Calculate the probability of the given sequence of events: The/Det dog/Noun eats/Verb bread/Noun. Note: Include start and end characters for all the sentences. [10 marks] ### Answer The repeated verb eats has a different emission count from the final paper; recalculate it rather than copying the previous numerical result. ### Given | Sentence | Tagged sequence | | --- | --- | | 1 | dog/Noun eats/Verb bread/Noun | | 2 | the/Det cat/Noun sleeps/Verb | | 3 | a/Det dog/Noun eats/Verb fish/Noun | Lowercase the corpus. Test sequence: the/Det dog/Noun eats/Verb bread/Noun. Add Start and End markers. ### To find Emission and transition matrices, then the probability of the supplied tagged sequence. ### Formula: which one and why $$ P(w\mid t)=\frac{C(t,w)}{C(t)} $$ $$ P(t_j\mid t_i)=\frac{C(t_i,t_j)}{\sum_kC(t_i,t_k)} $$ Emission divides the word’s tag-specific count by the total tokens with that tag. Transition divides the tag-pair count by all transitions leaving the previous tag, including End. $$ P(\mathbf w,\mathbf t)=P(t_1\mid\mathrm{Start})\prod_iP(w_i\mid t_i)\prod_{i=2}^{n}P(t_i\mid t_{i-1})P(\mathrm{End}\mid t_n) $$ ### Solution ### Step 1. Count tag totals and word frequencies Det: the=1, a=1, total2. Noun: dog=2, bread=1, cat=1, fish=1, total5. Verb: eats=2, sleeps=1, total3. ### Step 2. Construct emission matrix | Tag / word | a | bread | cat | dog | eats | fish | sleeps | the | | --- | --- | --- | --- | --- | --- | --- | --- | --- | | Det | 1/2 | 0 | 0 | 0 | 0 | 0 | 0 | 1/2 | | Noun | 0 | 1/5 | 1/5 | 2/5 | 0 | 1/5 | 0 | 0 | | Verb | 0 | 0 | 0 | 0 | 2/3 | 0 | 1/3 | 0 | ### Step 3. Count transitions from the three chains Start → Noun → Verb → Noun → End Start → Det → Noun → Verb → End Start → Det → Noun → Verb → Noun → End | From | Det count | Noun count | Verb count | End count | Total | | --- | --- | --- | --- | --- | --- | | Start | 2 | 1 | 0 | 0 | 3 | | Det | 0 | 2 | 0 | 0 | 2 | | Noun | 0 | 0 | 3 | 2 | 5 | | Verb | 0 | 2 | 0 | 1 | 3 | ### Step 4. Divide outgoing counts | From / to | Det | Noun | Verb | End | | --- | --- | --- | --- | --- | | Start | 2/3 | 1/3 | 0 | 0 | | Det | 0 | 1 | 0 | 0 | | Noun | 0 | 0 | 3/5 | 2/5 | | Verb | 0 | 2/3 | 0 | 1/3 | All nonterminal rows sum to one. No smoothing is requested. ### Step 5. Write every sequence factor | Event | Probability | | --- | --- | | Start → Det | 2/3 | | Det emits the | 1/2 | | Det → Noun | 1 | | Noun emits dog | 2/5 | | Noun → Verb | 3/5 | | Verb emits eats | 2/3 | | Verb → Noun | 2/3 | | Noun emits bread | 1/5 | | Noun → End | 2/5 | ### Step 6. Multiply all factors $$ P=\frac23\times\frac12\times1\times\frac25\times\frac35\times\frac23\times\frac23\times\frac15\times\frac25 $$ Start with subtotal 1. At each row, multiply the previous subtotal by exactly one next factor. Simplify the fraction after multiplication, but do not round it. | Factor being included | Previous subtotal × next factor = new subtotal | | --- | --- | | 1. Start → Det | $$ 1\times\frac{2}{3}=\frac{2}{3} $$ | | 2. Det emits the | $$ \frac{2}{3}\times\frac{1}{2}=\frac{1}{3} $$ | | 3. Det → Noun | $$ \frac{1}{3}\times1=\frac{1}{3} $$ | | 4. Noun emits dog | $$ \frac{1}{3}\times\frac{2}{5}=\frac{2}{15} $$ | | 5. Noun → Verb | $$ \frac{2}{15}\times\frac{3}{5}=\frac{2}{25} $$ | | 6. Verb emits eats | $$ \frac{2}{25}\times\frac{2}{3}=\frac{4}{75} $$ | | 7. Verb → Noun | $$ \frac{4}{75}\times\frac{2}{3}=\frac{8}{225} $$ | | 8. Noun emits bread | $$ \frac{8}{225}\times\frac{1}{5}=\frac{8}{1125} $$ | | 9. Noun → End | $$ \frac{8}{1125}\times\frac{2}{5}=\frac{16}{5625} $$ | The last subtotal is the complete product. Only now convert the fraction to a decimal. $$ P=\frac{16}{5625}\approx0.0028444444 $$ ### Final answer **P = 16/5625 ≈ 0.0028444444** for these words and specified tags jointly. The two occurrences of eats give P(eats | Verb) = 2/3. **Yaad rakho:** Same tag structure ho sakta hai, lekin word emission counts dobara count karna. **Boundary reminder:** Include P(Det | Start) = 2/3 and P(End | Noun) = 2/5. The handwritten key’s final line instead inserts the overall Det fraction 2/10 and omits End; the complete boundary-aware probability requested here is 0.0028444444. --- ## TEE · Q5A · Why TF–IDF is useful Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/mocks/timed-reexam-2026--reexam-2026-q5a/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/mocks/timed-reexam-2026/reexam-2026-q5a TEE · actual previous paper · 100 marks · 180 minutes Question. Justify with an appropriate example the importance of TF-IDF technique in the document representation and information retrieval task. [10 marks] ### Answer **TF–IDF** represents each document using weights that combine within-document term frequency with how unusual a term is across the collection. It supports information retrieval by giving larger weight to terms that are frequent in a document but not present in every document. ### Given Use three short documents: D₁ = “solar energy solar”; D₂ = “wind energy”; D₃ = “energy policy”. Use raw-count TF and natural-log unsmoothed IDF. N = 3. ### To find A TF–IDF representation illustrating why distinctive terms help retrieve relevant documents. ### Formula: which one and why $$ \mathrm{TFIDF}(t,d)=C(t,d)\ln\frac{N}{\mathrm{df}(t)} $$ C counts occurrences within a document; df counts documents containing the term. The logarithm reduces the scale of very large rarity ratios. ### Solution ### Step 1. Count term occurrences | Term | D₁ | D₂ | D₃ | df | | --- | --- | --- | --- | --- | | solar | 2 | 0 | 0 | 1 | | energy | 1 | 1 | 1 | 3 | | wind | 0 | 1 | 0 | 1 | | policy | 0 | 0 | 1 | 1 | ### Step 2. Calculate IDF $$ \mathrm{IDF}(solar)=\mathrm{IDF}(wind)=\mathrm{IDF}(policy)=\ln3\approx1.098612 $$ $$ \mathrm{IDF}(energy)=\ln(3/3)=0 $$ ### Step 3. Multiply each count by its IDF | Term | D₁ weight | D₂ weight | D₃ weight | | --- | --- | --- | --- | | solar | 2 × 1.098612 = 2.197224 | 0 | 0 | | energy | 0 | 0 | 0 | | wind | 0 | 1.098612 | 0 | | policy | 0 | 0 | 1.098612 | ### Step 4. Connect the representation to retrieval A query containing “solar” matches D₁’s high solar coordinate. A query containing “wind” matches D₂. “energy” alone cannot distinguish these documents under this unsmoothed collection because it occurs in all three. A retrieval system can compare query and document vectors, commonly by cosine similarity. ### Final answer TF–IDF preserves distinctive lexical evidence and reduces dominance by collection-wide frequent terms. D₁ is most relevant to a solar query under these features. **Yaad rakho:** Document mein frequent + collection mein rare = high TF–IDF. ### Strengths and limits The representation is sparse, interpretable and useful as a fast baseline. However, basic TF–IDF ignores word order and does not recognise synonyms or multiple word senses automatically. Its weights depend on the collection and on the chosen TF/IDF convention. Fit IDF on training or indexed documents, not on a hidden test set. --- ## TEE · Q5B · Sentence-window vectors and cosine Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/mocks/timed-reexam-2026--reexam-2026-q5b/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/mocks/timed-reexam-2026/reexam-2026-q5b TEE · actual previous paper · 100 marks · 180 minutes Question. Consider the paragraph: The farmer planted seeds in the soil. The rain watered the soil. The sun helped the plants grow. The farmer harvested the plants. The soil became rich and fertile. Target words: farmer, soil, sun, plant. Context words: plant, water, grow, harvest. i. Construct a co-occurrence matrix for the target and context words. Take the window size as one whole sentence. ii. Evaluate the similarity between the vectors of the target words “farmer-soil”, “farmer-sun”, and “farmer-plant” using the cosine similarity. iii. Plot the target word vectors by considering only two dimensions: “planted” and “harvested”. [10 marks] ### Answer ### Given Five sentence windows. For alignment with the supplied matrix, target plant means the noun plants, occurring in sentences 3 and 4. Context plant covers planted and plants; watered → water; harvested → harvest. Count a target–context pair once per sentence in which both are present, including the plant–plant sentence-presence diagonal. ### To find The target–context matrix, three cosine similarities and a two-coordinate plot. ### Formula: which one and why $$ M_{ij}=\sum_s\mathbf1[\text{target}_i\text{ and context}_j\text{ present in }s] $$ $$ \cos(\mathbf a,\mathbf b)=\frac{\sum_j a_jb_j}{\sqrt{\sum_j a_j^2}\sqrt{\sum_j b_j^2}} $$ This is a sentence-presence matrix, not a position-based sliding window that excludes a token from its own context. Stating the convention is necessary because those alternatives give a different diagonal. ### Solution ### Step 1. Normalize and identify each sentence’s contribution | Sentence | Targets present | Context lemmas present | | --- | --- | --- | | 1 | farmer, soil | plant | | 2 | soil | water | | 3 | sun, plant | plant, grow | | 4 | farmer, plant | plant, harvest | | 5 | soil | none | ### Step 2. Fill all matrix rows | Target | plant | water | grow | harvest | | --- | --- | --- | --- | --- | | farmer | 2 | 0 | 0 | 1 | | soil | 1 | 1 | 0 | 0 | | sun | 1 | 0 | 1 | 0 | | plant | 2 | 0 | 1 | 1 | Farmer meets context plant in sentences 1 and 4. Soil meets plant in sentence 1 and water in sentence 2. Sun meets plant and grow in sentence 3. The noun plant occurs in sentences 3 and 4, producing its last row. ### Step 3. Compute farmer–soil $$ \mathbf f\cdot\mathbf s=(2)(1)+(0)(1)+(0)(0)+(1)(0)=2 $$ $$ \|\mathbf f\|=\sqrt{2^2+0^2+0^2+1^2}=\sqrt5,\quad\|\mathbf s\|=\sqrt2 $$ $$ \cos(\mathbf f,\mathbf s)=\frac2{\sqrt{10}}\approx0.632456 $$ ### Step 4. Compute farmer–sun $$ \mathbf f\cdot\mathbf u=(2)(1)+(0)(0)+(0)(1)+(1)(0)=2 $$ $$ \|\mathbf u\|=\sqrt2,\quad\cos(\mathbf f,\mathbf u)=\frac2{\sqrt{10}}\approx0.632456 $$ ### Step 5. Compute farmer–plant $$ \mathbf f\cdot\mathbf p=(2)(2)+(0)(0)+(0)(1)+(1)(1)=5 $$ $$ \|\mathbf p\|=\sqrt{2^2+0^2+1^2+1^2}=\sqrt6 $$ $$ \cos(\mathbf f,\mathbf p)=\frac5{\sqrt{30}}\approx0.912871 $$ ### Step 6. Plot the requested two coordinates Interpret the axis names planted and harvested as their normalized context features plant and harvest. Farmer=(2,1), soil=(1,0), sun=(1,0), plant=(2,1). Farmer and plant overlap; soil and sun overlap. 001122farmer, plantsoil, sunplant (planted)harvest (harvested) The plot uses only the requested two coordinates; cosine uses all context columns. ### Final answer Using the stated noun-target convention: **farmer–soil = 0.632456, farmer–sun = 0.632456, farmer–plant = 0.912871.** **Yaad rakho:** Plant noun aur plant verb ko same maana ya separate? Pehle convention likho, phir count karo. ### Two details that prevent an incorrect answer **Arithmetic:** 5/√30 = 0.912871; the source key’s printed approximation 0.9444 does not equal its own fraction. Keep the exact fraction until the last step. **Normalization:** If the question instead requires all noun and verb forms of plant to be merged as targets, sentence 1 also contains the plant target. Its row becomes [3,0,1,1], and farmer–plant becomes 7/√55 ≈ 0.943880. The paper leaves this choice unstated; the main worked solution explicitly uses the noun-target interpretation that reproduces its supplied matrix. --- ## TEE · Q6A · Pronoun approaches and lexical relations Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/mocks/timed-reexam-2026--reexam-2026-q6a/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/mocks/timed-reexam-2026/reexam-2026-q6a TEE · actual previous paper · 100 marks · 180 minutes Question. i. Briefly describe any two approaches to pronoun resolution and mention one advantage of each. ii. Describe the difference between Homonymy and Synonymy with suitable examples. Discuss how these two relations differently affect natural language processing applications? [10 marks] ### Answer ### i. Two approaches to pronoun resolution | Approach | How it works | Advantage | | --- | --- | --- | | Rule-based / syntactic | Uses parse structure, agreement, recency and linguistic constraints. Hobbs’ algorithm searches parse trees for eligible antecedents. | Interpretable: a proposed antecedent can be traced to explicit rules. | | Statistical or neural | Learns candidate–pronoun compatibility or whole-document coreference patterns from labelled examples, using context and linguistic features or contextual embeddings. | Can learn interactions and contextual cues that are difficult to enumerate as hand-written rules. | For “Asha dropped the glass because it was slippery,” a system should evaluate candidate referents and context rather than merely choose the most recent noun. Rules can constrain number and syntax; learned models can use compatibility patterns, though neither approach is guaranteed to resolve every ambiguity. ### ii. Homonymy and synonymy | Relation | Definition | Example | | --- | --- | --- | | Homonymy | Same spelling or sound, unrelated meanings | bank as financial institution versus bank as river edge | | Synonymy | Different words with the same or closely related meanings in a context | car and automobile; big and large | Homonymy creates an ambiguity problem: a search for bank may retrieve finance or rivers. WSD uses context to choose the intended sense. Synonymy creates a vocabulary-matching problem: a document saying automobile may be relevant to a query saying car. Query expansion or semantic representations can improve recall by connecting related terms. The remedies differ. Do not merge unrelated homonym senses simply because their spelling matches. Do not insist on exact surface-form matching when different words express the same concept. Synonyms are also context-sensitive: “big mistake” and “large mistake” are not always equally natural. **Yaad rakho:** Homonym: same word-form, different meaning. Synonym: different words, similar meaning. --- ## TEE · Q6B · Hobbs: Ramesh and Shyam Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/mocks/timed-reexam-2026--reexam-2026-q6b/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/mocks/timed-reexam-2026/reexam-2026-q6b TEE · actual previous paper · 100 marks · 180 minutes Question. i. Explain Hobbs’ Algorithm for pronoun resolution. State its main steps and limitations. ii. Apply Hobbs’ Algorithm to the sentence: “When Ramesh gave Shyam a book, he smiled.” Identify the antecedent of “he” by showing traversal steps. [10 marks] ### Answer **Hobbs’ algorithm** is a syntax-based pronoun-resolution procedure that searches constituency parse trees in a specified order. It proposes noun phrases as antecedents and filters incompatible candidates, for example by number and gender agreement. **Read the tree labels first:** NP means noun phrase; S means sentence/clause; VP means verb phrase. X is the current NP or S search node, and p is the upward path used to reach it. Breadth-first means shallower nodes before deeper nodes; left-to-right fixes the order at the same depth. An N-bar (N′) node is the noun-centred part of an NP in a parse using that label. ### Complete ordered procedure 1. **Locate the pronoun.** Start at the NP immediately dominating the pronoun: the smallest NP containing it. 2. **Make the first climb.** Move upward to the first NP or S above the starting NP. Call it X and record the path p. 3. **Search initially to the left.** Traverse branches below X to the left of p, breadth-first and left-to-right. Propose an NP only if another NP or S lies between that candidate and X. Check agreement and other applicable constraints for each proposed candidate; accept the first eligible compatible one. 4. **Check the highest sentence.** If X is the highest S and no candidate has been accepted, search previous sentence trees, most recent first. In each, traverse breadth-first and left-to-right and propose NPs. Accept the first eligible compatible candidate; if the available context contains none, report no resolved antecedent. 5. **Otherwise climb again.** Move from X to the next NP or S above it. Make that node the new X and record the new upward path p. 6. **Test a higher NP.** If the new X is an NP, propose X only when p did not pass through its nominal N-bar child. Apply the same compatibility checks. Use the labels in the supplied parse; do not invent a missing N-bar branch. 7. **Search left again.** Traverse the new X’s branches to the left of p, breadth-first and left-to-right, proposing NPs. The extra intervening-NP/S requirement in Step 3 belongs to the initial search. 8. **Search permitted right branches.** If X is an S, traverse branches to the right of p, breadth-first and left-to-right, but do not descend below any NP or S encountered. Propose the encountered eligible NPs and test their compatibility. 9. **Repeat or finish.** Return to Step 4 until a compatible candidate is accepted or the available sentence trees are exhausted. Record the candidate order and why each candidate was accepted or rejected. **Why the initial restriction matters:** in “John saw him”, the subject NP John is directly under the clause S, with no additional NP or S between them. Step 3 therefore does not propose John as the ordinary local antecedent of him. A reflexive such as himself is a different grammatical case. The worked question below shows the specific parse and traversal; selecting an antecedent is not simply choosing the nearest name. ### Given The pronoun he is the subject of the main clause. The preceding subordinate clause is “When Ramesh gave Shyam a book”. Assume the temporal clause attaches to the whole main sentence. ### To find The first suitable antecedent in the left-to-right breadth-first search of the preceding clause. ``` S (main) ← first X ├── SBAR: When │ └── S (embedded) ← intervening S │ ├── NP: Ramesh ← first candidate │ └── VP │ ├── V: gave │ ├── NP: Shyam │ └── NP: a book ├── NP: he ← start └── VP: smiled ``` ### Step 1. Locate the pronoun NP Start at NP “he” and move upward to its S ancestor. Record the path from he. ### Step 2. Search branches before the pronoun The initial when-clause lies to the left of the path. Apply Step 3: its embedded S lies between each of its noun phrases and the main S, so these NPs satisfy the initial intervening-NP/S requirement. Traverse left-to-right and breadth-first. ### Step 3. Propose noun phrases The subject NP Ramesh is encountered before the deeper object NP Shyam and object NP a book. Ramesh agrees with the assumed singular-male pronoun. ### Step 4. Accept the first compatible antecedent Select Ramesh. Shyam is also a semantically possible referent in ordinary discourse, but the algorithm’s stated syntactic preference selects Ramesh first. ### Limitations - Requires a suitable and accurate constituency parse. - Agreement information may be unknown or insufficient when several people match. - Ignores much of the event semantics, discourse focus and world knowledge. - An ambiguous parse or different attachment can change candidate order. **Final answer:** **Ramesh**, under the shown parse and ordinary agreement assumptions. A syntactic preference is not a guarantee about the author’s intended meaning. **Yaad rakho:** Left branches ko breadth-first dekho; pehla eligible matching NP choose karo. --- ## TEE · Q7A · Information retrieval Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/mocks/timed-reexam-2026--reexam-2026-q7a/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/mocks/timed-reexam-2026/reexam-2026-q7a TEE · actual previous paper · 100 marks · 180 minutes Question. i. Explain the concept of Information Retrieval, highlighting its role in Recommender Systems and Search Engines. ii. Discuss the various steps involved in the Information Retrieval process. [10 marks] ### Answer ### i. Concept and roles **Information Retrieval (IR)** finds and ranks documents or other information objects relevant to a user’s information need. A query is the user’s expression of that need. IR typically returns a ranked set of items; information extraction instead identifies structured facts or fields inside those items. | Application | Role of IR | Example | | --- | --- | --- | | Search engine | Matches a submitted query against a large indexed collection and ranks candidate pages | A search for “symptoms of dehydration” returns relevant health pages | | Recommender system | Retrieves candidate items using a user profile, recent activity or a content representation; a later model may personalize ranking | Retrieve articles similar to topics the user reads | Retrieval is a component of many recommenders; recommendation also involves preferences, feedback, novelty and business constraints. A recommender need not wait for an explicit textual query. ### ii. Retrieval process Collect → Preprocess → Index → Process query → Retrieve candidates → Rank → Evaluate 1. **Collect and define documents.** Crawl or ingest documents, record metadata and remove unwanted duplicates. 2. **Preprocess.** Tokenize and normalize using rules suitable for both documents and queries. Preserve task-relevant information. 3. **Build representations and an index.** An inverted index maps each term to the documents containing it; a dense retrieval system stores vector representations. 4. **Process the query.** Apply compatible tokenization, spelling handling or expansion, then obtain query features. 5. **Retrieve candidates.** Use the index to identify potentially relevant documents efficiently. 6. **Score and rank.** A lexical method may use TF–IDF/BM25-style evidence; a semantic method compares representations. Rank scores estimate relevance rather than proving correctness. 7. **Present results.** Return titles, snippets and links, possibly with filtering or diversification. 8. **Evaluate and update.** Use relevance judgments and measures such as precision, recall or ranking metrics. Refresh the index as documents change. Example: for “solar energy storage”, an inverted index retrieves documents containing those terms; a ranking function rewards useful matches. A semantic system may also retrieve relevant material using related terms such as batteries. **Yaad rakho:** IR mein relevant document dhoondho aur rank karo; extraction mein document ke andar facts nikalo. --- ## TEE · Q7B · Gutenberg corpus and normalization Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/mocks/timed-reexam-2026--reexam-2026-q7b/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/mocks/timed-reexam-2026/reexam-2026-q7b TEE · actual previous paper · 100 marks · 180 minutes Question. i. Discuss how the Gutenberg Corpus can be utilized for various text analysis tasks in Natural Language Processing? ii. Explain the importance of text normalization in Natural Language Processing (NLP). Illustrate your answer with a suitable example of an application where text normalization plays a crucial role. [10 marks] ### Answer ### i. Using the Gutenberg corpus The Gutenberg corpus provides literary texts, commonly used through tools such as NLTK’s Gutenberg collection. It supplies naturally occurring sentences and longer documents for exploring language patterns. The available subset and licensing should be checked before reuse in a new distribution. | Task | Use of the corpus | | --- | --- | | Frequency and vocabulary study | Count tokens and word types, inspect common words and compare lexical diversity across texts | | Concordances and collocations | View a word in context and discover words that occur near each other | | N-gram language modeling | Estimate unigram or bigram counts and compare sequence probabilities | | Stylistic or authorship analysis | Compare sentence lengths, vocabulary and function-word patterns across authors | | Text preprocessing experiments | Study the effect of tokenization, case normalization and lemmatization on counts | A sound workflow selects files, inspects encoding and boundaries, tokenizes consistently, computes the relevant statistics and interprets them against corpus composition. Literary text is not automatically representative of chat, medical records or present-day search queries; domain mismatch matters. ### ii. Text normalization **Normalization** converts selected surface variants into a consistent form so that equivalent expressions do not unnecessarily produce different features. Possible steps include case folding, whitespace cleanup, Unicode normalization, contraction expansion and lemmatization. Choose these according to the task. ### Example: product search 1. Raw query: “ RUNNING shoes ”. Product description: “Comfortable run shoes”. 2. Trim extra whitespace and apply consistent case: “running shoes”. 3. If the application chooses lemmatization, running → run. The query and the indexed description now share the intended run and shoes features. 4. Apply the same normalization to stored documents and future queries, otherwise matching remains inconsistent. Normalization reduces avoidable vocabulary variation and can improve recall. It must preserve distinctions important to the task: lowercasing US to us may lose meaning, and removing “not” can reverse sentiment. Aggressive stemming or synonym replacement can create incorrect matches. **Yaad rakho:** Same meaning ke surface variants ko consistent banao; useful meaning delete mat karo. --- ## TEE · All seven units: final revision Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/revision/tee-revision--all-units/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/revision/tee-revision/all-units TEE · revision, not a replacement for full lessons TEE is 100 marks in 3 hours, scaled to 50 in the course total. Q1 is compulsory; choose any four of Q2–Q7. Each full question carries 20 marks. The two supplied 2025–26 papers follow this rule. Do not use the short M2 route as complete TEE preparation. | Unit | Must be able to explain | Must be able to work | | --- | --- | --- | | 1 | NLP levels, building blocks, NLU/NLG, approaches, pipeline and challenges | Classify ambiguity and identify linguistic units in examples. | | 2 | Tokenization, case, stopwords, stemming/lemmatization, encoding, BoW/TF-IDF | Construct vectors; compute TF-IDF, cosine/Jaccard/Euclidean/Manhattan/Minkowski. | | 3 | Corpora, annotation, dictionaries, regex, N-gram models and smoothing | List n-grams; count probabilities; smooth; compute sentence probability/perplexity. | | 4 | POS approaches, HMM, chunking/chinking, CFG parsing, NER | Build emission/transition matrices; score paths; draw a tree; trace parser operations. | | 5 | Lexical relations, WordNet, WSD, distributional/vector semantics, Word2Vec, GloVe | Lesk and NB; co-occurrence vectors; similarity; CBOW/Skip-gram examples. | | 6 | Pragmatics/discourse, reference, coreference, pronouns, Hobbs and other approaches | Identify chains; show a tree traversal and state parse assumptions. | | 7 | Classification, sentiment, spelling, translation, QA, IE, IR | Design an application pipeline; calculate relevant model/evaluation examples. | ### TEE revision sequence First revise the formula sheet and one full example per numerical family. Next read each unit’s definitions and comparisons. Then work one complete previous paper using its original choice rule. Finally revisit only the steps you got wrong, and read the other paper’s new question forms. Keep examples and diagrams beside definitions when revising; a term you cannot illustrate is not yet ready to explain. For a 10-mark numerical, show the count table or matrix and the intermediate probabilities. For a 10-mark theory answer, organise definition, mechanism/steps, diagram/example, advantages and limitations around the actual question. Do not add unrelated paragraphs just to fill space. --- ## TEE · Numerical answer framework Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/revision/tee-revision--numerical-method/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/revision/tee-revision/numerical-method TEE · revision, not a replacement for full lessons | Question clue | Use this method | First thing to write | | --- | --- | --- | | “next word”, “sentence probability” | N-gram model | List the needed word transitions. | | “unseen pair”, “Laplace” | Add-one smoothing | State V and count each history. | | “perplexity” | Reciprocal Nth root | State P(test sequence) and N. | | “transition/emission matrix” | HMM count estimation | Separate tag-to-tag counts from tag-to-word counts. | | “given tag sequence” | HMM joint path | Initial × emission × transition × emission … | | “best tag sequence” | Viterbi or enumerate tiny paths | Initialise one score per tag; retain predecessor maxima. | | “gloss overlap” | Simplified Lesk | Preprocess context and glosses consistently. | | “classify word sense with counts” | Naive Bayes WSD | Priors from examples, likelihoods from tokens. | | “parse tree” | Given CFG + chosen parser | Show grammar expansions or stack reductions. | ### Errors that change the answer 1. Dividing a conditional count by the whole corpus instead of its history count. 2. Adding 1 to the numerator without adding |V| to the denominator. 3. Forgetting the end transition. 4. Counting repeated words as new vocabulary types. 5. Using P(tag|word) in place of P(word|tag). 6. Calling an unnormalised score a posterior. 7. Silently mixing case, lemmatisation or boundary conventions. ### Short theory answer format For 2 marks: give the exact definition and one specific example or distinction. For 3 marks: add the mechanism or two ordered steps; include a small diagram/table where it carries information. The actual allocation is examiner-dependent, so these are writing guides, not an official marking scheme. --- ## TEE · Latest and older papers: exact question map Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/about/pyq-analysis--all-papers/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/about/pyq-analysis/all-papers ### How to use this question map Start with the newest final and re-exam, then use older papers to practise variations. The table links each original lettered part to its complete worked answer. **Repeated questions are history, not a prediction.** A syllabus topic stays required even when no recovered paper asks it. **Marks:** 100 marks are attempted: compulsory Q1 (20), then any four other 20-mark questions. Most recovered papers offer140 marks; NLP December2023 offers120 because the complete original ends at Q6. A mixed question retains one mark allocation; its units are not counted as separate questions. Blank individual allocations remain unknown. ### 2026-02-09 · re-exam · R26 140 marks offered; 100 attempted. | Question [marks] | Current unit and topic | Question form | Answer / scope | | --- | --- | --- | --- | | **Q1a [5]** | Unit 1 · NLP pipeline | theory | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/reexam-2026--reexam-2026-q1a/) | | **Q1b [5]** | Unit 1, 4 · Content and function words | mixed | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/reexam-2026--reexam-2026-q1b/) | | **Q1c [5]** | Unit 5 · Word2Vec and GloVe | theory | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/reexam-2026--reexam-2026-q1c/) | | **Q1d [5]** | Unit 6 · Cataphora, anaphora and coreference | theory | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/reexam-2026--reexam-2026-q1d/) | | **Q2a [10]** | Unit 1 · Five NLP tasks; syntax and semantics | theory | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/reexam-2026--reexam-2026-q2a/) | | **Q2b [10]** | Unit 3 · Add-one bigram sentence probability | numerical | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/reexam-2026--reexam-2026-q2b/) | | **Q3a [10]** | Unit 4 · Top-down CFG parsing | case | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/reexam-2026--reexam-2026-q3a/) | | **Q3b [10]** | Unit 5 · Lesk with supplied glosses | numerical | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/reexam-2026--reexam-2026-q3b/) | | **Q4a [10]** | Unit 2 · Bag of words and three cosine similarities | numerical | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/reexam-2026--reexam-2026-q4a/) | | **Q4b [10]** | Unit 4 · HMM estimation and given tag-path joint probability | numerical | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/reexam-2026--reexam-2026-q4b/) | | **Q5a [10]** | Unit 2 · Explain and illustrate TF-IDF | mixed | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/reexam-2026--reexam-2026-q5a/) | | **Q5b [10]** | Unit 5 · Co-occurrence, cosine and normalized plot | numerical | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/reexam-2026--reexam-2026-q5b/) · Target/context normalization and sentence-presence convention stated; alternative target lemmatization has a separately calculated result. | | **Q6a [10]** | Unit 5, 6 · Pronoun-resolution methods; homonymy and synonymy | theory | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/reexam-2026--reexam-2026-q6a/) | | **Q6b [10]** | Unit 6 · Hobbs antecedent traversal | case | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/reexam-2026--reexam-2026-q6b/) · Hobbs requires a parse and compatibility assumptions. The assumed parse and traversal order are disclosed. | | **Q7a [10]** | Unit 7 · Information-retrieval stages | theory | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/reexam-2026--reexam-2026-q7a/) | | **Q7b [10]** | Unit 2, 3 · Gutenberg and text normalization | theory | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/reexam-2026--reexam-2026-q7b/) | **Original offered marks by mapped topic, grouped without double counting:** Unit 1: 2 parts, 15 known marks; Unit 1, 4: 1 parts, 5 known marks; Unit 5: 3 parts, 25 known marks; Unit 6: 2 parts, 15 known marks; Unit 3: 1 parts, 10 known marks; Unit 4: 2 parts, 20 known marks; Unit 2: 2 parts, 20 known marks; Unit 5, 6: 1 parts, 10 known marks; Unit 7: 1 parts, 10 known marks; Unit 2, 3: 1 parts, 10 known marks. Joint-unit buckets remain joint; no invented split is assigned. These original-paper totals include any legacy tasks explicitly marked above; they are not a current-syllabus marks blueprint. ### 2025-12-02 · final · F25 140 marks offered; 100 attempted. | Question [marks] | Current unit and topic | Question form | Answer / scope | | --- | --- | --- | --- | | **Q1a [5]** | Unit 1 · NLU versus NLG | theory | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/final-2025--final-2025-q1a/) | | **Q1b [5]** | Unit 4 · Chunking, chinking and POS | theory | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/final-2025--final-2025-q1b/) | | **Q1c [5]** | Unit 5 · Hyponymy/hypernymy and WSD | theory | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/final-2025--final-2025-q1c/) | | **Q1d [5]** | Unit 6 · Pragmatic and discourse analysis | theory | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/final-2025--final-2025-q1d/) | | **Q2a [10]** | Unit 1, 4 · Morpheme/lexeme and NER applications | theory | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/final-2025--final-2025-q2a/) | | **Q2b [10]** | Unit 3 · Add-one bigram sentence probability | numerical | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/final-2025--final-2025-q2b/) | | **Q3a [10]** | Unit 1, 4 · Ambiguities and bottom-up CFG parse | mixed | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/final-2025--final-2025-q3a/) | | **Q3b [10]** | Unit 5 · Co-occurrence, cosine and vector plot | numerical | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/final-2025--final-2025-q3b/) | | **Q4a [10]** | Unit 2 · Three preprocessing techniques | theory | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/final-2025--final-2025-q4a/) | | **Q4b [10]** | Unit 4 · HMM estimation and given tag-path joint probability | numerical | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/final-2025--final-2025-q4b/) | | **Q5a [10]** | Unit 2 · TF-IDF matrix | numerical | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/final-2025--final-2025-q5a/) | | **Q5b [10]** | Unit 5 · Naive Bayes word-sense disambiguation | numerical | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/final-2025--final-2025-q5b/) · Fixed vocabulary/OOV treatment is stated; source-key OOV smoothing differs. Both conventions and their consequence are explained. | | **Q6a [10]** | Unit 5, 6 · Coreference/pronouns; CBOW and Skip-gram | theory | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/final-2025--final-2025-q6a/) | | **Q6b [10]** | Unit 6 · Hobbs antecedent traversal | case | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/final-2025--final-2025-q6b/) · Hobbs requires a parse and compatibility assumptions. The assumed parse and traversal order are disclosed. | | **Q7a [10]** | Unit 7 · Document-classification pipeline | theory | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/final-2025--final-2025-q7a/) | | **Q7b [10]** | Unit 2, 3 · Corpus structures and Jaccard example | mixed | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/final-2025--final-2025-q7b/) | **Original offered marks by mapped topic, grouped without double counting:** Unit 1: 1 parts, 5 known marks; Unit 4: 2 parts, 15 known marks; Unit 5: 3 parts, 25 known marks; Unit 6: 2 parts, 15 known marks; Unit 1, 4: 2 parts, 20 known marks; Unit 3: 1 parts, 10 known marks; Unit 2: 2 parts, 20 known marks; Unit 5, 6: 1 parts, 10 known marks; Unit 7: 1 parts, 10 known marks; Unit 2, 3: 1 parts, 10 known marks. Joint-unit buckets remain joint; no invented split is assigned. These original-paper totals include any legacy tasks explicitly marked above; they are not a current-syllabus marks blueprint. ### 2025-02-21 · re-exam · R25 140 marks offered; 100 attempted. Q1 total20 is printed, but the four individual part allocations are blank. Do not infer5marks as if printed. Sixth PDF page is blank. | Question [marks] | Current unit and topic | Question form | Answer / scope | | --- | --- | --- | --- | | **Q1a [not printed individually]** | Unit 2 · BoW and one-hot encoding | theory | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r25--q1a/) | | **Q1b [not printed individually]** | Unit 5 · Lesk WSD | theory | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r25--q1b/) | | **Q1c [not printed individually]** | Unit 4 · Syntax parsing approaches | theory | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r25--q1c/) | | **Q1d [not printed individually]** | Unit 1, 7 · NLU versus NLG and applications | theory | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r25--q1d/) | | **Q2a [10]** | Unit 2 · TF–IDF cosine after a fixed stoplist | numerical | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r25--q2a/) | | **Q2b [10]** | Unit 7 · Weather Naive Bayes | numerical | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r25--q2b/) | | **Q3a [5]** | Unit 6 · Coreference and anaphora | theory | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r25--q3a/) | | **Q3b [5]** | Unit 6 · Cohesion, coherence and discourse markers | theory | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r25--q3b/) | | **Q3c [10]** | Unit 7 · Question answering system | theory | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r25--q3c/) | | **Q4a [5]** | Unit 1 · NLP pipeline | theory | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r25--q4a/) | | **Q4b [5]** | Unit 1 · Morphology and syntax | theory | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r25--q4b/) | | **Q4c [10]** | Unit 3 · Most probable bigram continuations | numerical | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r25--q4c/) | | **Q5a [6]** | Unit 2 · Preprocess the NLP passage | case | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r25--q5a/) | | **Q5b [4]** | Unit 2 · Stemming and lemmatization tools | theory | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r25--q5b/) | | **Q5c [10]** | Unit 6 · Hobbs on John and Mary | case | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r25--q5c/) | | **Q6a [10]** | Unit 3 · Corpora and acquisition | theory | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r25--q6a/) | | **Q6b [10]** | Unit 4 · HMM components and estimates | mixed | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r25--q6b/) | | **Q7a [5]** | Unit 4 · Word classes, chunking and chinking | theory | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r25--q7a/) | | **Q7b [5]** | Unit 4 · Named entity recognition | theory | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r25--q7b/) | | **Q7c [10]** | Unit 5 · Co-occurrence with inconsistent requested comparisons | numerical | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r25--q7c/) · Source defect: follow-up names belong to another corpus. Exact supported matrix is complete; unsupported parts are explicitly unresolved from the original data. | **Original offered marks by mapped topic, grouped without double counting:** Unit 2: 4 parts, 20 known marks + 1 parts with no individual mark printed; Unit 5: 2 parts, 10 known marks + 1 parts with no individual mark printed; Unit 4: 4 parts, 20 known marks + 1 parts with no individual mark printed; Unit 1, 7: 1 parts, 0 known marks + 1 parts with no individual mark printed; Unit 7: 2 parts, 20 known marks; Unit 6: 3 parts, 20 known marks; Unit 1: 2 parts, 10 known marks; Unit 3: 2 parts, 20 known marks. Joint-unit buckets remain joint; no invented split is assigned. These original-paper totals include any legacy tasks explicitly marked above; they are not a current-syllabus marks blueprint. ### 2024-12-03 · final · F24 140 marks offered; 100 attempted. Four Q1 part allocations are blank; their group totals20. Other printed lettered allocations determine each 20-mark main question. | Question [marks] | Current unit and topic | Question form | Answer / scope | | --- | --- | --- | --- | | **Q1a [not printed individually]** | Unit 2, 7 · TF–IDF in retrieval | theory | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f24--q1a/) | | **Q1b [not printed individually]** | Unit 4 · NER and approaches | theory | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f24--q1b/) | | **Q1c [not printed individually]** | Unit 5 · Skip-gram and CBOW | theory | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f24--q1c/) | | **Q1d [not printed individually]** | Unit 6 · Pragmatics and discourse | theory | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f24--q1d/) | | **Q2a [6]** | Unit 2, 4 · Six preprocessing operations | theory | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f24--q2a/) | | **Q2b [4]** | Unit 2, 5 · Lexical similarity and cosine | mixed | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f24--q2b/) | | **Q2c [10]** | Unit 7 · Animal categorical Naive Bayes | numerical | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f24--q2c/) | | **Q3a [10]** | Unit 6 · Hobbs and ambiguous professor–student reference | case | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f24--q3a/) · Pronoun referent is ambiguous without extra gender/discourse information; structural algorithm preference is separated from semantic interpretation. | | **Q3b [10]** | Unit 7 · Classification and sentiment in industry | theory | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f24--q3b/) | | **Q4a [10]** | Unit 4 · Top-down syntax tree | case | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f24--q4a/) | | **Q4b [10]** | Unit 3 · Data acquisition and four augmentation methods | theory | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f24--q4b/) | | **Q5a [10]** | Unit 2 · Feature engineering, BoW and Euclidean | numerical | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f24--q5a/) | | **Q5b [5]** | Unit 5 · WSD approaches and uses | theory | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f24--q5b/) | | **Q5c [5]** | Unit 6 · Discourse analysis of a conversation | case | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f24--q5c/) | | **Q6a [10]** | Unit 3 · Henry bigram continuations | numerical | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f24--q6a/) · Case is retained as printed: Do and do are distinct. The corpus is not silently grammatically corrected. | | **Q6b [5]** | Unit 1 · Five linguistic levels | theory | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f24--q6b/) | | **Q6c [5]** | Unit 1 · NLU and NLG working together | theory | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f24--q6c/) | | **Q7a [10]** | Unit 4 · POS, chunks and boundary-aware HMM | mixed | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f24--q7a/) · Printed tags are used as training data; START and END are explicitly required and included. | | **Q7b [10]** | Unit 5 · Co-occurrence, dot products and plot | numerical | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f24--q7b/) | **Original offered marks by mapped topic, grouped without double counting:** Unit 2, 7: 1 parts, 0 known marks + 1 parts with no individual mark printed; Unit 4: 3 parts, 20 known marks + 1 parts with no individual mark printed; Unit 5: 3 parts, 15 known marks + 1 parts with no individual mark printed; Unit 6: 3 parts, 15 known marks + 1 parts with no individual mark printed; Unit 2, 4: 1 parts, 6 known marks; Unit 2, 5: 1 parts, 4 known marks; Unit 7: 2 parts, 20 known marks; Unit 3: 2 parts, 20 known marks; Unit 2: 1 parts, 10 known marks; Unit 1: 2 parts, 10 known marks. Joint-unit buckets remain joint; no invented split is assigned. These original-paper totals include any legacy tasks explicitly marked above; they are not a current-syllabus marks blueprint. ### 2024-02-12 · re-exam · R24 140 marks offered; 100 attempted. | Question [marks] | Current unit and topic | Question form | Answer / scope | | --- | --- | --- | --- | | **Q1a [5]** | Unit 6 · Anaphora versus cataphora | theory | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r24--q1a/) | | **Q1b [5]** | Unit 3 · Five corpora | theory | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r24--q1b/) | | **Q1c [5]** | Unit 1 · Linguistic foundations | theory | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r24--q1c/) | | **Q1d [5]** | Unit 3 · Five regex operators | theory | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r24--q1d/) | | **Q2a [10]** | Unit 3 · Wordlist puzzle and lexical resource | mixed | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r24--q2a/) · The dictionary is unspecified; an exact number of valid words would be fabricated. | | **Q2b [10]** | Unit 1, 7 · NLP applications and limitations | theory | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r24--q2b/) | | **Q3a [10]** | Unit 4 · Top-down restaurant parse | case | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r24--q3a/) · Original CFG is incomplete for the given bare-plural object. Exact failure and explicit repaired derivation are both shown. | | **Q3b [10]** | Unit 2 · Preprocessing pipeline | theory | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r24--q3b/) | | **Q4a [10]** | Unit 5 · Embeddings and Skip-gram | theory | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r24--q4a/) | | **Q4b [10]** | Unit 2, 4 · POS, NER, chunking and TTR | theory | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r24--q4b/) | | **Q5a [10]** | Unit 5 · Lexical and vector semantics | theory | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r24--q5a/) | | **Q5b [10]** | Unit 7 · Classification label structures | theory | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r24--q5b/) | | **Q6a [10]** | Unit 2, 7 · TF–IDF in document representation | mixed | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r24--q6a/) · The prompt requests an example but supplies no corpus. Numbers in the explanation are authored illustrations, not original numerical givens. | | **Q6b [10]** | Unit 4 · HMM tables and path | numerical | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r24--q6b/) | | **Q7a [10]** | Unit 7 · Healthcare and finance applications | theory | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r24--q7a/) | | **Q7b [10]** | Unit 7 · Naive Bayes sentiment | numerical | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r24--q7b/) | **Original offered marks by mapped topic, grouped without double counting:** Unit 6: 1 parts, 5 known marks; Unit 3: 3 parts, 20 known marks; Unit 1: 1 parts, 5 known marks; Unit 1, 7: 1 parts, 10 known marks; Unit 4: 2 parts, 20 known marks; Unit 2: 1 parts, 10 known marks; Unit 5: 2 parts, 20 known marks; Unit 2, 4: 1 parts, 10 known marks; Unit 7: 3 parts, 30 known marks; Unit 2, 7: 1 parts, 10 known marks. Joint-unit buckets remain joint; no invented split is assigned. These original-paper totals include any legacy tasks explicitly marked above; they are not a current-syllabus marks blueprint. ### 2023-12-01 · final · F23 120 marks offered; 100 attempted. Original has six main questions on three pages: Q1 compulsory and any four of Q2–Q6. Offered total 120, attempted 100. | Question [marks] | Current unit and topic | Question form | Answer / scope | | --- | --- | --- | --- | | **Q1a [5]** | Unit 5, 6 · WSD and pronoun resolution | theory | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f23--q1a/) | | **Q1b [5]** | Unit 5 · WordNet and its hierarchy | theory | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f23--q1b/) | | **Q1c [5]** | Unit 2, 7 · Sentiment feature extraction | numerical | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f23--q1c/) | | **Q1d [5]** | Unit 7 · Binary, multiclass and multilabel | theory | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f23--q1d/) | | **Q2a [10]** | Unit 3 · Text corpus structure and applications | theory | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f23--q2a/) | | **Q2b [10]** | Unit 5 · Lesk versus Walker and bank disambiguation | mixed | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f23--q2b/) · The original gives a sentence but no thesaurus/category mapping, so a unique numerical score cannot be recovered. | | **Q3a [10]** | Unit 6 · Hobbs algorithm and limitations | theory | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f23--q3a/) | | **Q3b [10]** | Unit 2 · Stemming and lemmatization | theory | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f23--q3b/) | | **Q4a [10]** | Unit 5 · Lexical and vector semantics | theory | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f23--q4a/) | | **Q4b [10]** | Unit 2 · Three-document TF–IDF | numerical | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f23--q4b/) | | **Q5a [10]** | Unit 5 · Word embeddings and CBOW | theory | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f23--q5a/) | | **Q5b [10]** | Unit 4 · HMM tables and given path | numerical | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f23--q5b/) | | **Q6a [10]** | Unit 7 · Text classification pipeline | theory | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f23--q6a/) | | **Q6b [10]** | Unit 2, 7 · VSM, TF and IDF in retrieval | theory | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f23--q6b/) | **Original offered marks by mapped topic, grouped without double counting:** Unit 5, 6: 1 parts, 5 known marks; Unit 5: 4 parts, 35 known marks; Unit 2, 7: 2 parts, 15 known marks; Unit 7: 2 parts, 15 known marks; Unit 3: 1 parts, 10 known marks; Unit 6: 1 parts, 10 known marks; Unit 2: 2 parts, 20 known marks; Unit 4: 1 parts, 10 known marks. Joint-unit buckets remain joint; no invented split is assigned. These original-paper totals include any legacy tasks explicitly marked above; they are not a current-syllabus marks blueprint. ### 2023-01-30 · re-exam · R23 140 marks offered; 100 attempted. Q1 header is unclear/conflicting [14/20], but all four parts print 5 marks each; sum=20. Main paper is 100 attempted marks. | Question [marks] | Current unit and topic | Question form | Answer / scope | | --- | --- | --- | --- | | **Q1a [5]** | Unit 2 · Normalization changes vocabulary | theory | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r23--q1a/) | | **Q1b [5]** | Unit 3 · Good corpus characteristics | theory | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r23--q1b/) | | **Q1c [5]** | Unit 5 · Word2Vec versus Doc2Vec | theory | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r23--q1c/) · legacy named-method extension; not separately named in current syllabus. Optional historical extension; not a separately prescribed current-syllabus method. | | **Q1d [5]** | Unit 1, 4 · Ambiguity in NLP | theory | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r23--q1d/) | | **Q2a [10]** | Unit 2 · Normalize the Tanisha–Simran passage | case | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r23--q2a/) · Stoplist and stemming algorithm are choices, not supplied numerical givens; both are stated. | | **Q2b [10]** | Unit 2, 5 · Text vectorization alternatives | theory | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r23--q2b/) | | **Q3a [10]** | Unit 1, 5, 6 · Semantics, pragmatics and two ambiguities | theory | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r23--q3a/) | | **Q3b [10]** | Unit 7 · Naive Bayes text classifier | mixed | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r23--q3b/) · Chinese corpus is an authored/reused teaching illustration here; it is actual given data in F22 Q5b, not this prompt. | | **Q4a [10]** | Unit 2 · Jaccard, cosine and Euclidean without preprocessing | numerical | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r23--q4a/) | | **Q4b [10]** | Unit 5 · CBOW and Skip-gram | theory | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r23--q4b/) | | **Q5a [10]** | Unit 3 · Password regex | case | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r23--q5a/) | | **Q5b [10]** | Unit 7 · Two NLP applications | theory | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r23--q5b/) | | **Q6a [10]** | Unit 6 · Mention-pair model | theory | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r23--q6a/) | | **Q6b [10]** | Unit 7 · Classification evaluation including ROC/AUC | mixed | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r23--q6b/) | | **Q7a [10]** | Unit 4 · Organization and facility mentions | case | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r23--q7a/) · Boundary and generic-mention decisions depend on the provided ACE annotation convention; the answer distinguishes them from a named-only interpretation. | | **Q7b [10]** | Unit 4 · Tagging and two complete parse trees | case | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r23--q7b/) | **Original offered marks by mapped topic, grouped without double counting:** Unit 2: 3 parts, 25 known marks; Unit 3: 2 parts, 15 known marks; Unit 5: 2 parts, 15 known marks; Unit 1, 4: 1 parts, 5 known marks; Unit 2, 5: 1 parts, 10 known marks; Unit 1, 5, 6: 1 parts, 10 known marks; Unit 7: 3 parts, 30 known marks; Unit 6: 1 parts, 10 known marks; Unit 4: 2 parts, 20 known marks. Joint-unit buckets remain joint; no invented split is assigned. These original-paper totals include any legacy tasks explicitly marked above; they are not a current-syllabus marks blueprint. **Current-syllabus qualification:** Q1c: historical named-method extension, excluded from current-only priority totals. The original whole-part marks remain visible so the paper is faithfully represented. ### 2022-11-22 · final · F22 140 marks offered; 100 attempted. | Question [marks] | Current unit and topic | Question form | Answer / scope | | --- | --- | --- | --- | | **Q1a [5]** | Unit 1 · Need and difficulties of NLP | theory | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f22--q1a/) | | **Q1b [5]** | Unit 1 · Levels of language analysis | theory | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f22--q1b/) | | **Q1c [5]** | Unit 7 · Medical chatbot | theory | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f22--q1c/) | | **Q1d [5]** | Unit 7 · Sentiment with logistic regression | mixed | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f22--q1d/) | | **Q2a [10]** | Unit 5 · CBOW and Skip-gram architecture | theory | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f22--q2a/) | | **Q2b [10]** | Unit 3 · Seven regex constructions | case | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f22--q2b/) | | **Q3a [10]** | Unit 2 · Text preprocessing | theory | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f22--q3a/) | | **Q3b [10]** | Unit 4 · Penn Treebank tagging | case | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f22--q3b/) · Context-sensitive tags and possessive tokenization are explicitly stated; reasonable corpus guidelines can differ on noun adjuncts. | | **Q4a [10]** | Unit 6 · Mention-pair coreference | theory | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f22--q4a/) | | **Q4b [10]** | Unit 7 · Text classification pipeline | theory | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f22--q4b/) | | **Q5a [10]** | Unit 3 · Corpora and structure | theory | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f22--q5a/) | | **Q5b [10]** | Unit 7 · Multinomial Naive Bayes | numerical | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f22--q5b/) | | **Q6a [10]** | Unit 5 · GloVe | theory | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f22--q6a/) | | **Q6b [10]** | Unit 2 · BoW and TF–IDF movie corpus | numerical | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f22--q6b/) | | **Q7a [10]** | Unit 4, 7 · Information extraction, chunking and NER | theory | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f22--q7a/) | | **Q7b [10]** | Unit 5 · Distributional hypothesis and cosine | mixed | [Read question + answer](https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f22--q7b/) | **Original offered marks by mapped topic, grouped without double counting:** Unit 1: 2 parts, 10 known marks; Unit 7: 4 parts, 30 known marks; Unit 5: 3 parts, 30 known marks; Unit 3: 2 parts, 20 known marks; Unit 2: 2 parts, 20 known marks; Unit 4: 1 parts, 10 known marks; Unit 6: 1 parts, 10 known marks; Unit 4, 7: 1 parts, 10 known marks. Joint-unit buckets remain joint; no invented split is assigned. These original-paper totals include any legacy tasks explicitly marked above; they are not a current-syllabus marks blueprint. ### Priority without skipping the syllabus The latest papers show the form and depth of questions, while the official current syllabus sets the learning scope. Authored drills, explanatory numbers and optional extensions are not counted as past questions. Source assumptions and ambiguous data are stated in their answers. --- ## TEE · What this NLP guide covers Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/about/scope--sources/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/about/scope/sources NLP · M2 and TEE · edition 3 October 2026 **M2 portion confirmed by you:** Unit 3 from PDF slide 62 to 160; Unit 4 complete (128 slides); Unit 5 slides 1–57. Slide 62 of Unit 3 starts N-grams. Slide 57 of Unit 5 completes the Naive Bayes WSD example. Vector-space models start on slide 58 and belong to TEE here. **M2 pattern confirmed by you:** both 2-mark questions compulsory; any two of the three 3-mark questions. Total attempted 10. Course policy gives one hour. Three newly authored, fully solved M2 practice papers use that pattern. **TEE:** all seven units of the official AY 2026–27 syllabus. The new ZIP supplies faculty slide packs for Units 1–5 plus a regular-expressions handout. Units 6–7 are syllabus-based explanations supplemented with the standard references below; their faculty slide packs were not supplied. The official syllabus includes GloVe and spelling correction, and both are retained even though the policy’s lecture table abbreviates them. **Previous papers found:** final exam dated 2 December 2025 and re-exam dated 9 February 2026, with answer keys, inside Semester V.zip. Programme copies were deduplicated. Every supplied question subpart is mapped to the current syllabus; no question was excluded as out of syllabus. Only wholly M2-eligible subparts appear in the M2 paper tab; mixed questions remain in the complete TEE papers. The current syllabus lists course code 702AI0C037. The supplied policy and old papers use 702AI0C009; the policy also retains an older programme/semester header. Topic coverage follows the current official syllabus and your exact M2 cutoff, rather than treating the old header as a different course. ### Conventions that must stay visible Course bigram answer keys count both start and end symbols in vocabulary size. Worked PYQs state and follow that convention; other exercises state their own vocabulary explicitly. A start symbol is conditioning context, so alternative textbook conventions may exclude it from prediction vocabulary and produce a different smoothed number. HMM path scores and Naive Bayes class scores are labelled as unnormalised when applicable. Any normalisation, case-folding or lemmatisation assumption is stated before counting. ### Reading references - Your uploaded course policy, official Semester V syllabus, five NLP slide packs and regular-expressions handout. - Your uploaded final and re-exam questions and answer keys (AY 2025–26). - [Jurafsky & Martin · Speech and Language Processing](https://web.stanford.edu/~jurafsky/slp3/) — terminology and standard methods. - [NLTK Book](https://www.nltk.org/book/) — corpus, tagging, parsing and information-extraction concepts. - [Tetreault · A Corpus-Based Evaluation of Centering and Pronoun Resolution](https://aclanthology.org/J01-4003.pdf) — precise Hobbs traversal. Examples are teaching problems unless explicitly identified by their paper and question number. Practice-paper marks indicate intended answer length, not an official forecast. Reading alone cannot guarantee a score; writing the solutions independently is what makes the procedures usable under exam time. ### TEE depth review · 10 October 2026 The core TEE route retains the faculty topics CYK/CKY, grammar conversion, interpolation and Stupid Backoff. Add-k and HMM forward/posterior calculations are available separately under Optional extensions; they are not part of the core reading or formula sequence. The two TEE timed-practice entries reproduce the supplied actual papers and are not additional unseen mock papers. Method references: [Jurafsky and Martin: Hidden Markov Models](https://web.stanford.edu/~jurafsky/slp3/A.pdf); [Jurafsky and Martin: Constituency Parsing](https://web.stanford.edu/~jurafsky/slp3/E.pdf). New numerical inputs and worked traces are teaching examples. --- ## TEE · Video companions in your NotebookLM Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/about/scope--videos/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/about/scope/videos NLP · M2 and TEE · edition 3 October 2026 Your M2 notebook contains the exact three slide ranges and checked numerical teaching examples. The requested video topics are N-grams and Laplace smoothing, HMM/Viterbi, and Naive Bayes WSD. Use the written solutions here for the complete calculations. [Open your NLP M2 notebook →](https://notebook.google.com/notebook/a270ef1e-c285-4d31-9db1-302090c5f8b1) The notebook is in your signed-in Google account. Video generation and availability are shown in its Studio panel. Generated narration is a supplementary explanation; the numerical results in this guide have been independently checked. ## Complete video catalog updated\_utc : 2026-10-10T11:17:01.698545+00:00 notebooks : m2 : tee : lesson\_map : m2 : foundations/language : - nlp-m2-01-foundations-counts-probability foundations/notation : - nlp-m2-01-foundations-counts-probability foundations/probability : - nlp-m2-01-foundations-counts-probability m2-language-models/words-counts-probability : - nlp-m2-01-foundations-counts-probability foundations/bayes : - nlp-m2-02-bayes-generative-discriminative m2-pos-tagging/generative-discriminative : - nlp-m2-02-bayes-generative-discriminative m2-language-models/n-grams-construction : - nlp-m2-03-ngrams-language-models m2-language-models/collocations-and-lm : - nlp-m2-03-ngrams-language-models m2-language-models/chain-rule-markov : - nlp-m2-03-ngrams-language-models m2-count-to-probability/unigram-estimation : - nlp-m2-03-ngrams-language-models m2-count-to-probability/bigram-count-table : - nlp-m2-04-bigram-trigram-tables m2-count-to-probability/trigram-estimation : - nlp-m2-04-bigram-trigram-tables m2-count-to-probability/class-count-table-questions : - nlp-m2-04-bigram-trigram-tables m2-evaluation-smoothing/laplace-basics : - nlp-m2-05-laplace-smoothing-discount m2-evaluation-smoothing/smoothed-sentence : - nlp-m2-05-laplace-smoothing-discount m2-evaluation-smoothing/adjusted-count-discount : - nlp-m2-05-laplace-smoothing-discount m2-evaluation-smoothing/evaluation-perplexity : - nlp-m2-06-perplexity-logs-calculator m2-evaluation-smoothing/log-probabilities : - nlp-m2-06-perplexity-logs-calculator m2-evaluation-smoothing/class-perplexity-characters : - nlp-m2-06-perplexity-logs-calculator foundations/calculator : - nlp-m2-06-perplexity-logs-calculator m2-pos-tagging/word-classes-pos : - nlp-m2-07-pos-tags-tagger-methods m2-pos-tagging/tagger-methods : - nlp-m2-07-pos-tags-tagger-methods m2-hmm/markov-chain : - nlp-m2-08-markov-hmm-components m2-hmm/hmm-components : - nlp-m2-08-markov-hmm-components m2-hmm/transition-emission-estimation : - nlp-m2-08-markov-hmm-components m2-hmm/hot-cold-paths : - nlp-m2-09-hmm-joint-paths m2-hmm/tagging-corpus-case : - nlp-m2-09-hmm-joint-paths m2-hmm-decoding/joint-vs-posterior : - nlp-m2-09-hmm-joint-paths m2-hmm-decoding/sequence-count : - nlp-m2-10-viterbi-decoding m2-hmm-decoding/viterbi-explained : - nlp-m2-10-viterbi-decoding m2-chunks-grammar/chunking-chinking : - nlp-m2-11-chunking-chinking m2-chunks-grammar/cfg-foundation : - nlp-m2-12-cfg-top-down-parsing m2-chunks-grammar/structural-ambiguity : - nlp-m2-12-cfg-top-down-parsing m2-parsing/top-down-theory : - nlp-m2-12-cfg-top-down-parsing m2-parsing/top-down-examples : - nlp-m2-12-cfg-top-down-parsing m2-parsing/shift-reduce-theory : - nlp-m2-13-bottom-up-shift-reduce m2-parsing/shift-reduce-examples : - nlp-m2-13-bottom-up-shift-reduce m2-ner/ner-definition : - nlp-m2-14-named-entity-recognition m2-ner/ner-approaches : - nlp-m2-14-named-entity-recognition m2-ner/ner-worked-examples : - nlp-m2-14-named-entity-recognition u5-meaning/u5-words : - nlp-m2-15-lexical-meaning-wordnet u5-meaning/u5-relations : - nlp-m2-15-lexical-meaning-wordnet u5-meaning/u5-wordnet : - nlp-m2-15-lexical-meaning-wordnet u5-wsd/u5-wsd-basics : - nlp-m2-16-lesk-wsd u5-wsd/u5-lesk : - nlp-m2-16-lesk-wsd u5-wsd/u5-walker : - nlp-m2-17-walker-graph-wsd u5-wsd/u5-graph : - nlp-m2-17-walker-graph-wsd u5-naive-bayes/u5-bayes-meaning : - nlp-m2-18-naive-bayes-wsd u5-naive-bayes/u5-bayes-steps : - nlp-m2-18-naive-bayes-wsd u5-naive-bayes/u5-bayes-mam : - nlp-m2-18-naive-bayes-wsd u5-naive-bayes/u5-bayes-practice : - nlp-m2-18-naive-bayes-wsd tee : foundations/language : - nlp-m2-01-foundations-counts-probability foundations/notation : - nlp-m2-01-foundations-counts-probability foundations/probability : - nlp-m2-01-foundations-counts-probability m2-language-models/words-counts-probability : - nlp-m2-01-foundations-counts-probability foundations/bayes : - nlp-m2-02-bayes-generative-discriminative m2-pos-tagging/generative-discriminative : - nlp-m2-02-bayes-generative-discriminative m2-language-models/n-grams-construction : - nlp-m2-03-ngrams-language-models m2-language-models/collocations-and-lm : - nlp-m2-03-ngrams-language-models m2-language-models/chain-rule-markov : - nlp-m2-03-ngrams-language-models m2-count-to-probability/unigram-estimation : - nlp-m2-03-ngrams-language-models m2-count-to-probability/bigram-count-table : - nlp-m2-04-bigram-trigram-tables m2-count-to-probability/trigram-estimation : - nlp-m2-04-bigram-trigram-tables m2-count-to-probability/class-count-table-questions : - nlp-m2-04-bigram-trigram-tables m2-evaluation-smoothing/laplace-basics : - nlp-m2-05-laplace-smoothing-discount m2-evaluation-smoothing/smoothed-sentence : - nlp-m2-05-laplace-smoothing-discount m2-evaluation-smoothing/adjusted-count-discount : - nlp-m2-05-laplace-smoothing-discount m2-evaluation-smoothing/evaluation-perplexity : - nlp-m2-06-perplexity-logs-calculator m2-evaluation-smoothing/log-probabilities : - nlp-m2-06-perplexity-logs-calculator m2-evaluation-smoothing/class-perplexity-characters : - nlp-m2-06-perplexity-logs-calculator foundations/calculator : - nlp-m2-06-perplexity-logs-calculator m2-pos-tagging/word-classes-pos : - nlp-m2-07-pos-tags-tagger-methods m2-pos-tagging/tagger-methods : - nlp-m2-07-pos-tags-tagger-methods m2-hmm/markov-chain : - nlp-m2-08-markov-hmm-components m2-hmm/hmm-components : - nlp-m2-08-markov-hmm-components m2-hmm/transition-emission-estimation : - nlp-m2-08-markov-hmm-components m2-hmm/hot-cold-paths : - nlp-m2-09-hmm-joint-paths m2-hmm/tagging-corpus-case : - nlp-m2-09-hmm-joint-paths m2-hmm-decoding/joint-vs-posterior : - nlp-m2-09-hmm-joint-paths m2-hmm-decoding/sequence-count : - nlp-m2-10-viterbi-decoding m2-hmm-decoding/viterbi-explained : - nlp-m2-10-viterbi-decoding m2-chunks-grammar/chunking-chinking : - nlp-m2-11-chunking-chinking m2-chunks-grammar/cfg-foundation : - nlp-m2-12-cfg-top-down-parsing m2-chunks-grammar/structural-ambiguity : - nlp-m2-12-cfg-top-down-parsing m2-parsing/top-down-theory : - nlp-m2-12-cfg-top-down-parsing m2-parsing/top-down-examples : - nlp-m2-12-cfg-top-down-parsing m2-parsing/shift-reduce-theory : - nlp-m2-13-bottom-up-shift-reduce m2-parsing/shift-reduce-examples : - nlp-m2-13-bottom-up-shift-reduce m2-ner/ner-definition : - nlp-m2-14-named-entity-recognition m2-ner/ner-approaches : - nlp-m2-14-named-entity-recognition m2-ner/ner-worked-examples : - nlp-m2-14-named-entity-recognition u5-meaning/u5-words : - nlp-m2-15-lexical-meaning-wordnet u5-meaning/u5-relations : - nlp-m2-15-lexical-meaning-wordnet u5-meaning/u5-wordnet : - nlp-m2-15-lexical-meaning-wordnet u5-wsd/u5-wsd-basics : - nlp-m2-16-lesk-wsd u5-wsd/u5-lesk : - nlp-m2-16-lesk-wsd u5-wsd/u5-walker : - nlp-m2-17-walker-graph-wsd u5-wsd/u5-graph : - nlp-m2-17-walker-graph-wsd u5-naive-bayes/u5-bayes-meaning : - nlp-m2-18-naive-bayes-wsd u5-naive-bayes/u5-bayes-steps : - nlp-m2-18-naive-bayes-wsd u5-naive-bayes/u5-bayes-mam : - nlp-m2-18-naive-bayes-wsd u5-naive-bayes/u5-bayes-practice : - nlp-m2-18-naive-bayes-wsd u1-introduction/u1-nlp : - tee-01 u1-introduction/u1-language : - tee-01 u1-introduction/u1-approaches : - tee-01 u1-introduction/u1-nlu-nlg : - tee-02 u1-introduction/u1-pipeline : - tee-02 u1-introduction/u1-challenges : - tee-02 u2-preprocess/u2-tokenize : - tee-03 u2-preprocess/u2-normalization : - tee-03 u2-preprocess/u2-stem-lemma : - tee-03 u2-preprocess/u2-ttr : - tee-03 u2-representation/u2-vocab : - tee-04 u2-representation/u2-label-onehot : - tee-04 u2-representation/u2-bow : - tee-04 u2-representation/u2-tfidf : - tee-05 u2-similarity/u2-cosine : - tee-05 u2-representation/u2-slide-combined : - tee-05 u2-similarity/u2-jaccard : - tee-06 u2-similarity/u2-distances : - tee-06 u2-similarity/u2-choice : - tee-06 u3-corpora/u3-corpus : - tee-07 u3-corpora/u3-types : - tee-07 u3-corpora/u3-access : - tee-07 u3-corpora/u3-resources : - tee-07 u3-corpora/u3-acquisition : - tee-07 u3-corpora/u3-links : - tee-07 u3-regex/regex-language : - tee-08 u3-regex/regex-functions : - tee-08 u3-regex/regex-worked : - tee-08 u3-regex/regex-reference : - tee-08 u5-vectors/u5-distribution : - tee-09 u5-vectors/u5-representation : - tee-09 u5-vectors/u5-sentence : - tee-09 u5-vectors/u5-word2vec : - tee-10 u5-glove/u5-glove-theory : - tee-11 u5-glove/u5-glove-loss : - tee-11 u6/reference : - tee-12 u6/resolution : - tee-12 u6/hobbs : - tee-13 u7/classification : - tee-14 u7/sentiment : - tee-14 u7/evaluation : - tee-14 u7/spelling : - tee-15 u7/translation : - tee-16 u7/qa : - tee-17 u7/extraction : - tee-18 u7/retrieval : - tee-18 written\_only : m2 tee : u3-smoothing-extensions/add-k : This worked lesson was added after the video plan. No matching video has been generated or reviewed. u3-smoothing-extensions/interpolation : This worked lesson was added after the video plan. No matching video has been generated or reviewed. u3-smoothing-extensions/stupid-backoff : This worked lesson was added after the video plan. No matching video has been generated or reviewed. u4-forward-cky/forward-probability : This worked lesson was added after the video plan. No matching video has been generated or reviewed. u4-forward-cky/cnf : This worked lesson was added after the video plan. No matching video has been generated or reviewed. u4-forward-cky/cky : This worked lesson was added after the video plan. No matching video has been generated or reviewed. service\_limit : status : blocked reset\_local : 2026-10-03T22:06:00+05:30 second\_account\_reset\_local : 2026-10-03T22:51:00+05:30 remaining\_not\_submitted : 12 ### M2 01 — NLP basics, counts and conditional probability id : nlp-m2-01-foundations-counts-probability title : M2 01 — NLP basics, counts and conditional probability title\_confirmed : false topic\_title : NLP basics, counts and conditional probability mode : m2 kind : focused status : queued duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### Bayes Rule & Models id : nlp-m2-02-bayes-generative-discriminative title : Bayes Rule & Models title\_confirmed : true topic\_title : Bayes rule and generative versus discriminative models mode : m2 kind : focused status : available duration : 10:51 reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 03: N-grams & Unigrams id : nlp-m2-03-ngrams-language-models title : NLP M2 03: N-grams & Unigrams title\_confirmed : true topic\_title : N-grams, chain rule and unigram models mode : m2 kind : focused status : available duration : 9:52 reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### Bigram & Trigram Probs id : nlp-m2-04-bigram-trigram-tables title : Bigram & Trigram Probs title\_confirmed : true topic\_title : Build bigram and trigram probabilities from a corpus mode : m2 kind : focused status : available duration : 10:05 reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### M2 05: Laplace Smoothing id : nlp-m2-05-laplace-smoothing-discount title : M2 05: Laplace Smoothing title\_confirmed : true topic\_title : Laplace smoothing, unseen sequences and adjusted counts mode : m2 kind : focused status : available duration : 11:11 reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### M2 06 — Perplexity, logarithms and calculator working id : nlp-m2-06-perplexity-logs-calculator title : M2 06 — Perplexity, logarithms and calculator working title\_confirmed : false topic\_title : Perplexity, logarithms and calculator working mode : m2 kind : focused status : queued duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — POS tagging and Brill | Reviewed explanation and worked rule id : nlp-m2-07-pos-tags-tagger-methods title : NLP M2 — POS tagging and Brill | Reviewed explanation and worked rule title\_confirmed : true topic\_title : POS tags, ambiguity and tagging methods mode : m2 kind : focused status : available duration : 9:06 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Tagger methods, Brill net gain and the full conditional rule; clarified tagging and imperative explanations. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### M2 08: Markov Paths & HMMs id : nlp-m2-08-markov-hmm-components title : M2 08: Markov Paths & HMMs title\_confirmed : true topic\_title : Markov paths, HMM components and training counts mode : m2 kind : focused status : available duration : 9:45 reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### HMM Path Probabilities id : nlp-m2-09-hmm-joint-paths title : HMM Path Probabilities title\_confirmed : true topic\_title : Complete HMM path scores and observation probabilities mode : m2 kind : focused status : available duration : 8:34 reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### M2 10 Viterbi Decoding id : nlp-m2-10-viterbi-decoding title : M2 10 Viterbi Decoding title\_confirmed : true topic\_title : Viterbi decoding with backpointers mode : m2 kind : focused status : available duration : 9:34 reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — Chunking and Chinking | Reviewed rules and worked examples id : nlp-m2-11-chunking-chinking title : NLP M2 — Chunking and Chinking | Reviewed rules and worked examples title\_confirmed : true topic\_title : Chunking and chinking from POS-tagged words mode : m2 kind : focused status : available duration : 8:22 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Correct POS labels including JJ = adjective, full NP rule application and chinking boundaries. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — CFG and top-down parsing | Reviewed derivations and trees id : nlp-m2-12-cfg-top-down-parsing title : NLP M2 — CFG and top-down parsing | Reviewed derivations and trees title\_confirmed : true topic\_title : CFG, ambiguity and top-down parsing mode : m2 kind : focused status : available duration : 8:54 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Complete grammars, derivations and valid parse trees; corrected the acceptance condition. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### M2 13 — Shift-reduce parsing and backtracking id : nlp-m2-13-bottom-up-shift-reduce title : M2 13 — Shift-reduce parsing and backtracking title\_confirmed : false topic\_title : Shift-reduce parsing and backtracking mode : m2 kind : focused status : queued duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### M2 14 — Named entities, boundaries and recognition approaches id : nlp-m2-14-named-entity-recognition title : M2 14 — Named entities, boundaries and recognition approaches title\_confirmed : false topic\_title : Named entities, boundaries and recognition approaches mode : m2 kind : focused status : queued duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### M2 15 — Lexical meaning, semantic relations and WordNet id : nlp-m2-15-lexical-meaning-wordnet title : M2 15 — Lexical meaning, semantic relations and WordNet title\_confirmed : false topic\_title : Lexical meaning, semantic relations and WordNet mode : m2 kind : focused status : queued duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### M2 16 WSD and Lesk Scoring id : nlp-m2-16-lesk-wsd title : M2 16 WSD and Lesk Scoring title\_confirmed : true topic\_title : Word-sense disambiguation and Lesk overlap scoring mode : m2 kind : focused status : available duration : 10:17 reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### M2 17 — Walker category scoring and graph-based WSD id : nlp-m2-17-walker-graph-wsd title : M2 17 — Walker category scoring and graph-based WSD title\_confirmed : false topic\_title : Walker category scoring and graph-based WSD mode : m2 kind : focused status : queued duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### Naive Bayes WSD Numerical id : nlp-m2-18-naive-bayes-wsd title : Naive Bayes WSD Numerical title\_confirmed : true topic\_title : Naive Bayes WSD: a complete corpus numerical mode : m2 kind : focused status : available duration : 11:35 reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### TEE 01 — NLP foundations: language units, tasks and learning approaches id : tee-01 title : TEE 01 — NLP foundations: language units, tasks and learning approaches title\_confirmed : false topic\_title : NLP foundations: language units, tasks and learning approaches mode : tee kind : focused status : queued duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLU, NLG, the NLP pipeline and language challenges id : tee-02 title : NLU, NLG, the NLP pipeline and language challenges title\_confirmed : false topic\_title : NLU, NLG, the NLP pipeline and language challenges mode : tee kind : focused status : not\_generated duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### Preprocessing and morphology: tokenize, normalize, stem, lemmatize, count TTR id : tee-03 title : Preprocessing and morphology: tokenize, normalize, stem, lemmatize, count TTR title\_confirmed : false topic\_title : Preprocessing and morphology: tokenize, normalize, stem, lemmatize, count TTR mode : tee kind : focused status : not\_generated duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### Vocabulary, label IDs, one-hot encoding and bag of words id : tee-04 title : Vocabulary, label IDs, one-hot encoding and bag of words title\_confirmed : false topic\_title : Vocabulary, label IDs, one-hot encoding and bag of words mode : tee kind : focused status : not\_generated duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### TEE 05 — TF-IDF and cosine: from the corpus to the final similarity id : tee-05 title : TEE 05 — TF-IDF and cosine: from the corpus to the final similarity title\_confirmed : false topic\_title : TF-IDF and cosine: from the corpus to the final similarity mode : tee kind : focused status : queued duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### TEE 06 — Jaccard and distances: intersection, union, Manhattan, Euclidean, Minkowski id : tee-06 title : TEE 06 — Jaccard and distances: intersection, union, Manhattan, Euclidean, Minkowski title\_confirmed : false topic\_title : Jaccard and distances: intersection, union, Manhattan, Euclidean, Minkowski mode : tee kind : focused status : queued duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### Corpora and lexical resources: where NLP evidence comes from id : tee-07 title : Corpora and lexical resources: where NLP evidence comes from title\_confirmed : false topic\_title : Corpora and lexical resources: where NLP evidence comes from mode : tee kind : focused status : not\_generated duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### Regular expressions: build and explain exact text patterns id : tee-08 title : Regular expressions: build and explain exact text patterns title\_confirmed : false topic\_title : Regular expressions: build and explain exact text patterns mode : tee kind : focused status : not\_generated duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### TEE 09 — Distributional semantics, co-occurrence matrices and sentence vectors id : tee-09 title : TEE 09 — Distributional semantics, co-occurrence matrices and sentence vectors title\_confirmed : false topic\_title : Distributional semantics, co-occurrence matrices and sentence vectors mode : tee kind : focused status : queued duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### TEE 10 — Word2Vec: CBOW and skip-gram, training pairs and softmax numericals id : tee-10 title : TEE 10 — Word2Vec: CBOW and skip-gram, training pairs and softmax numericals title\_confirmed : false topic\_title : Word2Vec: CBOW and skip-gram, training pairs and softmax numericals mode : tee kind : focused status : queued duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### GloVe: global counts, probability ratios and weighted loss id : tee-11 title : GloVe: global counts, probability ratios and weighted loss title\_confirmed : false topic\_title : GloVe: global counts, probability ratios and weighted loss mode : tee kind : focused status : not\_generated duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### Reference and coreference: mentions, antecedents and pronoun resolution id : tee-12 title : Reference and coreference: mentions, antecedents and pronoun resolution title\_confirmed : false topic\_title : Reference and coreference: mentions, antecedents and pronoun resolution mode : tee kind : focused status : not\_generated duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### Hobbs algorithm: parse-tree traversal with three complete traces id : tee-13 title : Hobbs algorithm: parse-tree traversal with three complete traces title\_confirmed : false topic\_title : Hobbs algorithm: parse-tree traversal with three complete traces mode : tee kind : focused status : not\_generated duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### Text classification, sentiment and evaluation numericals id : tee-14 title : Text classification, sentiment and evaluation numericals title\_confirmed : false topic\_title : Text classification, sentiment and evaluation numericals mode : tee kind : focused status : not\_generated duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### TEE 15 — Spelling correction: full edit-distance tables and noisy-channel ranking id : tee-15 title : TEE 15 — Spelling correction: full edit-distance tables and noisy-channel ranking title\_confirmed : false topic\_title : Spelling correction: full edit-distance tables and noisy-channel ranking mode : tee kind : focused status : queued duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### Machine translation: meaning preservation and worked processing id : tee-16 title : Machine translation: meaning preservation and worked processing title\_confirmed : false topic\_title : Machine translation: meaning preservation and worked processing mode : tee kind : focused status : not\_generated duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### Question answering: identify, retrieve and justify the answer id : tee-17 title : Question answering: identify, retrieve and justify the answer title\_confirmed : false topic\_title : Question answering: identify, retrieve and justify the answer mode : tee kind : focused status : not\_generated duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### Information extraction and retrieval: records, indexes and precision/recall id : tee-18 title : Information extraction and retrieval: records, indexes and precision/recall title\_confirmed : false topic\_title : Information extraction and retrieval: records, indexes and precision/recall mode : tee kind : focused status : not\_generated duration reviewed : false notebook\_url : youtube\_url corrected\_visuals : false upload\_pending\_correction : false correction\_note related\_topic\_ids These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — N-grams and Laplace | Reviewed worked numericals id : overview-ngrams-laplace title : NLP M2 — N-grams and Laplace | Reviewed worked numericals title\_confirmed : true topic\_title : NLP M2: N-Grams & Laplace mode : m2 kind : supplemental status : available duration : 7:41 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Complete counts and four Laplace factors; corrected formulas, conditional-factor wording and rounding. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids : - nlp-m2-03-ngrams-language-models - nlp-m2-04-bigram-trigram-tables - nlp-m2-05-laplace-smoothing-discount These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — HMM and Viterbi | Reviewed worked numericals id : overview-hmm-viterbi title : NLP M2 — HMM and Viterbi | Reviewed worked numericals title\_confirmed : true topic\_title : HMM Prob & Viterbi mode : m2 kind : supplemental status : available duration : 8:30 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Complete HMM factors, all Viterbi candidate scores and backpointers; clarified zero emissions. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids : - nlp-m2-08-markov-hmm-components - nlp-m2-09-hmm-joint-paths - nlp-m2-10-viterbi-decoding These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — Word-sense disambiguation | Reviewed Lesk and Bayes numericals id : overview-wsd title : NLP M2 — Word-sense disambiguation | Reviewed Lesk and Bayes numericals title\_confirmed : true topic\_title : WSD: Detective Math Toolkit mode : m2 kind : supplemental status : available duration : 6:11 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Lesk variants, conditional Bayes definitions, smoothing and both worked posteriors; corrected spoken claims. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids : - nlp-m2-16-lesk-wsd - nlp-m2-17-walker-graph-wsd - nlp-m2-18-naive-bayes-wsd These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — Probability and Bayes | Reviewed step-by-step numericals id : overview-probability-bayes title : NLP M2 — Probability and Bayes | Reviewed step-by-step numericals title\_confirmed : true topic\_title : NLP M2 — Probability and Bayes | Reviewed step-by-step numericals mode : m2 kind : supplemental status : available duration : 8:53 reviewed : true notebook\_url : youtube\_url corrected\_visuals : true upload\_pending\_correction : true correction\_note : Complete HMM and Naive Bayes calculations; clarified joint scores versus model posteriors. Full transcript and detected slides checked; short voice corrections included. The corrected YouTube upload is still processing. related\_topic\_ids : - nlp-m2-01-foundations-counts-probability - nlp-m2-02-bayes-generative-discriminative These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. ### NLP M2 — Perplexity | Reviewed formulas and solved numericals id : overview-perplexity-logs title : NLP M2 — Perplexity | Reviewed formulas and solved numericals title\_confirmed : true topic\_title : NLP M2 — Perplexity | Reviewed formulas and solved numericals mode : m2 kind : supplemental status : available duration : 9:35 reviewed : true notebook\_url : youtube\_url : corrected\_visuals : true upload\_pending\_correction : false correction\_note : Readable formulas and all worked examples; corrected comparison conditions and context claims. Full transcript and detected slides checked; short voice corrections included. related\_topic\_ids : - nlp-m2-06-perplexity-logs-calculator These are recorded generation statuses, not a live service check. Private YouTube and signed-in NotebookLM links require the owner's account. This export publishes existing catalog metadata, not private video files or an invented transcript. --- ## TEE · Add-k: preserve a normalized vocabulary row Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/about/optional-extensions--add-k/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/about/optional-extensions/add-k Additional explanation; no corresponding question in the two supplied TEE papers. No exam marks are implied. **Optional extension.** Add-k is a supporting generalization of Laplace smoothing. The supplied TEE papers ask add-one bigram sentence probabilities, not an add-k calculation. Complete the actual solved papers first. **Add-k smoothing generalizes add-one smoothing.** Add the same positive pseudo-count k to each allowed outcome. There are V outcomes, so the total pseudo-count is kV. Setting k=1 recovers Laplace smoothing. A smaller k usually changes the original counts less; it is not automatically the best choice for every dataset. $$ P_k(w\mid h)=\frac{C(h,w)+k}{C(h)+kV},\qquad k>0. $$ Use one fixed vocabulary, including an explicit unknown-word policy if needed. Choose k from training/validation evidence, not by fitting the final test results. Unigram smoothing uses N, the total token count, in place of C(h). **Question. After history h, counts for the complete next-token vocabulary [a,b,c] are [3,1,0]. Calculate add-k probabilities for k=0.5 and check the row sum. [Authored worked example]** ### Answer #### Given C(h)=4; V=3; k=1/2; the zero-count word c is an allowed outcome. #### To find All three probabilities and their interpretation. #### Formula and why it applies $$ P_k(w\mid h)=\frac{C(h,w)+k}{C(h)+kV} $$ Smoothing must adjust the denominator for every pseudo-count added across the row. #### Solution ##### Step 1. Form the new total $$ 4+(0.5)(3)=5.5. $$ ##### Step 2. Calculate a $$ P(a\mid h)=\frac{3+0.5}{5.5}=\frac7{11}\approx0.636364. $$ ##### Step 3. Calculate b and c separately $$ P(b\mid h)=\frac{1+0.5}{5.5}=\frac3{11}\approx0.272727. $$ $$ P(c\mid h)=\frac{0+0.5}{5.5}=\frac1{11}\approx0.090909. $$ ##### Step 4. Check normalization $$ \frac7{11}+\frac3{11}+\frac1{11}=1. $$ **Final answer.** The row is [7/11,3/11,1/11]. The unseen allowed outcome c receives positive probability, and the row remains a probability distribution. This worked lesson was added after the video plan. No matching video has been generated or reviewed. --- ## TEE · Forward HMM: sum paths and normalize posteriors Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/about/optional-extensions--forward-probability/ Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/about/optional-extensions/forward-probability Additional explanation; no corresponding question in the two supplied TEE papers. No exam marks are implied. **Optional extension.** Forward totals and tag-path posteriors are optional HMM extensions here. Both supplied TEE papers ask emission and transition matrices, then the joint probability of a given tag sequence. Complete the actual solved papers first. **The forward algorithm finds the total probability of observations when the hidden path is unknown.** Its cell αₜ(j) is the joint probability of the observed prefix o₁…oₜ and ending in hidden state j, summed over every earlier hidden path. It stores two totals per time step in the two-state example; it does not choose one winning predecessor. $$ \alpha_1(j)=\pi_jb_j(o_1) $$ $$ \alpha_t(j)=b_j(o_t)\sum_i\alpha_{t-1}(i)a_{ij} $$ $$ P(O)=\sum_j\alpha_T(j). $$ πⱼ is the first-state probability; aᵢⱼ is transition i→j; bⱼ(oₜ) is the probability that state j emits the current observation. Multiply each earlier total by its transition, **add all incoming alternatives**, then multiply by the current emission. With explicit END transitions, terminate using Σⱼ αT(j)aⱼ,END instead. Do not invent END factors in a fixed-length model that has none. | Question | Operation | Output | | --- | --- | --- | | Given one hidden path and the words | Multiply its start, transitions and emissions | P(path, observations) | | Find the best hidden path | Viterbi: maximum, with backpointers | Best path and its joint score | | Find observation probability | Forward: sum incoming paths | P(observations) | | Find a specified path posterior | Divide its joint score by P(observations) | P(path | observations) | **Question. Use the HMM tables below to compute P(1,3,1) with the forward algorithm. Compare with the Viterbi path H,H,C whose joint score is 0.00672, then obtain that path’s posterior. [Authored worked example]** ### Answer #### Given Use H,C state order and the tables shown. Three observations are 1,3,1. There is no END state. | Quantity | H | C | | --- | --- | --- | | Initial π | 0.8 | 0.2 | | From H → next state | 0.7 | 0.3 | | From C → next state | 0.4 | 0.6 | | State | Emit 1 | Emit 2 | Emit 3 | | --- | --- | --- | --- | | H | 0.2 | 0.4 | 0.4 | | C | 0.5 | 0.4 | 0.1 | #### To find Every forward cell, total observation probability, and P(H,H,C | 1,3,1). #### Formula and why it applies $$ \alpha_t(j)=b_j(o_t)\sum_i\alpha_{t-1}(i)a_{ij} $$ All hidden paths are mutually exclusive alternatives, so their scores must be added. Viterbi’s maximum answers a different question. #### Solution ##### Step 1. Initialize observation 1 $$ \alpha_1(H)=0.8(0.2)=0.16,\qquad\alpha_1(C)=0.2(0.5)=0.10. $$ ##### Step 2. Sum incoming paths to H at observation 3 $$ \alpha_2(H)=[0.16(0.7)+0.10(0.4)](0.4) $$ $$ =[0.112+0.040](0.4)=0.0608. $$ ##### Step 3. Sum incoming paths to C at observation 3 $$ \alpha_2(C)=[0.16(0.3)+0.10(0.6)](0.1) $$ $$ =[0.048+0.060](0.1)=0.0108. $$ ##### Step 4. Calculate final H at observation 1 $$ \alpha_3(H)=[0.0608(0.7)+0.0108(0.4)](0.2) $$ $$ =[0.04256+0.00432](0.2)=0.009376. $$ ##### Step 5. Calculate final C at observation 1 $$ \alpha_3(C)=[0.0608(0.3)+0.0108(0.6)](0.5) $$ $$ =[0.01824+0.00648](0.5)=0.01236. $$ ##### Step 6. Add the final-state alternatives $$ P(1,3,1)=0.009376+0.01236=0.021736. $$ ##### Step 7. Read the table as a trace | State / time | 1: observe 1 | 2: observe 3 | 3: observe 1 | | --- | --- | --- | --- | | H | 0.16 | 0.0608 | 0.009376 | | C | 0.10 | 0.0108 | 0.01236 | | Column total | 0.26 | 0.0716 | 0.021736 | For each new cell, draw arrows from both cells in the previous column. Along each arrow multiply by its transition; add the two contributions and multiply by the destination’s emission. ##### Step 8. Normalize the specified best path $$ P(H,H,C\mid1,3,1)=\frac{0.00672}{0.021736}=\frac{840}{2717}\approx0.309164. $$ The best single path has only part of the total mass. The remaining paths account for the rest. **Final answer.** P(observations)=0.021736. Viterbi selects H,H,C with joint score 0.00672; its posterior is about 0.309164, not 0.00672. **Question. With the same explicitly listed HMM, find the probability of observations 3,1 by forward calculation and by enumerating all four hidden paths. [Authored worked example]** ### Answer #### Given π=(0.8,0.2); transition rows H=(0.7,0.3), C=(0.4,0.6); emissions H=(0.2,0.4,0.4), C=(0.5,0.4,0.1) for labels 1,2,3; no END factor. #### To find P(3,1), and an independent path-sum check. #### Formula and why it applies Forward reuses prefix totals. Exhaustive enumeration is practical for this two-position example and must give the same total. #### Solution ##### Step 1. Initialize observation 3 $$ \alpha_1(H)=0.8(0.4)=0.32,\quad\alpha_1(C)=0.2(0.1)=0.02. $$ ##### Step 2. Calculate the second H cell $$ \alpha_2(H)=[0.32(0.7)+0.02(0.4)](0.2)=0.0464. $$ ##### Step 3. Calculate the second C cell $$ \alpha_2(C)=[0.32(0.3)+0.02(0.6)](0.5)=0.054. $$ ##### Step 4. Sum final cells $$ P(3,1)=0.0464+0.054=0.1004. $$ ##### Step 5. Enumerate each complete path | Path | Full multiplication | Joint score | | --- | --- | --- | | HH | 0.8 × 0.4 × 0.7 × 0.2 | 0.0448 | | HC | 0.8 × 0.4 × 0.3 × 0.5 | 0.048 | | CH | 0.2 × 0.1 × 0.4 × 0.2 | 0.0016 | | CC | 0.2 × 0.1 × 0.6 × 0.5 | 0.006 | $$ 0.0448+0.048+0.0016+0.006=0.1004. $$ **Final answer.** Both methods give 0.1004. The largest single-path score is 0.048, which is smaller than the sum. For K states and T observations, both dense forward and Viterbi recurrences take O(TK²) work. Forward needs only the preceding column for the probability total. Viterbi also stores backpointers to recover the path. Very long sequences need scaling or log-sum-exp for forward; ordinary logarithms alone do not turn a sum into a maximum. **Yaad rakho:** Viterbi = best raasta. Forward = saare raaston ka total. This is a supporting HMM extension, not an assertion that a separate forward-algorithm question appeared in the supplied papers. This worked lesson was added after the video plan. No matching video has been generated or reviewed.