# NLP and MAD Viva Room Viva revision for 19 October, 10 AM. Start with foundations, then learn the project story and each lab. These are practice questions, not a prediction of the examiner. Use this answer route: Aim → method → reason → main steps → observed result → meaning/limitation. Examples are illustrative unless the lesson says a result was recorded. No software test count is a fake-news detection accuracy score. ## NLP — Start here · NLP from zero NLP stands for Natural Language Processing. It is the area of computing that works with human language: reading it, finding information, classifying it or predicting language. A computer does not automatically know what a sentence means. We choose a representation and a method that help it perform a specific task. ### Say it in the viva NLP helps computers process human language. A typical workflow is to collect text, prepare it, represent it numerically, apply a rule or learned model, and inspect the result. Different tasks need different information: sentiment needs clues such as not, named entity recognition benefits from capitalization, and evaluation requires examples that were not used to fit the model. ### Understand the concepts **Natural language:** Language people use to communicate, with ambiguity, context and exceptions. Example: 'That was sick' can mean illness or praise depending on context. **Text, document and corpus:** Text is written language. A document is one unit we analyse. A corpus is a collection of those units. Example: One review is a document; 2,000 reviews form a corpus. **Token:** A unit selected by a tokenizer: a word, punctuation mark, subword or another defined piece. Example: An exclamation mark can be its own token. **Type:** A distinct token form under the chosen normalization. Example: 'cat cat dog' has three token occurrences and two types. **Feature and vector:** A feature is an input property; a vector is an ordered numeric representation containing features. Example: Word counts, TF-IDF weights or embedding coordinates. **Label:** The category or target answer associated with an example. Example: positive, negative, PERSON or sci.space, depending on the task. **Model and parameter:** A model is a mathematical decision procedure; parameters are values it uses, often estimated during training. Example: A classifier learns weights connecting word features with classes. **Training and inference:** Training changes learned parameters using data; inference applies the resulting model to make a prediction. Example: Practising with answered questions versus answering a new question. **Classification:** Assigning an input to a defined category or categories. Example: Review → positive or negative. **Language modelling:** Assigning probabilities to language sequences, often by predicting the next token. Example: After 'a cup of', estimate probabilities for tea, coffee and other tokens. **Leakage:** Evaluation information influences training or selection in a way that makes the result less independent. Example: An exact copy of a test review is present in training. **Generalization:** How well a method works on new relevant examples beyond the ones used to fit it. Example: Correctly classifying a review written after training. **Overfitting:** Learning training-specific details that do not transfer well to new examples. Example: Near-perfect training accuracy with much poorer held-out performance. **Ground truth or reference label:** The label used as the evaluation reference, which should itself be checked for quality. Example: A human-annotated positive review or a defined exact-Jaccard threshold. **Baseline:** A simple comparison method that sets a reference level. Example: Always predict the majority class. ### What we did and why 1. **1. Start with a concrete task** — Suppose we want to sort movie reviews into positive and negative. 'The film was enjoyable' is one input document. Positive is its label. A collection of such reviews is a corpus. Why: The task determines what the output should mean. Detecting sentiment is different from finding names or checking truth. 2. **2. Break the text into usable pieces** — A character is a symbol; a token is a chosen text unit. Tokenization might turn 'Not good!' into ['Not','good','!']. Preprocessing can also normalize case or spacing. Why: Different tokenizers can produce different units. Removing not would destroy an important clue. Output/example: Illustrative: 'Not good!' has 9 characters including the space and punctuation, and this tokenizer gives 3 tokens. 3. **3. Turn selected information into numbers** — A feature is a measurable property supplied to a method. A vector is an ordered list of numbers. With vocabulary [good,bad,not], 'not good' has the illustrative count vector [1,0,1]. A matrix stores many document vectors as rows. Why: Column order must stay consistent. The first value always refers to good in this example. 4. **4. Distinguish rules, training and inference** — A rule-based system follows instructions written by a person. A learned model estimates parameters from examples. Training learns these parameters; inference uses the finished parameters on a new input. Why: Loading a pretrained model and predicting is inference; it does not mean we trained that model ourselves. Output/example: Illustrative: learn sentiment weights from labelled reviews, then classify a newly typed review. 5. **5. Keep learning separate from checking** — Training data fits the model. Validation data guides choices such as settings. Test data checks the final chosen approach. Vocabulary and IDF are learned components too, so fit them only on training text. Why: If the model sees the test answers or identical copies during learning, the evaluation no longer measures a genuinely separate challenge. 6. **6. Read a tiny confusion matrix** — Illustrative test: 10 reviews, 5 actually positive and 5 actually negative. Predictions give TP=4, FN=1, FP=2, TN=3. So 7 predictions are correct. Accuracy=7/10=70%; positive precision=4/(4+2)=66.7%; positive recall=4/(4+1)=80%; F1=2×4/(2×4+2+1)=72.7%. Why: Accuracy, precision and recall answer different questions. None is the confidence of every individual prediction. Output/example: Rows true, columns predicted, order [negative,positive]: [[3,2],[1,4]]. 7. **7. Explain a result without exaggerating** — State what was measured, on which examples, and what a mistake means. A cosine score is a similarity between vectors; accuracy is a fraction of correct labels; a regex match checks a pattern. These are different quantities. Why: An output looking plausible does not prove general correctness. A failed example can reveal an important limitation. ### Results and their meaning - **Illustrative tokenization: Not good! → Not | good | !** — Three tokens under the stated example tokenizer; not a saved lab measurement. - **Illustrative count vector: [good,bad,not] → [1,0,1]** — For 'not good'; feature order is fixed. - **Illustrative confusion matrix: [[3,2],[1,4]]** — Rows true, columns predicted; [negative,positive]. Ten examples total. - **Illustrative metrics: Accuracy70%; precision66.7%; recall80%; F1 72.7%** — TP4,FP2,FN1,TN3; all numbers are a teaching example. ### Viva questions **1. What is NLP in one sentence?** NLP is the area of computing that processes human language to perform tasks such as finding names, classifying text or predicting words. **2. What is the difference between a word and a token?** A word is a linguistic unit; a token is the unit chosen by the program. Tokens may include punctuation or parts of words. **3. What is a corpus?** A collection of texts used for analysis. One review is a document; a collection of reviews is a corpus. **4. What is a feature?** A piece of information represented numerically for a method, such as a word count or a TF-IDF weight. **5. What is a vector?** An ordered list of numbers. In text processing, each position represents a feature or learned coordinate. **6. What is a label?** The target category for an example, such as positive or negative in sentiment analysis. **7. What is a model?** A procedure that maps an input representation to an output using rules or learned parameters. I must explain how it was obtained and what task it performs. **8. How are training and inference different?** Training learns parameters from data. Inference applies already learned parameters to a new input. **9. Does loading a pretrained model mean I trained it?** No. Someone trained it earlier. Loading and calling it is using the pretrained model for inference unless I explicitly update its parameters. **10. How is classification different from language modelling?** Classification predicts a task label such as positive. Language modelling predicts token probabilities or scores a sequence of tokens. **11. Why preprocess text?** To make it suitable for a task and representation. But the steps must preserve the information the task needs. **12. Why should I not always remove stopwords and lowercase?** Words such as not carry sentiment, and capitalization helps recognize names. A fixed cleaning recipe can remove useful clues. **13. Why have train, validation and test sets?** Training fits parameters, validation helps choose settings, and the test set evaluates the final chosen method on separate examples. **14. What is data leakage?** It is when information from evaluation examples influences learning or selection, for example through copied texts across the split or fitting features on the full dataset. **15. What is accuracy?** The fraction of predictions that match the reference labels. In the tiny example, 7 correct out of 10 gives 70% accuracy. **16. What is precision?** For a chosen positive class, it is the fraction of predicted positives that are truly positive. Four correct positives out of six predicted positives gives 66.7%. **17. What is recall?** For a chosen positive class, it is the fraction of actual positives that were found. Four found out of five actual positives gives 80%. **18. What is F1?** The harmonic mean of precision and recall. It becomes low if either precision or recall is low. The tiny example gives about 72.7%. **19. Why can accuracy be misleading?** If one class dominates, always predicting it can give high accuracy while missing the other class. I should inspect class-wise precision, recall, F1 and the confusion matrix. **20. What should I say when a notebook output is wrong?** Describe the failure honestly, explain the likely cause from the method, and say how I would test an improvement. I should not claim the saved result is correct merely because the code ran. ### Check yourself A pretrained model predicts labels without changing its learned parameters. This is: - A. Inference - B. Training from scratch - C. Data annotation Answer: A. Inference applies an existing model. There are 5 actual positives and the model finds 4. Which score is 4/5? - A. Precision - B. Recall - C. Vocabulary size Answer: B. Recall measures how many actual positives were found. Which preparation choice is safest for sentiment meaning? - A. Always delete every common word - B. Preserve negation and choose cleaning based on the task - C. Always delete not Answer: B. Negation may change the sentiment, so commonness alone is not a reason to remove it. ### Remember - Input → prepare → represent → learn/apply → predict → evaluate. - Memory hook: corpus is the collection; token is the piece; vector is the numbers; label is the answer. - Training learns, inference uses, test checks. - Precision: trust the positive calls. Recall: find the actual positives. - If completely blank: say the task, input, representation, method, output and one limitation. ### Avoid these mistakes - A program running without errors does not mean its predictions are correct. - A similarity score is not automatically a probability. - A test accuracy is not every prediction's confidence. - A reference label is not infallible; evaluation depends on label quality and scope. - Preprocessing is a task decision, not a ritual list of deletions. ### Source/result notes Conceptual starting lesson derived from the inspected lab sequence. Tiny examples and metric numbers here are illustrative, not experimental results. - NLP_Lab0.docx - NLP_Lab1.docx - NLP_Lab2.docx - NLP_Lab9_SentimentAnalysis&Text Classification.docx - [NLTK: text classification concepts](https://www.nltk.org/book/ch06.html) - [NLTK: processing raw text](https://www.nltk.org/book/ch03.html) --- ## NLP — Lab 0 · Files, Python basics and regular expressions NLP means Natural Language Processing: making computers work with human language. Before a computer can analyse a review, it must read the text and identify useful pieces. Think of file handling as opening a notebook, and a regular expression as a carefully described search pattern. ### Say it in the viva In this lab I practised reading and writing text files and finding patterns with regular expressions. I read complete files, selected characters and lines, and used patterns to check beginnings, endings, spaces and email-like text. I also built a simple resume parser. It follows rules; it does not understand a resume like a person. ### Understand the concepts **Index:** A position number. Python starts at zero. Example: In 'The rain', the first space has index 3. **Token versus character:** A character is a single symbol. A token is a text unit chosen by a tokenizer, often a word or punctuation mark. Example: 'Hi!' has three characters but may be two tokens: 'Hi' and '!'. **Raw string:** The r prefix makes Python preserve most backslashes so a regex engine receives the intended pattern. Example: r'\d+' means a sequence of digits to the regex engine. **Character class:** Square brackets mean one character selected from the listed set or range. Example: [A-Za-z] matches one ASCII letter; [A-Z|a-z] also permits the literal | character. **Greedy matching:** A greedy quantifier consumes as much text as it can while allowing the full pattern to match. Example: In a line containing two @ signs, .+@ can extend to the last one; .+?@ prefers the first possible one. **Validation versus extraction:** Validation checks whether input follows a rule. Extraction returns selected information from it. Example: Check an email-like format versus extract the email field from a document. **Flags:** Flags change matching behaviour. IGNORECASE ignores letter case; MULTILINE lets ^ and $ work at line boundaries. Example: Combine flags with bitwise OR: re.IGNORECASE | re.MULTILINE. ### What we did and why 1. **1. Understand the basic objects** — A string stores text. A list stores ordered items, such as lines. A dictionary connects a key to a value, such as a field name to its extracted text. A function is a reusable operation; a module contains reusable functions. Why: Later NLP pipelines use strings as input, token lists in the middle and dictionaries for results. 2. **2. Read a file and understand its cursor** — The file cursor is the current reading position. read(5) consumes five characters; the next read starts after them. seek(0) moves back to the beginning. readline reads one line; readlines makes a list of remaining lines. Why: Using a with block closes the file even if an error occurs. Output/example: Saved notebook cell 6: read(5) returned 'Name:'. Cell 9 selected the fifth line using index 4. 3. **3. Write or append deliberately** — Mode r reads; w writes and replaces an existing file; a appends at the end; x creates only if the file does not exist. a+ allows reading and appending. Newlines are written as \n. Why: Rerunning an append cell can add the same content again. Output/example: Cells 11–12 saved ['Hello World\n', 'Hello Python'] after writing and reading the demonstration file. 4. **4. Describe a pattern** — In he..o, each dot matches one character except a newline by default. ^ checks the beginning; $ checks the end. [a-m] selects one character in a range. \s matches whitespace. search finds a match; findall collects matches; sub replaces matches. Why: A pattern describes allowed text structure rather than an exact sentence. Output/example: Cell 18: ['hello']. Cell 27: the first space began at index 3. Cell 29: 'The9rain9in9Spain'. 5. **5. Interpret the resume result** — The parser looked for specific field labels. Its Email pattern expected Email:, but the saved input used a different label. Several fields were not found, and the final output was 'Resume is Invalid'. Why: This means the input did not satisfy that parser's rules. It is not a judgement about the person's resume quality. ### Results and their meaning - **Read five characters: Name:** — Observed in cell 6; read(5) returns characters, not five words. - **First whitespace position: 3** — Observed in cell 27; Python positions are zero-based. - **Whitespace replacement: The9rain9in9Spain** — Observed in cell 29. - **Resume verdict: Resume is Invalid** — Saved rule-based result caused by missing or differently labelled fields; not a learned classification. ### Viva questions **1. What did you do in Lab 0?** I learned to read and write text files and to search text with regular expressions. These operations prepare raw text for later NLP tasks. **2. Why do we use file handling in NLP?** Text often comes from files rather than a typed Python string. File handling brings that text into the program and saves processed results. **3. What is the difference between read, readline and readlines?** read returns characters or the remaining file. readline returns the next line. readlines returns the remaining lines as a list. **4. Why did we use seek(0)?** Reading moves the file cursor forward. seek(0) returns to the start so the file can be read again. **5. What is a regular expression?** It is a pattern language for matching text structure. For example, a digit pattern can find numbers without knowing the exact number beforehand. **6. What do ^, $, dot and \d mean?** ^ marks the start, $ the end, dot matches one character except newline by default, and \d matches a digit. [0-9] is the explicit ASCII-digit range. **7. Is an email matching the regex definitely real?** No. The regex checks a selected format. It does not prove the mailbox exists or can receive mail. **8. Why was the resume marked invalid?** The saved input did not contain all the exact labels required by the parser. For example, the email label differed. I would normalize accepted labels or define clearer input rules. **9. Is this machine learning?** No. The rules were written manually. There is no training dataset or learned model. **10. If ma'am points out [A-Z|a-z], what should I say?** Inside square brackets, | is a literal allowed character, not an alternative operator. [A-Za-z] is the correct ASCII-letter class for that intention. The original format check can therefore accept an unintended character. ### Check yourself After reading a file completely, how do you read it again using the same handle? - A. Use seek(0) first - B. Call len(file) - C. Change every word to lowercase Answer: A. seek(0) resets the cursor to the beginning. A regex accepts an email string. What has been established? - A. The owner has read the message - B. The mailbox exists - C. The string matches the chosen format Answer: C. Format matching does not check delivery or identity. ### Remember - Read → find → extract → save. - File cursor = bookmark; seek(0) = move the bookmark to the start. - Regex checks shapes, not meaning. - Memory hook: ^ starts, $ stops, [] selects, + repeats. ### Avoid these mistakes - Do not say \w means only digits; it includes letters, digits and underscore in the usual interpretation. - Do not say a successful regex proves semantic correctness. - The notebook imports the third-party regex package as re; Python also has a built-in re module. - Avoid repeating personal resume fields during a demonstration; explain the pattern using neutral examples. ### Source/result notes Teacher handout and saved submitted notebook inspected. Outputs below are saved observations, not a new execution. - NLP_Lab0.docx - Submitted_Lab0.ipynb - [Python regular expression documentation](https://docs.python.org/3/library/re.html) --- ## NLP — Lab 1 · Text preprocessing A sentence is convenient for a person, but a program often needs smaller units. Preprocessing is like sorting ingredients before cooking: the preparation depends on the dish. A sentiment model needs 'not'; a name recognizer benefits from capital letters. There is no single cleaning recipe that is best for every task. ### Say it in the viva I applied common text preprocessing operations: case conversion, sentence and word tokenization, stopword and punctuation removal, stemming, lemmatization, POS tagging and frequency counting. I compared how these operations change the text. The main lesson is that preprocessing should match the task because removing a word or changing case can remove useful information. ### Understand the concepts **Corpus:** A collection of texts used for analysis. Example: A collection of customer reviews. **Tokenization:** Choosing boundaries between text units. Example: 'Good food!' can become ['Good', 'food', '!']. **Stopword:** A word often removed for a particular task because it is common, such as 'the'. It is not automatically meaningless. Example: Removing 'not' from 'not good' reverses an important clue. **Stemming:** Rule-based reduction that may produce a form that is not a dictionary word. Example: The saved Porter result for analysis is analysi. **Lemmatization:** Mapping a word to a dictionary base form, using available lexical and grammatical information. Example: computers → computer; a verb-aware analysis can map running → run. **POS:** Part of speech: the grammatical role of a token. Example: 'book' is a noun in 'read a book' and a verb in 'book a ticket'. **Normalization:** Making selected surface differences consistent. Example: Lowercasing merges NLP and nlp, but can hide the difference between a name and an ordinary word. ### What we did and why 1. **1. Split the paragraph** — Sentence tokenization finds sentence boundaries. Word tokenization splits sentences into words and often punctuation tokens. Why: Splitting makes it possible to count, tag and analyse smaller units. Output/example: Cell 5 split the saved paragraph into three sentences. 2. **2. Select useful tokens** — The notebook removes English stopwords and punctuation. It keeps content words such as Natural, Language, Processing and analysis. Why: This reduces common features in some bag-of-words tasks, but can remove sentiment or grammatical information. Output/example: Cell 8 contains 20 retained tokens. Lowercased frequency counting later gives 'language': 2 and 'nlp': 2. 3. **3. Compare stemming with lemmatization** — Stemming applies rough word-cutting rules. Lemmatization aims for a dictionary base form using lexical information; a POS-aware lemmatizer can distinguish word roles. Why: Similar surface forms can share a feature, but aggressive reduction can merge different meanings. Output/example: Cell 9: language → languag, analysis → analysi. Cell 10: computers → computer. 4. **4. Add grammatical labels** — POS tagging assigns grammatical roles such as noun, verb and adjective. In this notebook, tags are assigned after filtering. Why: Context helps tagging, so tagging the original sentence before deleting words is usually more defensible. Output/example: Cell 11 includes Natural/JJ, enables/VBZ and computers/NNS. 5. **5. Count and compare** — Counter counts repeated tokens. The manual tokenizer strips punctuation then splits on spaces, which is simple but does not handle every contraction or boundary correctly. Why: Frequency shows repetition, not necessarily importance or sentiment. Output/example: Cell 16: language and nlp occur twice; the cleaned example has 18 distinct lowercase tokens. ### Results and their meaning - **Sentence count: 3** — Saved cell 5. - **Filtered tokens: 20** — Saved cell 8; stopword and punctuation filtering. - **Word reduction: language → languag; computers → computer** — First is stemming in cell 9; second is noun lemmatization in cell 10. - **Top repeated terms: language: 2; nlp: 2** — Saved cell 16 after lowercase counting. ### Viva questions **1. What is preprocessing?** It is the set of steps used to prepare raw text for a task, such as tokenization and normalization. I choose the steps based on what information the task needs. **2. What is the difference between sentence and word tokenization?** Sentence tokenization separates sentences. Word tokenization separates smaller tokens, often including punctuation as separate tokens. **3. Why did you lowercase for counting?** It combines forms such as NLP and nlp into one count. I would preserve the original text if capitalization mattered, for example in named entity recognition. **4. What is the difference between a stem and a lemma?** A stem is produced by reduction rules and may not be a real word. A lemma is a dictionary base form. Lemmatization can use POS information to choose the correct base. **5. Which is always better: stemming or lemmatization?** Neither is always best. Stemming is simple and fast; lemmatization is usually more linguistically interpretable. The useful choice depends on the task and evaluation. **6. Why did 'enables' remain unchanged in your lemmatization?** The notebook used the default noun setting. It did not tell the lemmatizer that enables is a verb. Supplying the appropriate POS can change the result. **7. Should we remove all stopwords?** No. Common words can carry meaning. 'Not good' is very different from 'good', and deleting words before POS tagging removes grammatical context. **8. What do JJ, VBZ and NNS mean?** JJ means adjective, VBZ means third-person singular present-tense verb, and NNS means plural noun. They are fine-grained POS tags. **9. Why is split() not a complete tokenizer?** It mainly separates whitespace. It does not automatically make good decisions about punctuation, contractions, abbreviations or language-specific boundaries. **10. What did the frequency output tell you?** In the cleaned example, language and nlp appeared twice. That describes repetition in this small paragraph, not general importance across language. ### Check yourself Why can deleting 'not' be harmful? - A. It makes every sentence longer - B. It is always punctuation - C. It can reverse the sentiment clue Answer: C. Negation can change positive language into a negative statement. Which operation can return a non-dictionary form? - A. Sentence counting - B. Reading a file - C. Stemming Answer: C. A rule-based stem such as languag need not be a dictionary word. ### Remember - Split → normalize selectively → reduce word forms if needed → tag/count. - Stem = rough cut; lemma = dictionary form. - Memory hook: keep the meaning needed by the task. - If blank: 'Preprocessing prepares text, but the correct steps depend on the task.' ### Avoid these mistakes - Do not say preprocessing always removes punctuation or always lowercases text. - The default WordNet lemmatizer does not automatically infer the full grammatical context. - POS tags are predictions and can be less reliable after removing sentence structure. - A rough punctuation-removal rule can turn don't into dont rather than preserve its negation. ### Source/result notes Teacher handout and saved submitted notebook inspected. Saved examples are distinguished from suggested improvements. - NLP_Lab1.docx - Submitted_Lab1.ipynb - [NLTK: processing raw text](https://www.nltk.org/book/ch03.html) - [NLTK: grammatical tagging](https://www.nltk.org/book/ch05.html) --- ## NLP — Lab 2 · Turning words into numbers A classifier cannot directly calculate with the sentence 'great movie'. We represent it using numbers. Imagine assigning every word a column in a spreadsheet. One-hot encoding marks one word, bag of words counts the words in a document, and TF-IDF adjusts counts using how widely each word occurs. ### Say it in the viva I compared four text representations. Label encoding gives words identifiers; one-hot encoding gives each word its own position; bag of words counts vocabulary terms in a document; TF-IDF gives more weight to terms that are distinctive within a corpus. I then used TF-IDF features for four-class news classification and examined unseen words. ### Understand the concepts **Feature:** A numeric input used by a model. Example: The TF-IDF weight of the word orbit. **Vocabulary:** The set of terms assigned feature positions. Example: Three documents and 12 known terms give a 3 × 12 document matrix. **Sparse vector:** A vector with mostly zero entries. Sparse storage keeps mainly the nonzero values and their positions. Example: A short review uses only a small part of a 40,000-word vocabulary. **TF versus DF:** TF measures a word's use within one document. DF counts documents containing that word. Example: Ten repetitions in one document still contribute only one to DF. **IDF:** Inverse document frequency: a corpus-based weight that is lower for terms appearing in many documents. Example: A word occurring everywhere is less useful for distinguishing these documents. **L2 normalization:** Rescaling a nonzero vector so the square root of the sum of squared entries is 1. Example: It reduces the effect of simply having a longer document. **OOV:** Out of vocabulary: a term not present in the fitted vocabulary. Example: A new character name can disappear from a word-based representation. ### What we did and why 1. **1. Create a vocabulary** — The two introductory sentences produce 10 cleaned words. A sorted vocabulary gives a stable order to the numeric representation. Why: IDs identify words. Their numerical distance does not represent similarity. Output/example: Cells 7–9: manual IDs are 1-based; the sklearn LabelEncoder IDs are 0-based. The manual label of india is 6. 2. **2. Compare word vectors with document vectors** — A one-hot vector has one 1 at the word's vocabulary position. A bag-of-words document vector counts every vocabulary term. The three TV-review examples use 12 vocabulary terms. Why: Rows mean different things: word occurrences in the one-hot example, documents in the BoW example. Output/example: Cell 13: one-hot shape (10, 10), manual and sklearn outputs match. Cell 18: BoW shape (3, 12). 3. **3. Explain TF-IDF with one calculation** — The manual version uses TF = count/document length and IDF = ln(number of documents/document frequency). 'amazing' occurs once in an eight-token review and in one of three documents, so its weight is (1/8) × ln(3) = 0.1373. Why: Document frequency counts how many documents contain a term, not its total occurrences. Output/example: Cells 22–23: game has IDF 0 because it appears in all three documents; amazing has TF-IDF 0.1373. 4. **4. Understand why sklearn differs** — The default sklearn calculation uses raw term counts, smoothed IDF ln((1+N)/(1+df))+1, then L2 normalization. These choices differ from the manual formula. Why: Even with smooth_idf=False and norm=None, sklearn still adds 1 to IDF and uses raw counts by default. The notebook's statement that those options reproduce the manual numbers is incorrect. Output/example: Cells 26–27: amazing is 0.4687 in the first normalized document; each row has L2 norm 1. 5. **5. Train and interpret** — The local notebook uses graphics, hockey, space and Middle East politics from 20 Newsgroups. Vocabulary and IDF are fitted on training documents only. It compares Naive Bayes, Logistic Regression and a linear SVM. Why: These are saved results on this subset, not proof that one model is universally superior. Choosing a 'best' model on the test set should not become repeated test-set tuning. Output/example: Cells 31–35: 2,341 training and 1,558 test documents; 41,042 features. Saved accuracies: NB 91.66%, LR 91.21%, SVM 90.95%. 6. **6. Handle new words** — A fitted vocabulary stays fixed during transform. Unknown words are ignored by a word-based TF-IDF vectorizer. Why: Representing a new word by hashing does not mean the model has learned its meaning. Output/example: Cell 40: the all-unseen example has 0 nonzero word features; character n-grams give 1 nonzero feature, hashing gives 5. ### Results and their meaning - **Vocabulary examples: 10 introductory words; 12 review terms** — Saved cells 7 and 16. - **Manual TF-IDF: amazing = 0.1373; game = 0** — Saved cell 23, using normalized TF and ln(N/df). - **Classification feature matrix: train (2341, 41042); test (1558, 41042)** — Saved cell 32; 0.2373% of training entries are nonzero. - **Saved classifier accuracies: NB 91.66%; LR 91.21%; linear SVM 90.95%** — Saved cell 33, on the four selected historical categories. ### Viva questions **1. Why convert text to numbers?** The algorithms use mathematical operations, so we need a numeric representation that preserves information relevant to the task. **2. Why are word IDs not semantic scores?** The IDs are arbitrary. Word 10 is not twice the meaning of word 5. IDs are useful as labels or embedding lookup indices, but should not be treated as ordered measurements. **3. How do one-hot and bag of words differ?** One-hot represents one category or word using a single active position. Bag of words represents a document using counts over the vocabulary. **4. What does TF-IDF mean in plain English?** A word is useful if it appears in this document but is not equally common across all documents. TF measures within-document use and IDF adjusts for corpus-wide commonness. **5. Does TF-IDF preserve sentence order?** Ordinary unigram TF-IDF does not. Word bigrams add limited local order, but the whole sentence structure is still not represented. **6. Why do the manual and sklearn values differ?** They use different term-frequency scaling, IDF offsets and normalization. I should compare formulas and settings before expecting identical numbers. **7. If ma'am points out the claimed manual/sklearn equality, what should I say?** That sentence in the notebook is too strong. Turning off smoothing and normalization does not remove sklearn's +1 IDF offset or change raw counts into normalized TF. The displayed outputs themselves show they differ. **8. What does fit_transform do differently from transform?** fit_transform learns the vocabulary and IDF from training text and converts it. transform applies the already learned representation to new text without learning from it. **9. Why is the feature matrix so sparse?** Each document contains only a small fraction of all vocabulary terms. The saved training matrix has only 0.2373% nonzero entries. **10. What happens if every word is unknown?** The word-based vector can be all zeros. A classifier may still return a class from its learned parameters, but the input has supplied no recognised word features. **11. What did your model comparison establish?** On the saved four-class test subset, NB scored 91.66%, slightly higher than the other two models. It does not prove features always matter more than model choice, or that these scores apply to other datasets. ### Check yourself A word appears 20 times in only one of three documents. What is its document frequency? - A. 20 - B. 1 - C. 3 Answer: B. DF counts documents containing the term, not total repetitions. What should be fitted on test text? - A. The vocabulary - B. Nothing in the trained feature pipeline - C. The IDF weights Answer: B. Apply the training-fitted representation to the held-out test text. ### Remember - ID → identity. One-hot → one position. BoW → counts. TF-IDF → distinctive counts. - Memory hook: frequent here, uncommon everywhere. - Fit training text; transform test text. - The formula settings explain the numbers. ### Avoid these mistakes - TF-IDF is a feature representation, not a classifier by itself. - TF-IDF down-weighting is not always literal stopword removal: sklearn common terms can retain positive weight. - A high-dimensional one-hot or BoW representation is not automatically infeasible; sparse storage matters. - The top NB feature probabilities in the notebook are not necessarily the most discriminative features without comparison across classes. - Lab 2 uses hockey; Lab 9 uses baseball and additional cleaning, so their accuracies are not directly comparable. ### Source/result notes Teacher handout and own local Lab 2 notebook inspected. The downloaded Teams folder named Lab 2 contains a different corpus exercise; these observations come from the local vectorization notebook. Saved results were not rerun for this guide. - NLP_Lab2.docx - Local_Lab2_PartB.ipynb - [Scikit-learn TfidfVectorizer](https://scikit-learn.org/stable/modules/generated/sklearn.feature_extraction.text.TfidfVectorizer.html) --- ## NLP — Lab 3 · Exploring text corpora A corpus is a collection of texts. Exploring it is like checking the contents of a library before making conclusions: how many books, what categories, how long are they, and which words occur? A frequency count gives a summary; a concordance lets us see what a word actually means in its surrounding sentence. ### Say it in the viva I explored the Gutenberg and Brown corpora, downloaded plain text, extracted visible text from HTML, and analysed local documents. I counted characters, words and sentences, calculated type-token ratio, examined frequent words and collocations, and viewed concordance lines. These describe the corpus; they do not by themselves prove meaning, quality or sentiment. ### Understand the concepts **Type and token:** A token is an occurrence; a type is a distinct form under the chosen normalization. Example: 'Dog dog cat' has 3 tokens and 2 types after lowercasing. **TTR:** Type-token ratio, a simple measure of vocabulary variety within a text. Example: 2 distinct forms / 3 occurrences = 0.667. **Frequency distribution:** A table connecting each item to its occurrence count. Example: the → 6386 in the saved Brown news result. **Concordance:** A display of occurrences of a search word with nearby context. Example: Look at what comes before and after whale. **Collocation:** A word combination whose co-occurrence is notable under a chosen measure. Example: The saved Moby Dick output includes Sperm Whale and White Whale. **HTML extraction:** Separating visible text from page markup; menus and repeated navigation may still need cleaning. Example: A web page's raw source includes tags that are not article content. **Exploratory analysis:** Inspecting data before choosing methods or drawing conclusions. Example: Check lengths and genres before comparing vocabulary. ### What we did and why 1. **1. Choose a corpus and inspect its units** — Gutenberg contains literary texts; Brown contains texts grouped by genre. fileids lists documents. raw returns a text string; words returns tokens; sents returns lists of sentence tokens. Why: The object returned by each method determines what len is counting. Output/example: Cells 5 and 9: 18 Gutenberg texts; 500 Brown files in 15 genres. 2. **2. Summarise a text** — The notebook counts Emma's raw characters, token list and sentence list. Average characters per token uses those totals, so punctuation and whitespace conventions affect interpretation. Why: A corpus reader's token count is not necessarily a count of alphabetic English words only. Output/example: Cell 7: Emma has 887,071 raw characters, 192,427 tokens and 7,752 sentences; displayed average 4.61. 3. **3. Compare types and tokens** — Tokens count every occurrence. Types count distinct forms. Type-token ratio is distinct types divided by total tokens. Why: Longer texts often repeat more, so raw TTR is strongly affected by text length. Output/example: Cell 7: Emma TTR 0.0382. Illustrative example: 'cat cat dog' has 3 tokens, 2 types and TTR 2/3. 4. **4. Inspect frequency and context** — A frequency distribution counts occurrences. A concordance shows the same word with its neighbouring words. Collocations find word combinations that occur together in a noteworthy way. Why: Context helps distinguish different uses that a count alone mixes together. Output/example: Cell 11: Brown news has 100,554 tokens and 4,623 sentences; alphabetic counts include the 6,386 and of 2,861. Cell 15: whale concordance reports 1,080 matches. 5. **5. Extract and visualise external text** — The notebook extracts web text and PDF text, filters selected tokens and makes word clouds. A word cloud displays frequency through visual size. Why: Web pages change, PDF extraction can be imperfect, and repeated headers can affect counts. Output/example: Cell 21: the four local budget documents have 5,970; 7,853; 8,156; and 9,137 cleaned tokens. ### Results and their meaning - **Corpus sizes: Gutenberg: 18 texts; Brown: 500 files, 15 genres** — Saved cells 5 and 9. - **Emma: 192427 tokens; 7752 sentences; TTR 0.0382** — Saved cell 7, using that notebook's tokenization. - **Brown news: the: 6386; of: 2861; and: 2186** — Saved cell 11 alphabetic frequency results. - **Concordance: whale: 1080 matches** — Saved cell 15; five were displayed. ### Viva questions **1. What is a corpus?** A corpus is a collection of text used for language analysis. Gutenberg is mainly literary material, while Brown contains multiple genres. **2. What is the difference between raw(), words() and sents()?** raw gives a string, words gives a sequence of tokens, and sents gives a sequence of tokenised sentences. Their lengths count different units. **3. Why count unique words?** It helps describe vocabulary variety. I must define normalization first because 'Dog' and 'dog' can otherwise count as different types. **4. Does higher TTR mean better writing?** No. TTR depends on text length, genre, tokenization and normalization. It is a descriptive ratio, not a quality grade. **5. Why do very common words dominate a frequency list?** Function words such as the and of occur in many sentences. Filtering them may make topic words more visible, but the appropriate filtering depends on the question. **6. How is concordance useful?** It shows a word in context so I can inspect its use, ambiguity and surrounding phrases instead of relying on a total count. **7. What is the difference between a bigram and a collocation?** A bigram is any pair of adjacent tokens. A collocation is a noteworthy word combination selected using frequency or an association criterion; not every bigram qualifies. **8. What does a word cloud establish?** It visually summarises the frequencies supplied to it. It does not automatically establish topic importance, sentiment or factual truth. **9. Why do the two saved corpus notebooks have different counts?** They use different cleaning choices and external-text snapshots. For example, keeping only alphabetic tokens changes both the denominator and vocabulary. I should compare like with like instead of mixing totals. **10. If ma'am asks whether these web counts are fixed, what should I say?** They are the saved run's counts. Live pages and downloaded editions can change, and extraction methods also affect the result. ### Check yourself In 'cat cat dog', how many types are there? - A. 2 - B. 1 - C. 3 Answer: A. There are two distinct forms: cat and dog. Which tool is best for inspecting the surrounding words of every occurrence? - A. Concordance - B. File size - C. Character count Answer: A. A concordance displays a term in context. ### Remember - Count tells how much; concordance tells where and how used. - Types = unique; tokens = all occurrences. - Memory hook: library → books → sentences → words → context. - Always state what was counted and what was removed. ### Avoid these mistakes - A raw character count includes more than letters. - Do not compare TTR across very different text lengths as a fair vocabulary ranking. - Frequency is not the same as meaning or importance. - Do not merge the longer Task B notebook's filtered totals with the shorter notebook's unfiltered totals. ### Source/result notes Teacher handout and both saved corpus notebooks inspected. The numerical examples here consistently use the shorter submitted corpus notebook; the other version uses different cleaning and has different totals. - NLP_Lab3.docx - Submitted_Lab3.ipynb - [NLTK: accessing text corpora](https://www.nltk.org/book/ch02.html) - [NLTK: processing raw text](https://www.nltk.org/book/ch03.html) --- ## NLP — Lab 4 · N-grams and language models When someone says 'a cup of', you expect words such as tea or coffee. A language model assigns probabilities to possible next tokens. An n-gram model does this by counting short sequences rather than understanding the whole conversation. A bigram uses one previous token; a trigram uses two. ### Say it in the viva I built unigram, bigram and trigram models, calculated a transition-based sentence probability, predicted next words and compared held-out perplexity. Higher-order models use more context but have fewer examples for each context. In my saved run, trigram perplexity was worse because longer contexts were sparse and the simple smoothing method was limited. ### Understand the concepts **Conditional probability:** The probability of an event given information already known. Example: P(tea | cup of) concerns the next word after that context. **Markov assumption:** Approximate the next-token probability using only a limited recent history. Example: A trigram uses the previous two tokens instead of the entire paragraph. **Data sparsity:** Many possible combinations occur rarely or never in the available data. Example: An exact three-word phrase is harder to observe repeatedly than one word. **Smoothing:** Adjusting estimated probabilities so unseen events can receive nonzero probability. Example: Add one to every possible continuation's count. **Perplexity:** A measure derived from how much probability the model assigns to the actual next tokens. It is not a percentage accuracy. Example: Lower held-out perplexity means less average surprise under a matched evaluation. **Unknown token:** A special representation for words outside the training vocabulary. Example: Map an unseen name to using a consistent training/evaluation policy. ### What we did and why 1. **1. Form overlapping groups** — Unigrams contain one token, bigrams two and trigrams three. For an uninterrupted sequence of 100 tokens, there are 100 unigram positions, 99 bigram positions and 98 trigram positions. Why: Overlapping windows preserve limited local word order. Output/example: Cell 2 uses the first 100 alphabetic lowercase tokens from Emma. The group counts here follow directly from that code. 2. **2. Turn counts into conditional probabilities** — For a bigram model, P(next|previous) = count(previous,next)/count(previous as a context). In the four-sentence toy corpus, This is appears 3 times; is my once; my cat once out of two my contexts. Why: The code omits an initial-word probability and sentence-boundary probabilities. Describe it as a transition product conditional on the first word, not a complete sentence model. Output/example: Cell 3: for 'This is my cat', the transition product is 1 × 1/3 × 1/2 = 1/6 ≈ 0.1667. 3. **3. Pick the most frequent continuation** — The next-word function chooses the highest-count continuation of the supplied context. Why: The 's' result reflects possessive tokenization in the corpus, not a good human completion. Output/example: Cell 4: 'she had' → 'been'; 'Emma' → 's'. 4. **4. Smooth zero counts** — An unseen n-gram has zero unsmoothed probability and can make a sequence probability zero. Add-one smoothing adds one to counts and adds the vocabulary size to the denominator. Why: Smoothing reserves probability for unseen events but can spread too much mass across a large vocabulary. 5. **5. Read perplexity correctly** — Perplexity is exp(average negative log probability) on evaluation tokens. Lower is better when evaluated with the same setup. Logs avoid multiplying many tiny values directly. Why: The saved code lacks a complete unknown-word treatment and explicit sentence boundaries. These scores illustrate the implementation; they are not a robust benchmark comparison. Output/example: Cell 5: 153,941 training tokens and 38,486 test tokens. Unigram PP 443.06; bigram 891.60; trigram 3776.69. ### Results and their meaning - **Toy transition product: 0.1667** — Saved cell 3; 1 × 1/3 × 1/2. - **Next-word predictions: she had → been; Emma → s** — Saved cell 4; the second result exposes tokenization effects. - **Test perplexity: 443.06 / 891.60 / 3776.69** — Saved unigram / bigram / trigram values in cell 5; limited evaluation implementation. ### Viva questions **1. What is a language model?** It assigns probabilities to token sequences, often by predicting the next token from previous context. **2. What is the difference between a bigram and a trigram model?** A bigram predicts from one previous token. A trigram predicts from two. More context can help, but requires more examples of each context. **3. How did you get 0.1667?** The saved toy calculation multiplies P(is|this)=1, P(my|is)=1/3 and P(cat|my)=1/2. Their product is 1/6. **4. Is that the full probability of the sentence?** Not in the saved implementation. It does not include the first word or sentence boundaries, so I would call it the product of transitions after the given first word. **5. Why do unseen sequences cause problems?** Without smoothing their estimated probability is zero. A single zero makes the product for an entire sequence zero. **6. What is add-one smoothing?** It adds one to every possible event count. The denominator must also increase by the number of possible outcomes so the conditional probabilities remain normalized. **7. What does perplexity mean?** It measures average surprise at the actual test tokens. A model assigning those tokens higher probability has lower perplexity, when the evaluation conditions are the same. **8. Why is the trigram model worse in this notebook?** It has more specific contexts with fewer examples, and simple add-one smoothing over a large vocabulary can be poor. Higher order does not automatically mean better prediction. **9. Why did Emma predict s?** The tokenization leaves the possessive suffix as a separate token. The model follows that frequent pattern. This is a useful example of preprocessing affecting the output. **10. What would you improve?** I would define sentence boundaries and an unknown-word policy, keep preprocessing consistent, use a proper held-out comparison, and try interpolation or a stronger smoothing method. **11. Why use log probabilities?** Multiplying many small probabilities can underflow to zero on a computer. Summing their logarithms is numerically safer. ### Check yourself How much previous context does a trigram model use? - A. No previous tokens - B. Two previous tokens - C. Three previous tokens Answer: B. The trigram includes two context tokens and the token being predicted. Does a higher n always give lower test perplexity? - A. No, sparsity and estimation matter - B. Only because all probabilities are 1 - C. Yes Answer: A. Longer contexts require more data and suitable smoothing. ### Remember - Unigram: no previous word. Bigram: one. Trigram: two. - Count → conditional probability → prediction → held-out surprise. - Memory hook: more context, fewer examples. - Saved result: unigram had the lowest perplexity here. ### Avoid these mistakes - Perplexity is not an accuracy percentage. - Compare perplexity only with matched data, tokenization and vocabulary handling. - The next-word output can be grammatical-looking without being meaningful. - The notebook's first demonstration filters alphabetic tokens, while its perplexity cell uses the corpus token list more broadly. ### Source/result notes Teacher handout and saved submitted notebook inspected. The perplexities are recorded results with implementation limitations explained below. - NLP_Lab4_NGramModeling.docx - Submitted_Lab4.ipynb - [Stanford Speech and Language Processing, language-model chapters](https://web.stanford.edu/~jurafsky/slp3/index.html) --- ## NLP — Lab 5 · POS tagging and named entities In 'Apple bought a company', Apple plays a grammatical role and also names an organization. POS tagging answers 'what role does this word play in the sentence?' Named entity recognition answers 'which span names a person, place, organization, date or similar thing?' They are related but different tasks. ### Say it in the viva I used NLTK and spaCy for POS tagging and named entity recognition, then extracted simple features from airline reviews. POS labels describe grammar, while NER labels describe named items. The tools produced different labels on the example, and some were wrong. I also noticed that lowercasing and stopword removal can damage information needed for these tasks. ### Understand the concepts **POS tagging:** Assigning a grammatical category to each token. Example: A noun names something; a verb expresses an action or state. **NER:** Locating spans that refer to named items and assigning entity types. Example: New Delhi can be one location span even though it has two tokens. **Entity span:** The consecutive part of the original text selected as an entity. Example: United States of America should usually be handled as one named span. **GPE:** Geopolitical entity, such as a country, state or city. Example: India. **Chunking:** Grouping neighbouring tokens into larger units, such as phrases or entities. Example: A multi-token organization name. **Pipeline:** A sequence of processing components where later steps use earlier results. Example: Tokenization → grammatical features → entity recognition. **Information extraction:** Turning selected facts or mentions in free text into structured fields. Example: Organization: Apple; amount: $6 million. ### What we did and why 1. **1. Tag grammatical roles** — The notebook tokenizes a sample and applies POS tagging. Fine-grained labels distinguish noun types and verb forms. Why: Grammar helps later extraction, such as finding noun phrases or describing actions. Output/example: Cell 5 includes Apple/NNP, aiming/VBG, buy/VB and 15/CD. 2. **2. Recognize named spans** — NLTK ne_chunk groups tagged tokens into named-entity spans. It returns a tree with labels. Why: A model prediction is an output to inspect, not automatically ground truth. Output/example: Cell 6 labels India and United States as GPE, but also incorrectly labels Apple as GPE and Xolo INC as PERSON. 3. **3. Compare with spaCy** — spaCy's pipeline tokenizes and applies its trained components. pos_ gives a coarse POS category, tag_ a more detailed tag, and ents gives entity spans. Why: This example shows different behaviour; it does not establish that one library always has better accuracy. Output/example: Cell 9: Apple/ORG, Xolo INC/ORG, $6 million/MONEY, 15%/PERCENT. 4. **4. Extract review features** — The review CSV contains 3,616 rows. The notebook lowercases, tokenizes and removes stopwords, then counts tokens, distinct words and distinct bigrams. Why: Distinct bigrams use a set; their count is not always simply token count minus one. Output/example: Cells 13–15: the first review has 61 tokens, 54 unique words and 60 distinct bigrams. 5. **5. Explain the preprocessing problem** — The saved cleaning changes 'Not a reliable airline' into text beginning 'reliable airline'. It also removes capitalization. POS and NER are later applied only to the first 20 cleaned reviews. Why: For sentiment retain negation; for NER preserve case and sentence context. Feature preparation should be task-specific. Output/example: Cells 14 and 16 show the removed negation and the head(20) restriction. ### Results and their meaning - **NLTK sample errors: Apple → GPE; Xolo INC → PERSON** — Saved cell 6; these are incorrect labels in the example's intended reading. - **spaCy sample: Apple → ORG; $6 million → MONEY; 15% → PERCENT** — Saved cell 9. - **Review coverage: 3616 rows; POS/NER example restricted to 20** — Saved cells 13 and 16. - **First review features: 61 tokens, 54 unique words, 60 distinct bigrams** — Saved cell 15 after preprocessing. ### Viva questions **1. How is POS different from NER?** POS describes grammatical roles such as noun and verb. NER identifies named spans and labels them as person, organization, location and other entity types. **2. Can a proper noun and an entity be different?** Yes. POS is a token-level grammatical category. NER can cover multiple tokens, and dates or monetary expressions are entities without being ordinary proper nouns. **3. What did you observe when comparing NLTK and spaCy?** Their predictions differed. NLTK mislabelled Apple and Xolo INC in the saved example, while spaCy gave the intended organization labels there. One example is not a full accuracy comparison. **4. What does GPE mean?** A geopolitical entity, such as a country or city. It is distinct from a general place such as a river. **5. What do pos_ and tag_ mean in spaCy?** pos_ is the broad grammatical category, such as NOUN. tag_ is the more detailed tag, such as a plural noun tag. **6. Why can lowercasing hurt NER?** Capitalization is a useful clue for names. Removing it can make a name look like an ordinary word. **7. What was wrong with the review cleaning?** It removed not from a negative statement, leaving a positive-looking phrase. The cleaning therefore lost a key sentiment clue. **8. Did you run NER on all 3,616 reviews?** No. The saved notebook creates features for the dataset but applies the displayed POS and NER step to only the first 20 rows. **9. What is unique word count?** It counts distinct tokens after the chosen preprocessing, while word count includes every occurrence. **10. Does NER tell us whether a statement is true?** No. It identifies mentions and their types. It does not verify the claim or determine relations between all those mentions. **11. How would you evaluate NER properly?** Use separately annotated examples and compare predicted spans and labels with the annotations, reporting precision, recall and F1 rather than judging only one successful sentence. ### Check yourself Which question is answered by POS tagging? - A. Is this news true? - B. Which company owns this website? - C. What grammatical role does the token play? Answer: C. POS describes grammar. What is the main problem with deleting 'not' from 'not reliable'? - A. It guarantees higher accuracy - B. It creates a named entity - C. It may reverse the sentiment clue Answer: C. Negation must be considered in sentiment preprocessing. ### Remember - POS = grammatical job; NER = named thing. - Preserve the clues needed by the task. - Memory hook: 'Apple' can be NNP as grammar and ORG as an entity. - Inspect errors, not only successful labels. ### Avoid these mistakes - Do not claim the sample's automatic entity labels are all correct. - The airline dataset's original public source is not established by the saved local filename alone. - The displayed NER sample is 20 reviews, not the whole dataset. - Keeping only keywords before tagging can remove grammatical context. ### Source/result notes Teacher handout and saved submitted notebook inspected. Errors and the limited review sample are explicitly identified. - NLP_Lab5_InformationExtraction_POS_NER.docx - Submitted_Lab5.ipynb - [NLTK: grammatical tagging](https://www.nltk.org/book/ch05.html) - [NLTK: extracting information from text](https://www.nltk.org/book/ch07.html) --- ## NLP — Lab 6 · NER dataset and visual application NER turns text into labelled mentions: a person's name, a country, an organization or a region. This lab first looks at an already annotated dataset, then uses an existing spaCy model to highlight entities in a new paragraph. Looking at a labelled dataset and training a model on that dataset are different activities. ### Say it in the viva I explored a token-level NER dataset, counted its entity tags and made a dictionary assigning each lowercase word its most frequent tag. For the visual application, I used spaCy's pretrained English model and displaCy to highlight named entities. The dictionary is not used by that prediction function, so I did not train this spaCy model on the displayed dataset. ### Understand the concepts **Token-level annotation:** A label attached to each token rather than one label for the whole document. Example: New/B-geo and York/I-geo form one two-token place. **BIO encoding:** Beginning, Inside and Outside labels mark entity boundaries. Example: A B-per followed by I-per indicates a multi-token person mention. **Majority-tag lookup:** Assign each word the tag most often seen with it in the available examples. Example: A word used mostly as a location receives a location tag even in a different context. **Pretrained model:** A model whose parameters were learned previously and are reused. Example: Loading en_core_web_sm does not retrain it on the CSV. **NORP:** spaCy's category for nationalities and religious or political groups. Example: The saved example labels Iraqi and American as NORP. **GPE versus LOC:** GPE covers geopolitical entities such as countries or cities. LOC covers other geographic areas or features. Example: The saved labels are Iraq/GPE and Middle East/LOC. **Span:** A continuous range of characters or tokens representing one mention. Example: Two name tokens can form one PERSON span. **Visualization:** A display of existing results; it does not itself train or evaluate the predictor. Example: displaCy colours the entity spans supplied to it. ### What we did and why 1. **1. Inspect the dataset** — The CSV has Sentence #, Word, POS and Tag columns. Each row is a token, not a whole sentence. A sentence identifier appears at the start of a sentence; subsequent rows can have missing values. Why: A million token rows must not be described as a million documents or sentences. Output/example: Cell 2: 1,048,575 rows and four columns. 2. **2. Read BIO tags** — B- begins an entity, I- continues it, and O means outside an entity. Suffixes such as per, org and geo identify the type. Why: Most tokens are outside entities, so overall token accuracy can hide poor performance on the names we care about. Output/example: Cell 3: O = 887,908; B-geo = 37,644; B-tim = 20,333; B-org = 20,143. 3. **3. Build a majority-tag dictionary** — The code lowercases each word, groups its occurrences, and stores the most frequent tag. This is a simple context-free baseline representation. Why: The same word can have different meanings or entity roles in different contexts, which a single fixed tag cannot capture. Output/example: Cell 4 creates word_tags, but later code never uses this dictionary. 4. **4. Run the pretrained pipeline** — Cell 5 loads en_core_web_sm and passes the new text through it. The function keeps tokens whose ent_iob_ is not O and records their character positions and entity labels. Why: The snippet is a suggested span-level inspection, not the exact token-loop code in the submission. 5. **5. Explain the displayed output** — The submitted function sends each entity token as a separate manual span to displaCy. The display therefore splits multiword mentions into separate highlighted tokens. Why: Using full doc.ents spans would preserve multiword names such as Middle East as a single display unit. Output/example: Saved cell 6: Saddam and Hussein each PERSON; Iraqi and American each NORP; Iraq GPE; Middle and East each LOC. ### Results and their meaning - **Dataset size: 1048575 token rows × 4 columns** — Saved cell 2. - **Most common tag: O: 887908** — Saved cell 3; entity classes are much less frequent. - **Saved visual labels: PERSON, NORP, GPE and LOC** — Cell 6 HTML confirms the labels; some multiword mentions are displayed as separate tokens. - **Evaluation score: Not reported** — No held-out entity evaluation is present in the submitted six-cell notebook. ### Viva questions **1. What did this lab do?** It explored labelled NER data and displayed entities predicted by a pretrained spaCy model. It demonstrates information extraction and visualization. **2. What does the dataset row count represent?** It represents 1,048,575 token rows, not that many sentences. Each row has a word, POS tag and entity tag. **3. What do B, I and O mean?** B begins an entity, I continues an entity, and O is outside an entity. The type suffix says whether it is a person, organization, place or another category. **4. Why are there so many O labels?** Most ordinary words in text are not named entities. The saved dataset contains 887,908 O-tagged tokens, which makes class imbalance important. **5. What does word_tags contain?** It maps each lowercase word to its most frequent dataset tag. It ignores the surrounding sentence. **6. Did word_tags produce the highlighted predictions?** No. The ner function does not use that dictionary. It calls the separately pretrained spaCy model. **7. Did you train spaCy on this dataset?** No. Loading en_core_web_sm uses an already trained model. There is no spaCy training loop or update step in this submitted notebook. **8. Why are Saddam and Hussein highlighted separately?** The custom display code adds each token as an independent span. It should iterate over complete doc.ents if the goal is to preserve full multiword entity spans. **9. What is the difference between Iraq and Iraqi in the result?** Iraq is labelled GPE, a geopolitical entity. Iraqi is labelled NORP, a nationality or group category in this model. **10. What does manual=True mean in displaCy?** The code supplies the text and entity offsets directly to the visualizer. It does not mean the labels were manually annotated by a person. **11. What is the accuracy of this lab's NER system?** The notebook does not report a held-out NER evaluation. A highlighted example demonstrates output, not measured accuracy. **12. How would you improve and evaluate it?** Preserve complete entity spans, create a sentence-level train/test split if training a model, and compare predicted spans and types with held-out annotations using precision, recall and F1. ### Check yourself Which component actually predicts the displayed entities? - A. The CSV viewer - B. The word_tags dictionary - C. The pretrained spaCy pipeline Answer: C. The ner function calls nlp(text) and reads its token annotations. What should represent 'New York' when it is one named place? - A. Two unrelated document labels - B. A complete entity span - C. An accuracy percentage Answer: B. An entity can span multiple tokens. ### Remember - Dataset labels → inspect distribution → lookup baseline; separate pretrained model → displayed entities. - Memory hook: B begins, I inside, O outside. - A token is not a sentence; a display is not an evaluation. - If blank: 'I used a pretrained NER model; I did not train it on this CSV.' ### Avoid these mistakes - Do not say the CSV trained the spaCy predictor; it was inspected and used to build an unused lookup. - The manual token display fragments multiword entities. - An attractive visualization is not an accuracy result. - Do not present statements in the example paragraph as verified facts; the task only labels mentions. ### Source/result notes The actual six-cell submitted Kaggle notebook and its saved HTML output were inspected. A separate local rule-based notebook exists but is not the implementation described here. - NLP_Lab6_USeCaseofNER.docx - Submitted_Lab6_Kaggle.ipynb - [spaCy named entities and linguistic annotations](https://spacy.io/usage/linguistic-features#named-entities) --- ## NLP — Lab 7 · Word sense disambiguation The word 'bank' can mean a financial institution or the side of a river. The letters are the same, so the word alone is not enough. 'Deposit money' and 'river water' provide context. Word sense disambiguation, or WSD, chooses the intended sense of an ambiguous word in its sentence. ### Say it in the viva I treated WSD as supervised classification. The dataset provided a target word, a context sentence and a meaning. I created sense labels, converted the target and context to TF-IDF features, and trained Logistic Regression. The saved accuracy was 32.57%. The notebook also exposes limitations: duplicated rare-class examples before splitting and predictions that can belong to the wrong target word. ### Understand the concepts **Sense:** One particular meaning of a word. Example: bank as a financial institution versus bank as the edge of a river. **WSD:** Word Sense Disambiguation: selecting the intended sense from the surrounding context. Example: 'She deposited cash at the bank' supports the financial meaning. **Polysemy and homonymy:** Polysemy refers to related meanings of a form; homonymy to unrelated meanings sharing a form. The boundary can be debated. Example: paper as material and an article are related; bat as an animal and sporting equipment is a common homonym example. **WordNet:** A lexical resource that groups words into sets of related meanings called synsets and records relations between them. Example: A synset has a definition and can have example sentences. **Lesk method:** A knowledge-based WSD approach that compares words in the context with words in candidate sense descriptions. Example: A financial definition sharing words such as money with the context may receive more overlap. **Supervised classification:** Learning to assign labels from examples with known labels. Example: Learn from context sentences already tagged with their intended sense. **Data leakage:** Information from evaluation examples enters training or model selection, making evaluation less independent. Example: An identical copied sentence appears on both sides of a split. **Class imbalance:** Some labels have many examples while others have few. Example: A sense represented by only one example is difficult to learn and evaluate separately. ### What we did and why 1. **1. Identify the target and context** — The target is the ambiguous word whose meaning we want. The context is the sentence containing clues. A sense is a particular meaning assigned in the dataset. Why: A meaning inventory must be defined before predictions can be judged. Output/example: Cell 4 loads 4,064 rows and four original columns, then removes the serial-number column. 2. **2. Make labels** — The notebook numbers distinct meanings separately within each target word and creates labels such as bank_1 and bank_2. These are dataset-specific identifiers. Why: These IDs are not universal WordNet synset identifiers; numbering depends on the dataset ordering. Output/example: Cell 5 maps bank_1 to a financial institution and bank_2 to a riverbank meaning in the saved data. 3. **3. Represent and train** — Each input combines 'TARGET: ... CONTEXT: ...'. A TF-IDF vectorizer uses unigrams and bigrams, and Logistic Regression predicts one global class from all sense labels. Why: Adding the target as text does not enforce that the prediction belongs to that target word. Output/example: Cells 6–7: 3,316 training rows and 829 test rows after singleton-class duplication. 4. **4. Interpret the metric honestly** — The saved test accuracy is 32.57%, with warnings because some labels are never predicted. Before splitting, 81 singleton examples were copied, increasing 4,064 rows to 4,145. Why: Copying before splitting can put identical examples in train and test. These are not new independent samples, so this is not a clean generalization estimate. Output/example: Cell 9: accuracy 0.32569360675512665. 5. **5. Examine failures and a better design** — The bank example returns the intended financial sense, but bat, spring and shore examples return last_2. A better formulation restricts allowed senses to the requested word, or trains a classifier per ambiguous word. Why: A label for a different word is invalid for this WSD request even if the classifier can technically output it. Output/example: Cell 11: bank robbery → bank_1; the other three demonstrations → last_2. ### Results and their meaning - **Original dataset: 4064 rows** — Saved cell 4. - **After singleton duplication: 4145 rows; train 3316, test 829** — Saved cell 6; 81 copied examples create a leakage risk. - **Saved test accuracy: 32.57%** — Saved cell 9; not a clean generalization claim. - **Custom predictions: bank → bank_1; bat/spring/shore examples → last_2** — Saved cell 11; the last three illustrate invalid cross-word predictions. ### Viva questions **1. What is WSD?** It is choosing the intended meaning of an ambiguous word using its context. The same word can mean different things in different sentences. **2. How is WSD different from POS tagging?** POS chooses a grammatical role, while WSD chooses a meaning. Financial bank and river bank can both be nouns, so the same POS does not resolve the sense. **3. What is a synset?** A WordNet synset represents a particular sense through a set of synonymous words and a definition. It is more specific than a surface word alone. **4. Did you use Lesk in the actual trained experiment?** The notebook imports Lesk-related functions, but the executed classifier uses TF-IDF and Logistic Regression. I should not claim that the displayed results are Lesk results. **5. What is the idea behind Lesk?** Compare the context words with candidate sense descriptions and prefer the sense with more overlap. It uses a lexical resource rather than learning a classifier from labelled contexts. **6. What does bank_1 mean?** It is a locally created label for one meaning of bank in this dataset. The suffix is an identifier, not a probability or a universal sense number. **7. What did your accuracy mean?** The saved run matched the exact test label on about 32.57% of rows. Because of the split and duplication issue, I would not present it as a clean estimate of performance on new independent examples. **8. Why did you duplicate singleton classes?** The code did it to make stratified splitting possible for very rare classes. That is a weak workaround because copying before splitting can leak identical examples. A better dataset and split policy is needed. **9. Why did a bat example predict last_2?** The classifier chooses from a global set of sense labels without restricting them to the target word. Sparse data and that formulation allow an invalid cross-word prediction. **10. If ma'am points out those wrong outputs, what should I say?** Those are real failures in the saved implementation. They show why a global label classifier is not enough. I would constrain candidate senses by the target word and evaluate on a clean split. **11. How would you evaluate an improved version?** Keep duplicate contexts together, avoid copying before the split, use enough examples per sense, and report overall and per-word performance on held-out data. **12. Can removing stopwords hurt WSD?** Yes. Some short words and phrases carry useful contextual relationships. The impact should be checked rather than assuming all common words are useless. ### Check yourself Both meanings of bank are nouns. Which task distinguishes the meaning? - A. File writing - B. WSD - C. Counting characters Answer: B. POS alone cannot distinguish two noun senses. What is wrong with copying an example before train/test splitting? - A. It guarantees more independent data - B. The same example may appear in both sets - C. It changes all text into numbers Answer: B. Identical examples can leak across the split. ### Remember - Same word + different context = potentially different sense. - Knowledge-based: definitions. Supervised: labelled examples. - Memory hook: bank + money versus bank + river. - Saved limitation: copied examples and unrestricted cross-word labels. ### Avoid these mistakes - Imported code is not evidence that an algorithm was executed. - Do not describe 32.57% as high accuracy. - The duplicated singleton rows are not additional independent data. - A target token in the input is a clue, not an enforced restriction on output labels. ### Source/result notes Teacher handout and actual submitted notebook inspected. The saved run has important modelling and split limitations; these are explained rather than presented as successful general performance. - NLP_Lab7_WSD.docx - Submitted_Lab7.ipynb - [NLTK WordNet guide](https://www.nltk.org/howto/wordnet.html) - [NLTK WSD and Lesk API](https://www.nltk.org/api/nltk.wsd.html) --- ## NLP — Lab 8 · Word embeddings and document similarity Imagine placing words on a map so words used in similar contexts tend to lie nearby. An embedding is that numeric location in many dimensions. To compare whole sentences, this notebook averages their word vectors and compares the direction of the resulting vectors. Similar direction can suggest related content, but it does not prove copying. ### Say it in the viva I explored Word2Vec and pretrained GloVe embeddings. I represented each document by the average of its word vectors and used cosine similarity to compare documents. The Word2Vec model was trained on only three short examples, so it is a demonstration. The application displays cosine similarity as a plagiarism percentage, but that label is not a validated measure of plagiarism. ### Understand the concepts **Embedding:** A learned numeric vector used to represent a word or other item. Example: A 100-dimensional word vector has 100 numeric coordinates. **Dense versus sparse:** Dense vectors usually have many nonzero entries in relatively few dimensions. Sparse vocabulary vectors usually have many zeros. Example: 100 embedding values versus one feature per vocabulary term. **Distributional idea:** Words used in similar contexts often have related meanings. Example: tea and coffee may both occur after 'a cup of'. **CBOW and Skip-gram:** Two Word2Vec prediction directions: context to target, and target to context. Example: CBOW: 'a cup of ___' helps predict tea; Skip-gram starts from tea to predict neighbours. **GloVe:** Global Vectors: embeddings trained using aggregated word co-occurrence statistics. Example: A pretrained model has already learned from a much larger corpus than the lab's three sentences. **Cosine similarity:** A normalized measure of vector direction. For ordinary real nonzero vectors it lies between -1 and 1. Example: Same direction gives 1; perpendicular directions give 0. **Static embedding:** One vector per word form regardless of the sentence in which it occurs. Example: A single bank vector mixes financial and riverbank usage. **Pretrained model:** Parameters learned earlier on another dataset and reused. Example: Loading pretrained GloVe instead of learning all vectors from three examples. ### What we did and why 1. **1. Learn the representation** — Word2Vec learns vectors through a local prediction task. CBOW predicts a target word from nearby context; Skip-gram predicts nearby words from a target. Why: Similar contextual usage can lead to similar vectors. The 100 coordinates are learned dimensions, not 100 named human concepts. Output/example: Cell 4 uses vector_size=100, window=5, min_count=1 and sg=0, which selects CBOW. 2. **2. Make a document vector** — The code looks up vectors for recognized words and averages them. This turns a variable-length document into one fixed-length vector. Why: The average is simple, but discards word order and can blur negation and multiple topics. 3. **3. Compare with cosine similarity** — Cosine similarity is dot(a,b)/(norm(a)×norm(b)) for nonzero vectors. It compares direction rather than raw length. Why: These are saved similarities from a tiny three-sentence training corpus, not measured semantic accuracy. Output/example: Cell 4: document pairs 1–2 = 0.6104072; 1–3 = 0.5116333; 2–3 = 0.85622215. 4. **4. Use pretrained GloVe** — GloVe learns word vectors from global co-occurrence statistics. The notebook downloads glove-wiki-gigaword-100, then averages its vectors. Why: The displayed GloVe comparison numbers are not trustworthy evidence of those newly computed GloVe values. The document wording also changes, so the comparison is not controlled. Output/example: Cell 5 computes variables named similarity12/13/23 but prints sim12/13/23 instead. 5. **5. Interpret the application correctly** — The interactive example compares two related AI/environment texts. It prints cosine 0.9518, then multiplies by 100 and labels it plagiarism. Why: The supported statement is that the averaged vectors are highly similar. No calibrated model or human-labelled plagiarism evaluation establishes a 95.18% copying probability. Output/example: Cell 6: cosine 0.9518; displayed 'Plagiarism Percentage: 95.18%'. ### Results and their meaning - **Word2Vec settings: 100 dimensions; window 5; min_count 1; CBOW** — Saved cell 4; trained on three short documents. - **Word2Vec similarities: 0.6104 / 0.5116 / 0.8562** — Saved pairs 1–2 / 1–3 / 2–3. - **Interactive similarity: 0.9518** — Saved cell 6; the 95.18% display is this value ×100, not a validated plagiarism percentage. - **GloVe comparison: Displayed comparison cannot be relied upon** — Cell 5 computes similarity12 etc. but prints sim12 etc. ### Viva questions **1. Why use embeddings instead of one-hot vectors?** One-hot vectors give distinct words separate positions without representing similarity. Learned dense embeddings can encode patterns of contextual similarity in fewer dimensions. **2. What is Word2Vec learning?** It learns word vectors through predicting a word from its neighbours or neighbours from a word. It is not manually assigning meanings to each coordinate. **3. What is the difference between CBOW and Skip-gram?** CBOW uses context words to predict the target. Skip-gram uses the target to predict context words. The saved code uses CBOW because sg=0. **4. What does vector_size=100 mean?** Each word has 100 learned numeric coordinates. It does not mean 100 words or 100 directly named semantic features. **5. What does window=5 mean?** It sets the maximum context distance used around a target during training; implementation details can select shorter effective windows. **6. How do you turn word vectors into a document vector?** This notebook averages recognized word vectors. It gives a fixed-length representation but loses word order and can weaken important words such as not. **7. Why cosine similarity?** It compares vector direction after accounting for length. That is useful for comparing representations, but the interpretation still depends on how the vectors were created. **8. How does GloVe differ from Word2Vec?** Word2Vec uses a local context prediction objective. GloVe trains using global word co-occurrence statistics. Both can produce static word embeddings. **9. Is training Word2Vec on three sentences enough?** No, not for reliable general semantic representations. It demonstrates the process and API, but the corpus is far too small for a strong model. **10. If ma'am points out the GloVe print variables, what should I say?** The cell prints old variable names instead of the newly computed GloVe similarities. I should correct the variable names and rerun before comparing the numerical results. **11. Does 95.18% mean that much content was plagiarised?** No. It is cosine similarity multiplied by 100. It does not measure copied words or the probability of plagiarism; related original texts can also have high similarity. **12. What happens if all words are unknown?** The helper returns a zero vector when it has no recognized word vectors. That means insufficient representation, not proof that the document is unrelated. **13. Can static embeddings distinguish two meanings of bank?** They use one vector per word form, so they do not produce a different bank vector for each sentence. Contextual embeddings address that limitation. ### Check yourself What does CBOW predict? - A. A plagiarism verdict directly - B. Only the next document title - C. A target word from its context Answer: C. CBOW predicts a target using surrounding words. A cosine score is 0.95. Which statement is justified? - A. 95% of words were copied - B. The documents have identical meaning - C. The two vectors point in similar directions Answer: C. The score concerns the representation, not a calibrated copying claim. ### Remember - Word2Vec: local prediction. GloVe: global co-occurrence. - Word vectors → average → cosine. - Memory hook: same direction means similar representation, not proven copying. - CBOW = Context Builds One Word; Skip-gram = one word predicts neighbours. ### Avoid these mistakes - Cosine similarity is not a probability or a percentage of matching words. - The displayed GloVe comparison has stale-variable references; do not memorize those numbers as valid GloVe results. - The three-sentence Word2Vec experiment is too small to support accuracy claims. - Averaging loses word order and can make opposite statements look similar. - No explicit reproducibility controls cover all multithreaded Word2Vec variation in the saved code. ### Source/result notes Teacher handout and submitted notebook inspected. Saved Word2Vec values are reported; the GloVe comparison print bug and uncalibrated plagiarism label are explicitly flagged. - NLP_Exp_8_Word2Vec.docx - Submitted_Lab8.ipynb - [Gensim Word2Vec documentation](https://radimrehurek.com/gensim/models/word2vec.html) - [Stanford GloVe project](https://nlp.stanford.edu/projects/glove/) --- ## NLP — Lab 9 · Sentiment and text classification Sentiment classification asks whether a review is positive or negative. Topic classification asks what a text is about. Both use labelled examples to learn patterns. Think of training as practising with answers and testing as answering new questions without looking at the answer key. ### Say it in the viva I completed two supervised text-classification tasks using TF-IDF features. Task 1 classified movie reviews as positive or negative; Task 2 classified posts into four topics. I compared Multinomial Naive Bayes and Logistic Regression on held-out data, retained negation during cleaning and fitted features only on training text. Logistic Regression scored 86.75% on sentiment; Naive Bayes scored 89.54% on the selected topics. ### Understand the concepts **Supervised learning:** Learning a mapping from inputs to known labels using labelled examples. Example: Review text → positive or negative. **EDA:** Exploratory data analysis: inspect the dataset before modelling. Example: Check class counts and whether some reviews are empty or extremely long. **Train versus test:** Training data fits the model. Test data evaluates the finished setup on held-out examples. Example: Test answers are used for scoring, not for learning vocabulary or tuning repeatedly. **Stratified split:** A split that approximately preserves class proportions. Example: The sentiment test set contains 200 positive and 200 negative reviews. **Naive Bayes:** A probabilistic classifier that simplifies how features depend on each other given a class. Example: Positive-review words provide evidence for the positive class. **Logistic Regression:** A classification model that learns feature weights and converts combined scores into class probabilities. Example: Its name includes regression, but it can perform binary or multiclass classification. **Confusion matrix:** A table showing true classes against predicted classes. Example: 20 true negative reviews were predicted positive by sentiment LR. **Macro-F1:** Compute F1 separately for each class, then average equally across classes. Example: A small class matters as much as a large one in the macro average. **Baseline:** A simple reference method used to judge whether a model adds value. Example: Always choosing one class gives 50% accuracy on the balanced sentiment test set. ### What we did and why 1. **1. Inspect the data before training** — EDA means exploratory data analysis: inspect labels, class counts, empty text and length distributions. Movie reviews start with 2,000 examples; exact cleaned duplicates are removed. Topics use four selected 20 Newsgroups categories. Why: Missing or repeated examples and class imbalance can distort results. Output/example: Movie reviews: 1,997 retained, split into 1,597 train and 400 test. Topics: 2,261 train and 1,511 test after cleaning. 2. **2. Clean without deleting sentiment clues** — The cleaner normalizes Unicode, removes HTML/URLs/emails and simplifies spacing. It expands negative contractions into explicit not, then lowercases. It does not remove stopwords or stem words. Why: Deleting not can reverse a sentiment clue. Removing exact duplicates reduces one simple form of leakage; it does not guarantee all near-duplicates are absent. Output/example: The cleaner preserves explicit negation in can't, won't and isn't. Exact cleaned overlap between final training and test sets is zero. 3. **3. Split before fitting features** — Movie reviews use a stratified split with seed 81. Topics keep the official train/test separation. Each task fits its TF-IDF vocabulary and IDF only on training text, then transforms test text. Why: Learning test vocabulary or IDF during fitting would leak information about the evaluation set. 4. **4. Compare two algorithms** — Multinomial Naive Bayes uses class priors and feature likelihoods under a simplifying conditional-independence assumption. Logistic Regression learns weighted features to predict class probabilities. Both are efficient with sparse text features. Why: The winner changes by task. Model choice should be based on validation evidence, not repeated tuning against the test set. Output/example: Sentiment: NB 85.00%, LR 86.75%. Topics: NB 89.54%, LR 88.95%. 5. **5. Explain the metric and the errors** — Accuracy is the fraction correct. Precision asks whether predicted positives are right; recall asks whether actual positives were found. F1 balances them. Macro averaging gives each class equal importance. Why: Errors include mixed reviews, sarcasm and posts with overlapping topic vocabulary. A high total score does not mean every example is handled. Output/example: Sentiment LR confusion matrix, rows true and columns predicted [negative, positive]: [[180,20],[33,167]]. That is 347/400 correct. 6. **6. State the scope** — The datasets are historical English movie reviews and four historical discussion topics. The experiment tests these tasks and settings. Why: These scores do not establish performance on every current review, language, topic or news claim. Output/example: Sentiment macro-F1: NB 0.8498, LR 0.8674. Topic macro-F1: NB 0.8954, LR 0.8896. ### Results and their meaning - **Final splits: Sentiment 1597/400; topics 2261/1511** — Training/test counts after the saved cleaning and duplicate rules. - **Sentiment comparison: NB 85.00%; LR 86.75%** — 400 test reviews; LR correctly predicts 347. - **Topic comparison: NB 89.54%; LR 88.95%** — 1511 test posts; NB correctly predicts 1353. - **Sentiment LR confusion matrix: [[180,20],[33,167]]** — Rows true, columns predicted; class order negative, positive. ### Viva questions **1. What are the two tasks?** Binary sentiment classification for movie reviews and four-class topic classification for newsgroup posts. **2. What are the topic classes?** Computer graphics, baseball, space and Middle East politics. These are four selected categories, not all 20 Newsgroups categories. **3. What did EDA show?** I inspected balance, missing text, duplicates and lengths. The sentiment test set is balanced, and the topic posts have a broad length range with some very long documents. **4. Why keep negation?** Not good differs from good. The preprocessing expands negative contractions and keeps not instead of deleting it as a stopword. **5. Why use TF-IDF unigrams and bigrams?** Unigrams capture individual terms; bigrams capture short phrases such as not good. TF-IDF reduces the relative influence of terms common across many documents. **6. Why fit the vectorizer only on training text?** The vocabulary and IDF are learned information. Keeping test text out of fitting makes evaluation more independent. **7. What is naive about Naive Bayes?** It assumes features are conditionally independent given the class. Real words are related, but the approximation can still work well for text classification. **8. Why compare Logistic Regression?** It is a strong efficient linear classifier for sparse text features and provides a different learning assumption from Naive Bayes. **9. Which model performed better?** Logistic Regression was better on the saved sentiment test: 86.75% versus 85.00%. Naive Bayes was better on topics: 89.54% versus 88.95%. **10. Explain precision, recall and F1 in one example.** For positive reviews, precision asks how many predicted positive reviews truly are positive. Recall asks how many actual positive reviews we found. F1 is their harmonic mean, balancing the two. **11. How do you read the sentiment confusion matrix?** Rows are true labels and columns are predictions. LR correctly classified 180 negatives and 167 positives. It called 20 negatives positive and 33 positives negative. **12. Does 86.75% mean the model is 86.75% confident on every review?** No. It is the fraction correct across this test set. An individual predicted probability is a separate quantity and may need calibration. **13. What does removing duplicate leakage guarantee?** It removes exact repeated cleaned texts across the final split. It does not prove that paraphrases or all related examples are absent. **14. What would you improve next?** Study errors on a separate validation set, consider better representations and evaluate on a new independently collected dataset. I would keep the test set reserved for final evaluation. ### Check yourself Why should the test set not be used to learn IDF? - A. It would leak evaluation information into fitting - B. It makes documents unreadable - C. TF-IDF cannot transform test text Answer: A. IDF is learned corpus information. A model finds 80 of 100 actual positives. Which quantity is 80%? - A. Recall - B. Training size - C. Vocabulary coverage Answer: A. Recall = true positives / all actual positives. Which model had the highest saved sentiment accuracy? - A. Multinomial Naive Bayes - B. No classifier was trained - C. Logistic Regression Answer: C. LR scored 86.75%, compared with NB's 85.00%. ### Remember - Inspect → clean → split → fit features → train → predict → measure. - Precision: trust the positive predictions. Recall: find the actual positives. - Memory hook: train learns; test checks. - Remember two scores: sentiment LR 86.75%; topics NB 89.54%. ### Avoid these mistakes - Do not confuse class probability, similarity and test accuracy. - Do not claim that duplicate removal eliminates every possible form of leakage. - The models use TF-IDF and classical classifiers; no deep neural network was trained for this lab. - Historical four-class results are not universal news classification accuracy. ### Source/result notes Latest executed Lab 9 notebook and machine-readable results inspected. Counts and scores below come from the saved final run, not invented examples. - NLP_Lab9_SentimentAnalysis&Text Classification.docx - Final_Lab9.ipynb - Lab9_results.json - [Cornell movie-review polarity data](https://www.cs.cornell.edu/people/pabo/movie-review-data/) - [Scikit-learn 20 Newsgroups dataset](https://scikit-learn.org/stable/datasets/real_world.html#the-20-newsgroups-text-dataset) - [Scikit-learn TfidfVectorizer](https://scikit-learn.org/stable/modules/generated/sklearn.feature_extraction.text.TfidfVectorizer.html) --- ## NLP — Practicum · MinHash and locality-sensitive hashing Imagine comparing every pair of pages in a large folder to find copies. Most pairs are unrelated, so that wastes comparisons. This practicum makes compact fingerprints of word groups, uses them to shortlist likely similar pairs, then checks that shortlist exactly. It is like using an index to choose which books deserve a closer comparison. ### Say it in the viva I implemented near-duplicate text detection using word shingles, MinHash signatures and locality-sensitive hashing. MinHash estimates Jaccard similarity; LSH creates a shortlist of candidate pairs; exact Jaccard verifies the candidates. On 48 constructed documents, the default setting checked 72 instead of 1,128 pairs and recovered all 39 pairs meeting the chosen threshold. This measures lexical overlap, not semantic truth or plagiarism. ### Understand the concepts **Near-duplicate:** Two texts sharing enough surface content under a specified rule. Example: A copied paragraph with a few changed words. **Shingle:** A consecutive group of tokens used as a feature. Example: Three-word shingles preserve local order. **Jaccard similarity:** The fraction of distinct elements shared by two sets relative to everything in either set. Example: A={a,b,c}, B={b,c,d}: intersection 2, union 4, so Jaccard=0.5. **Hash function:** A deterministic mapping from an input to a numeric value. Example: The same shingle and hash function give the same value. Hashing here is not encryption. **MinHash signature:** A compact list of minimum hash values used to estimate set overlap. Example: A document's many shingles become 128 stored values. **LSH:** Locality-sensitive hashing: arrange hashes so similar objects are more likely to become candidates. Example: One matching band is enough to shortlist a document pair. **Candidate versus verified pair:** A candidate passes the approximate shortlist. A verified pair also passes exact Jaccard. Example: The default shortlist has 72 pairs, but only 39 satisfy the exact threshold. **Candidate recall:** The fraction of all exact-reference positive pairs that the shortlist includes. Example: 39 found out of 39 gives recall 1 on this corpus. **Mean absolute error:** Average absolute difference between an estimate and its exact reference. Example: Compare estimated MinHash similarity with exact Jaccard for each pair. ### What we did and why 1. **1. Build an understandable teaching corpus** — There are 12 topics, each with a base passage, an exact copy, a controlled word-edit version and a sentence-rotation version: 48 documents. All are original constructed examples, not a real-world benchmark. Why: Controlled changes help us see how the method behaves, but do not establish general performance. Output/example: Total possible pairs = 48 × 47 / 2 = 1,128. 2. **2. Represent local word order with shingles** — Normalize Unicode and case, then make sets of overlapping three-word sequences. Do not stem or remove stopwords. A set ignores repeated identical shingles. Why: Three-word shingles capture some local order while still permitting partial overlap. Output/example: Illustrative example: 'the small cat sleeps' → {'the small cat', 'small cat sleeps'}. 3. **3. Define the exact reference** — Jaccard(A,B) = size of intersection / size of union. The lab calls a pair a near-duplicate if exact shingle Jaccard is at least 0.70. Why: These are positives under the chosen lexical definition, not independent human labels proving plagiarism. Output/example: The all-pairs reference contains 39 positive pairs. 4. **4. Build compact MinHash signatures** — For each of 128 seeded hash functions, hash every shingle and keep the smallest value. The fraction of equal signature positions estimates set Jaccard. Why: The ideal MinHash equality result motivates the method. The implemented affine hash family is a practical approximation, not a claim of perfect random permutations. Output/example: Seed 81; 128 hashes; stable SHA-256-based shingle IDs; all-pair mean absolute estimation error 0.00574. 5. **5. Use LSH to shortlist candidates** — Split each 128-value signature into bands. A pair becomes a candidate if at least one complete band matches. The default uses 32 bands of 4 rows, and duplicate candidate pairs are stored only once. Why: More bands with fewer rows usually increase sensitivity and candidate volume; fewer bands with more rows can miss similar pairs. Output/example: Default: 72 candidates out of 1,128 possible pairs. 6. **6. Verify and compare trade-offs** — Calculate exact Jaccard only for candidates and retain those at least 0.70. Compare the retained pairs with the all-pairs reference. Why: Exact verification removes false candidates, but cannot recover a true pair LSH never shortlisted. Output/example: 32×4: 39 verified, 0 missed. 8×16: 37 verified, 2 missed. 4×32: 18 verified, 21 missed. 7. **7. Test the meaning limitation** — A negation change can leave most shingles unchanged, while a loosely worded paraphrase can share none. Why: Lexical similarity is different from matching meaning. The tool is a near-duplicate candidate method, not a fact checker or a calibrated plagiarism detector. Output/example: Saved challenge cases: negation variant Jaccard 0.7826; loose paraphrase 0.0000. ### Results and their meaning - **Corpus and exact reference: 48 documents; 1128 pairs; 39 positives** — Constructed teaching corpus; positives mean Jaccard ≥0.70. - **Default LSH 32×4: 72 candidates; 39 verified; 0 missed** — 93.62% fewer exact pair checks; candidate precision 39/72 ≈54.17%, candidate recall 100% on this corpus. - **Banding comparison: 4×32: 18/39 found; 8×16: 37/39; 16×8: 39/39** — Candidate counts are 18, 37 and 52 respectively; more permissive settings increase work. - **MinHash error: MAE 0.00574 overall; 0.01817 within families** — Actual absolute estimation errors on this corpus. - **Meaning limitation: Negation 0.7826; loose paraphrase 0.0000** — High lexical overlap can accompany changed meaning; similar meaning can have low overlap. ### Viva questions **1. What problem does the practicum solve?** It reduces the number of exact pair comparisons needed to find text pairs with high lexical overlap. It first shortlists likely matches, then verifies them. **2. What is new beyond basic Jaccard?** MinHash compresses sets into signatures, and LSH uses banding to generate candidates. Exact Jaccard alone still requires comparing every pair if no shortlist is used. **3. Why use three-word shingles?** They preserve some local word order. Individual words are less specific, while longer shingles become more sensitive to small edits. Three is a teaching choice, not a universal optimum. **4. How is Jaccard calculated?** Count shared distinct shingles, then divide by the number of distinct shingles in either document. The result lies between 0 and 1 for nonempty unions. **5. What does MinHash keep?** For each hash function it keeps the smallest hashed shingle value. Matching proportions across many signature positions estimate Jaccard similarity. **6. Why use a fixed seed?** It creates the same hash coefficients across runs so that comparisons are reproducible. It does not make the approximation exact. **7. Why not use Python's ordinary hash directly for persistent fingerprints?** Built-in string hashing can vary across processes. The lab uses stable hash-derived IDs so repeated runs can reproduce the same signatures. **8. What does 32×4 mean?** Thirty-two bands with four signature rows each. The product is 128, the signature length. **9. What is the candidate-probability formula?** Under the ideal independent-row model, if similarity is s, rows per band are r and bands are b, the candidate probability is 1−(1−s^r)^b. It describes a probability curve, not a hard guarantee. **10. Why do more bands usually create more candidates?** There are more chances for at least one band to match, and each smaller band requires fewer equal rows. This tends to improve recall but increases exact checking work. **11. What did the default result show?** It reduced exact pair checks from 1,128 to 72, a 93.62% reduction, while retaining all 39 threshold-positive pairs on this constructed corpus. **12. Is 93.62% a measured speed increase?** No. It is the reduction in the number of exact pair comparisons. Signature construction and hashing also take time, and this tiny corpus does not establish an overall speedup. **13. Why do exact verification after LSH?** Approximate candidates can be below the desired similarity threshold. Exact verification removes them. It cannot recover pairs that were never candidates. **14. Does a high score prove plagiarism?** No. It measures overlap under a shingle definition. Boilerplate, quotations or changed negation can have high overlap; a paraphrase can have low overlap. **15. How are empty texts handled?** Texts with fewer than three tokens produce no three-word shingles and are excluded from candidate generation. The lab defines empty-set Jaccard as 0 rather than treating empty inputs as duplicate evidence. ### Check yourself Why is exact verification still required after LSH? - A. Some candidates fall below the desired exact threshold - B. Every candidate is automatically a duplicate - C. It trains a sentiment classifier Answer: A. LSH is a shortlist, not the final threshold decision. How many rows are in a 32×4 signature? - A. 128 - B. 36 - C. 8 Answer: A. Bands × rows per band = 32 ×4 =128. A true pair is not shortlisted. Can exact verification recover it? - A. Yes, automatically - B. Only by changing its label - C. No, it never gets checked Answer: C. Verification operates only on candidate pairs. ### Remember - Shingles → MinHash signature → LSH candidates → exact Jaccard. - Memory hook: shortlist first, verify second. - Default numbers: 48 documents, 1128 possible pairs, 72 candidates, 39 verified. - Meaning and copying intent need more evidence than surface overlap. ### Avoid these mistakes - The 48 documents are constructed examples, not a public benchmark. - The 39 positives are defined by exact Jaccard ≥0.70, not by a human plagiarism study. - Candidate recall 1 in this run does not guarantee no misses on new corpora. - 93.62% fewer exact comparisons is not 93.62% faster runtime. - The overall MAE is dominated by many unrelated low-overlap pairs; within-family MAE is larger at 0.01817. ### Source/result notes Latest executed practicum notebook, source and results inspected. The corpus is constructed educational material, and all reported measurements are limited to that corpus. - Final_Practicum.ipynb - Practicum_results.json - near_duplicate_detection.py - [Mining of Massive Datasets, Chapter 3: Finding Similar Items](https://infolab.stanford.edu/~ullman/mmds/ch3n.pdf) --- ## NLP — TruthLens: explain the project from zero Imagine a message says, 'The Eiffel Tower is not located in Paris,' and includes a convincing photograph. The photograph alone cannot establish that statement. TruthLens separates the claim, finds reference material and compares what the reference says with the claim. If the evidence is insufficient, it says so. ### Say it in the viva Our project is TruthLens, a multimodal news-verification prototype. It accepts text and images. We use NLP to select factual claims, retrieve evidence and compare the meaning of a claim with a source passage using a pretrained DeBERTa NLI model. OCR reads text inside images, while CLIP provides image-caption similarity. The application shows source links and leaves a claim unresolved when the evidence is insufficient. ### Understand the concepts **Natural Language Processing (NLP):** Methods that help computers process human language, including its words, structure and meaning. Example: Splitting an article into sentences or comparing two statements. **Multimodal:** Using more than one type of input. Here the types, or modalities, are text and images. Example: A news caption plus its photograph. **Claim:** The statement being checked. Example: The observatory opened in 2004. **Evidence:** Reference information used to check a claim; its relevance, date and source matter. Example: A source passage describing when that same observatory opened. **Classifier:** A model that assigns an input to one of a defined set of classes. Example: The NLI classifier predicts entailment, contradiction or neutral for a pair of texts. **Prototype:** A working implementation used to demonstrate and study an idea, with known limits. Example: TruthLens works as a source-based assistant, but has not established reliable accuracy on all current news. ### What we did and why 1. **1. Understand the problem** — Misleading information can come from false words, an unrelated photograph or missing context. Checking only whether a photograph looks realistic does not check the story. Why: The same genuine photograph can be reused with an incorrect date or caption. 2. **2. Identify a claim** — A claim is a statement that can potentially be checked. 'The Eiffel Tower is located in Paris' is a factual claim. 'Paris is the most beautiful city' is an opinion and does not have one objective truth label. Output/example: Checkable statement → find evidence; personal preference → not the main scope. 3. **3. Find and interpret evidence** — Evidence is information used to assess the claim. The app retrieves Wikipedia passages and, when available, published fact-check reviews. It shows those sources so the reader can inspect the context. Why: A model prediction without a source would not explain why a news claim should be trusted. 4. **4. Keep the outputs separate** — Text support, image-origin patterns and image-caption similarity answer different questions. TruthLens presents them separately instead of averaging them into one invented truth percentage. Output/example: A supported claim does not certify the whole article or photograph. 5. **Two-minute presentation story** — Our project is TruthLens, a multimodal news-verification prototype. We chose this problem because an online post can mislead through its words, its image or the connection between them. A real photograph can still have a false caption. The user enters text, an image or both. If the image contains writing, OCR extracts it. We normalize the text while preserving names, numbers and negation, split it into sentences and select up to three candidate claims. The evidence service retrieves Wikipedia passages and published fact-check reviews. A pretrained DeBERTa NLI model compares each source passage with the claim. The application also checks whether the subject, relationship and details match. For images, CLIP measures image-caption similarity and an optional classifier gives an experimental image-origin signal. These findings stay separate because similarity and image origin do not establish news truth. Our contribution is the evidence workflow, the checks, the interface and the evaluation; we did not train the underlying models. The report records a bounded 450-pair experiment. The current version improves selected failure cases but still leaves many claims unresolved. The next step is independent current-news evaluation and stronger retrieval. The main learning was that meaningful context and clear uncertainty matter as much as a model prediction. Why: Say the short introduction first. Use this fuller story only when asked to explain the complete project. ### Results and their meaning - **Implemented input modes: Text, image, and text + image** — The available findings depend on which modules run successfully. - **Typical overall findings: Supported, potentially misleading, conflicting/mixed evidence, or not enough evidence** — Image-only analysis remains 'Image origin unverified' because the image signal is experimental. ### Viva questions **1. What is your project?** TruthLens is a multimodal news-verification prototype. It checks selected text claims against source passages and provides separate image findings. It explains the result with source links and can say that the evidence is insufficient. **2. What problem are you solving?** Online misinformation can combine misleading text with a genuine or generated image. Our aim is to help a reader examine the claim and its supporting context instead of trusting appearance alone. **3. What does multimodal mean in your project?** It means we accept both text and images. OCR connects image text to the claim-checking pipeline, and CLIP compares the image with a caption. The current system combines these components as a workflow; it is not a newly trained joint multimodal classifier. **4. Where exactly is NLP used?** NLP is used in text normalization, sentence segmentation, claim selection, search keywords and semantic comparison using NLI. Rules also check names, dates, quantities and negation. **5. What is your contribution if the models already exist?** The project contribution is the integration: selecting claims, retrieving and ranking passages, checking their relevance, combining model outputs with explicit rules and presenting explanations and sources. We also recorded component evaluations and investigated failure cases. Follow-up: Do not claim that you invented DeBERTa, CLIP or their training datasets. **6. Is this the same as an AI-generated text detector?** No. This project checks factual claims against evidence. Whether a person or an AI wrote a sentence does not decide whether its content is true. **7. Why can you not use only the image detector?** Image origin and news truth are different questions. A real photo can have a false caption, and a generated illustration can explain a true event. The image classifier also makes confident mistakes. **8. What should I say if I forget the introduction?** Say: 'We take a claim, find a source, compare their meanings and show the evidence. Images add OCR and visual information. When the evidence is weak, the application says it is uncertain.' **9. Why build this instead of using only keyword search?** Keyword search can find related passages, but it does not reliably handle negation, paraphrases or swapped subjects. We use it for retrieval, then add NLI and explicit checks to compare meaning. Keyword matching is a useful simpler alternative, but we did not report a formal keyword-only accuracy benchmark. ### Check yourself A real photograph is attached to a false caption. What follows? - A. Image authenticity alone cannot verify the claim - B. NLP is unnecessary - C. The whole news story must be true Answer: A. Text truth and image authenticity are separate questions. Which statement best describes the implemented project? - A. A detector of AI-written text - B. A newly trained universal fake-news classifier - C. A source-based workflow using pretrained models and explicit rules Answer: C. This is what the code and report actually implement. ### Remember - Memory hook: CLAIM → SOURCE → COMPARE → EXPLAIN. - Multimodal = text plus image. - The goal is evidence-based assistance, not a universal truth certificate. ### Avoid these mistakes - Do not call every output a final real/fake verdict. - Do not say the project detects who wrote text. - Do not describe the current application as a newly trained end-to-end multimodal model. ### Source/result notes Verified against the current v1.2.1 README and source, and the submitted report/presentation. This lesson describes the implemented prototype. - README.md - TruthLens_Mini_Project_Report.pdf - TruthLens_Mini_Project_Presentation.pptx - core.js - [TruthLens live demonstration](https://truthlens-dhruv.vercel.app/) - [FEVER paper: claims and evidence](https://aclanthology.org/N18-1074/) --- ## NLP — TruthLens: how a sentence becomes a finding Use one example throughout: 'The Eiffel Tower is not located in Paris.' First preserve the word 'not'. Next retrieve a passage about the Eiffel Tower. Finally compare that same location statement with the claim. The two sentences have many common words, but the negation reverses their meanings. ### Say it in the viva We normalize the input without removing meaningful words, split it into sentences and select up to three candidate claims using rules. We retrieve source passages and check whether they concern the same subject, relationship and details. NLI predicts support, contradiction or neutral for each passage-claim pair. Strong scores still have to pass the evidence rules before the application makes a provisional finding. ### Understand the concepts **Normalization:** Making equivalent text representations consistent while preserving relevant meaning. Example: Tidying repeated spaces without deleting the word 'not'. **Sentence segmentation:** Dividing a passage into sentence units. Example: Two complete sentences become two candidate units for analysis. **Heuristic:** A practical explicit rule that helps solve a task but is not guaranteed to work in every case. Example: Prioritizing a candidate sentence containing a number or a factual verb. **Information retrieval:** Finding relevant material from a collection or online source. Example: Looking up passages about the Eiffel Tower before comparing the claim. **Lexical versus semantic matching:** Lexical matching compares words; semantic matching concerns meaning. Shared words alone cannot resolve negation or swapped subjects. Example: 'Earth orbits the Sun' and 'The Sun orbits Earth' share words but reverse the relationship. **Premise and hypothesis:** In NLI, the premise is the given information and the hypothesis is the statement being tested against it. Example: Premise: 'A woman is walking a dog.' Hypothesis: 'A person is walking an animal.' **Entailment:** The premise supports the hypothesis. Example: Walking a dog supports walking an animal. **Contradiction:** The premise directly conflicts with the hypothesis. Example: 'The shop is open' conflicts with 'The shop is not open' in the same context. **Neutral:** The premise does not establish either the hypothesis or its opposite. Example: 'The school plans to buy computers' does not establish that it already bought them. **Abstention:** The application declines a decisive answer because its requirements were not met. This is different from the model's neutral class. Example: The model predicts support, but the evidence omits the date; the application stays unresolved. ### What we did and why 1. **1. Normalize without destroying meaning** — The app applies Unicode NFC normalization, removes control characters and tidies spaces. It preserves capitalization, punctuation, dates, numbers and negation in the text passed to the model. A separate lowercase keyword representation helps searching. Why: Removing 'not' changes 'does not cause harm' into 'does cause harm'. Treating every task like bag-of-words preprocessing would damage this comparison. Output/example: Meaning stays intact; harmless formatting differences are reduced. 2. **2. Segment and select claims** — Sentence segmentation finds sentence boundaries, mainly using JavaScript Intl.Segmenter. The rule-based selector ignores questions and very short fragments, merges repeated sentences and prioritizes candidate statements with factual wording or numbers. The application checks at most three selected claims. Why: The work is bounded so the browser does not run an unlimited number of comparisons. It is a heuristic, meaning a practical rule rather than a trained claim-extraction model. Output/example: The report records selection limits; it does not imply that every sentence was checked. 3. **3. Retrieve and rank complete passages** — Search keywords help retrieve Wikipedia introductions and published reviews through the evidence API. The app examines complete sentences and nearby sentence pairs, preferring passages about the same subject and relationship. It uses up to three selected passages per claim in the current UI. Why: A passage about the Moon orbiting Earth is on a related topic, but it does not establish the claim that Earth orbits the Sun. Adjacent qualifications must remain attached. 4. **4. Make the NLI pair** — NLI means Natural Language Inference. The source passage is the premise; the statement being checked is the hypothesis. The model sees both texts together and predicts their relationship. Output/example: Premise: 'The tower is in Paris.' Hypothesis: 'The tower is not in Paris.' Expected relationship: contradiction. 5. **5. Apply the decision rules** — A high score is accepted only when the passage is suitable. The rules consider subject, relationship, negation, quantities, units and dates. For example, '5 mg' and '5 kg' are not the same amount. Current-event claims need current verification, so a background passage alone cannot settle them. Why: The NLI model can be highly confident about an irrelevant or incomplete passage. Rules reduce selected mistakes but also reject some valid paraphrases. 6. **6. Explain uncertainty and source disagreement** — No decisive evidence leads to an unresolved result. Qualifying sources that oppose each other lead to a conflict. In v1.2.1, a dated, unqualified publisher rating may inform the assessment only for the same complete stable claim wording, with attribution. Related or media-dependent reviews do not automatically transfer their ratings. Output/example: A missing fact is not proof of falsehood. A publisher's rating is the publisher's assessment, not the model's own truth probability. ### Results and their meaning - **Claim selection: Up to 3 claims; over-700-character claims omitted intact** — The app reports omitted content rather than silently treating a fragment as a complete claim. - **Decision thresholds: Legacy route: score ≥0.90 and opposite-class margin ≥0.50; new bounded routes: score ≥0.95 and margin over other classes ≥0.80** — These are explicit engineering rules, not learned or calibrated real-world truth probabilities. Understand their purpose; memorizing the decimals is secondary. ### Viva questions **1. Why do you preserve stopwords and negation?** The full model input must preserve meaning. A short word such as 'not' can reverse a statement. We use a separate keyword representation for search, but we do not remove those words from the claim given to NLI. **2. Are you using a trained claim extractor?** No. Claim selection is heuristic. We split text into sentences, remove unsuitable fragments and use explicit priorities. This is transparent and lightweight, but it can miss important claims. **3. Why do you retrieve evidence before using the model?** NLI answers how a claim relates to a given passage. It needs that passage as context; it is not a live database of truth. Retrieval supplies material that the model and the user can compare with the claim. **4. Explain entailment, contradiction and neutral with one example.** If the passage says the observatory opened in 2004, 'It opened in 2004' is entailed and 'It did not open in 2004' is contradicted. 'It has a large telescope' is neutral because the passage gives no information about the telescope. **5. Why are keyword matches insufficient?** The same words can express different meanings. 'Earth orbits the Sun' and 'The Sun orbits Earth' reverse the roles. Negation, units and dates also matter, so we combine word checks with NLI and explicit rules. **6. Why does the app sometimes say uncertain for an obvious fact?** Its selected passage may omit a required detail or use wording that the conservative rules do not recognize. Uncertainty means this pipeline did not establish the claim; it does not mean the claim is false. **7. What changed in v1.2.1?** We improved complete-sentence passage selection and added bounded wording rules for cases such as orbit/revolve and active/passive authorship. We also improved exact publisher-review handling and made image-only uncertainty clearer. These were policy and presentation changes, not new model training. **8. What happens when Google Fact Check is unavailable?** Wikipedia retrieval can still provide background passages. If a provider or model fails, the app reports that failure and retains useful independent findings. It does not invent a score or treat a network failure as proof of fake news. ### Check yourself A passage says the Moon orbits Earth. Does it directly establish that Earth orbits the Sun? - A. It proves the Earth claim is false - B. Yes, because both mention orbiting - C. No, the subject and relationship differ Answer: C. A related topic is not necessarily relevant evidence for the same claim. The source does not mention the claimed date. What is the careful application response? - A. Always false - B. Unresolved unless suitable evidence is available - C. Always true Answer: B. Missing evidence is not a direct contradiction. ### Remember - Memory hook: CLEAN → SELECT → SEARCH → COMPARE → CHECK. - Premise = source; hypothesis = claim. - Entailment supports; contradiction opposes; neutral does not settle. - Not enough evidence means 'not established', not 'false'. ### Avoid these mistakes - Neutral is not the same as false. - An unresolved application result is not automatically a correct neutral prediction. - A related review cannot be copied onto a different claim. - Do not say that the app performs full syntactic parsing or learned named-entity recognition; its current context checks are rules. - The three-claim limit concerns selected claims, not a guarantee about the entire article. ### Source/result notes Verified in current core.js, evidence-policy.js, app.js and lib/evidence.mjs. Current code takes precedence over older architecture notes. - core.js - evidence-policy.js - app.js - evidence.mjs - RELEASE_1.2.1.md - [Google Fact Check claims search](https://developers.google.com/fact-check/tools/api/reference/rest/v1alpha1/claims/search) - [Wikipedia TextExtracts](https://www.mediawiki.org/wiki/Extension:TextExtracts) --- ## NLP — TruthLens: what the models actually do A model is a stored set of learned numerical parameters. Training changes those parameters using examples. Inference uses the learned parameters to produce an output for a new input. TruthLens mainly performs inference with models that other researchers already trained. ### Say it in the viva We use pretrained DeBERTa for the relationship between a source passage and a claim, Tesseract OCR to read image text, CLIP to compare image and text representations, and an experimental SMOGY image-origin classifier. The models run on the user's device through Transformers.js and ONNX Runtime Web. A small Node backend retrieves evidence and keeps the Google API key on the server. ### Understand the concepts **Training versus inference:** Training adjusts model parameters using examples. Inference uses fixed learned parameters to make a prediction. Example: Studying solved questions is like training; answering a new question is like inference. **Tokenization:** Breaking text into the model's vocabulary units and converting them to IDs. Example: An unfamiliar word may become several subword tokens. **Embedding:** A vector, or list of numbers, representing information useful for a model. Example: CLIP creates text and image vectors that can be compared in a shared representation space. **Attention:** A mechanism that lets a model weigh other tokens when forming a token's representation. Example: Surrounding words help distinguish 'bank' in a river sentence from 'bank' in a money sentence. **DeBERTa:** A transformer language-model family using disentangled content and relative-position attention. This project uses an existing small NLI checkpoint, not a model trained by the group. Example: It compares a complete source passage with a claim. **Softmax:** A function that transforms logits into non-negative relative class scores whose sum is one. Example: Three scores might be 0.94, 0.04 and 0.02; that does not certify 94% real-world truth. **OCR:** Optical Character Recognition: converting visible printed characters in an image into machine-readable text. Example: Reading a headline from a screenshot. **CLIP:** Contrastive Language–Image Pre-training: a model trained to connect image and text representations. Example: A dog photograph may be more similar to 'a dog outdoors' than 'an airplane'. **Cosine similarity:** The normalized dot product of two vectors, measuring how closely their directions align. Example: For nonzero vectors pointing in exactly the same direction it is 1. This is a geometric similarity value, not a truth percentage. **Quantization:** Representing model numbers with reduced precision to reduce model storage and potentially resource use. It can affect predictions. Example: The NLI and CLIP models use q8; the default image classifier uses q4. **API and Web Worker:** An API is a defined interface for requests and responses. A Web Worker executes browser work separately from the main page thread. Example: The browser requests evidence from /api/evidence while model work runs in a worker. ### What we did and why 1. **1. Learn the AI vocabulary** — AI is the broad field of building systems that perform tasks associated with intelligence. Machine learning learns patterns from data. Deep learning is machine learning with multi-layer neural networks. A transformer is one type of neural-network architecture, used here for language and visual representations. Why: You should know the relationship between these words instead of using them interchangeably. 2. **2. Understand pretrained versus fine-tuned** — A pretrained model has already learned parameters. Fine-tuning means further training it for a task or dataset. We load an existing task-trained NLI checkpoint and use it for inference; we did not fine-tune it for this deployed application. Output/example: The separate ml/train.py is a research workflow, not evidence that a custom model was trained or deployed. 3. **3. Follow the language model** — The tokenizer turns the passage and claim into tokens and token IDs. DeBERTa processes the pair using contextual representations and produces three raw class scores called logits. Softmax turns them into relative scores that sum to one. The code reads the model's label mapping instead of assuming the class order. Why: A token is not always a whole word, and a normalized model score is not the probability that a news story is true. Output/example: The complete passage plus claim is limited to 512 tokens; an oversized pair is rejected rather than silently cut. 4. **4. Follow OCR and CLIP** — OCR converts pixels containing letters into text. The user can review and correct that text before checking claims. CLIP turns an image and its caption into numerical vectors called embeddings, then compares their directions with cosine similarity. Why: OCR asks 'What text is visible?' CLIP asks 'How related are the image and text representations?' Neither proves when or where a photograph was taken. Output/example: CLIP accepts at most 77 tokens, including special tokens. The app rejects longer captions intact. 5. **5. Understand the image-origin signal** — The optional SMOGY classifier looks for patterns associated with generated images. It is skipped for user-marked screenshots or graphics. The observed classifier mistakes were strong enough that the app keeps image origin unverified and displays experimental scores separately. Why: Changing from q4 to full precision did not fix the observed errors. More numerical precision is not a guarantee of better task reliability. 6. **6. Explain where the work runs** — HTML, CSS and JavaScript provide the interface. A Web Worker runs inference separately from the page's main work so the interface can remain responsive. Transformers.js loads the models; ONNX Runtime Web executes them with WebAssembly. Node/Vercel functions retrieve reference information. No content database is used. Why: Image bytes stay in the browser, while selected claim text goes to the evidence service. Browser inference avoids a paid inference service but still needs downloads, memory and processing time. Output/example: Local start: Node 22 and npm start. First NLI use downloads roughly 170 MB; enabled visual modules can raise total downloads above 400 MB. ### Results and their meaning - **NLI checkpoint: onnx-community/deberta-small-long-nli** — Pinned checkpoint, q8, paired passage/claim input and three NLI labels. - **Visual components: Tesseract OCR + CLIP ViT-B/32 + experimental SMOGY** — They have separate roles and separate limitations. - **Deployment: Static frontend + Node functions on Vercel** — The actual model inference runs on the user's device. ### Viva questions **1. Did you train your own deep-learning model?** No. The deployed app uses existing pretrained, task-trained checkpoints for inference. Our work is the application pipeline, evidence rules, integration and evaluation. A separate training script is included for future research, but it is not the deployed model. **2. Why use DeBERTa?** We needed a compact model trained for three-way NLI, with browser-compatible weights. A small development comparison showed it reduced a specific high-confidence Earth/Moon error seen with the earlier model. That supports this choice for the prototype, not a claim that it is universally best. **3. What is a transformer or attention in simple words?** A transformer builds representations using the relationships between tokens. Attention lets it weigh relevant context, so a word can be represented differently in different sentences. That helps compare meaning beyond shared keywords. **4. What is the difference between OCR and NLP?** OCR reads letters from image pixels and produces text. NLP then processes that text and its meaning. In our project, OCR is the bridge from a screenshot to the claim-checking pipeline. **5. How does CLIP work here?** CLIP creates one vector for the caption and another for the image. We compute cosine similarity between them to indicate semantic alignment. The score cannot verify the photograph's authenticity, date or location. **6. Why not average NLI, CLIP and the image score?** Those scores represent different tasks and have not been calibrated into one common truth scale. Averaging them would create a misleading overall percentage. We show each result separately. **7. Why run the models in the browser?** It avoids a paid model-inference backend and keeps image processing on the device. The trade-offs are large first downloads and dependence on the user's browser memory and processing capacity. **8. What is your backend and where is the API key?** The backend uses Node serverless functions on Vercel for evidence retrieval. The Google Fact Check key is a server-side environment variable. It is not embedded in frontend JavaScript or included as a real key in the source ZIP. **9. Why choose pretrained models instead of training from scratch?** Training a strong language model from scratch needs substantial data and computing resources. Existing task-trained models let us study the NLP application within a student project's resources. We still need to evaluate their errors instead of assuming they work perfectly. ### Check yourself Which operation changes learned model parameters? - A. Training or fine-tuning - B. Inference - C. Displaying the result Answer: A. Inference uses existing parameters; training updates them. What does a CLIP cosine similarity of 0.7 mean? - A. The news is 70% true - B. A raw degree of alignment between the two embeddings - C. The photograph is 70% authentic Answer: B. No calibrated truth or authenticity probability follows from that scalar. ### Remember - Memory hook: NLI compares meanings; OCR reads letters; CLIP compares representations. - Training learns; inference uses what was learned. - Browser = models and images. Server = evidence lookup and secret key. - A model score is evidence about the model's prediction, not a certificate of truth. ### Avoid these mistakes - Pretrained does not mean the group trained it. - q4 and q8 describe reduced numerical precision, not four or eight accuracy classes. - An embedding is a numerical representation, not a human-readable explanation. - CLIP similarity is not an authenticity score. - The browser can cache model files, but full offline operation is not guaranteed. - There is no database containing verified copies of all news. ### Source/result notes Model roles, revisions, quantization, input limits and browser execution checked in current core.js, ml.worker.js and inference.js. - core.js - ml.worker.js - inference.js - app.js - package.json - README.md - THIRD_PARTY_NOTICES.md - [DeBERTa paper](https://arxiv.org/abs/2006.03654) - [Tasksource NLI checkpoint](https://huggingface.co/tasksource/deberta-small-long-nli) - [CLIP paper](https://proceedings.mlr.press/v139/radford21a.html) - [Transformers.js runtime](https://github.com/huggingface/transformers.js/tree/3.8.1) - [Tesseract.js OCR](https://github.com/naptha/tesseract.js) --- ## NLP — TruthLens: explain results honestly and handle follow-ups A result is useful only when you can say what was tested. '168 tests passed' concerns software behavior. '68.7% accuracy' concerns a particular 450-pair language-model experiment. Neither means that TruthLens detects 68.7% of all fake news. Keep the experiment, the application rules and the live demonstration separate. ### Say it in the viva We evaluated 450 fixed labelled claim–passage pairs, with 150 examples in each NLI class. The raw model achieved 68.7% diagnostic accuracy and 0.661 macro F1. The older frozen application policy made only 22 decisive predictions, with three errors, and left 428 cases unresolved. These results use supplied contexts and do not measure live retrieval or general current-news accuracy. The current v1.2.1 rules were assessed separately on inspected development cases. ### Understand the concepts **Accuracy:** The fraction of evaluated examples assigned the correct class. Example: In this experiment, 309 correct out of 450 gives 68.7%. **Precision:** For a selected class, how many predictions of that class were correct. Example: Learning example: if 8 of 10 predicted-support cases are truly support, support precision is 80%. **Recall:** For a selected class, how many actual members of that class were found. Example: Learning example: finding 8 of 20 actual support cases gives support recall of 40%. **F1 and macro F1:** F1 is the harmonic mean of precision and recall: 2PR/(P+R). Macro F1 averages each class's F1 equally. Example: It prevents the largest class from dominating the summary simply through its size. **Confusion matrix:** A table comparing actual classes with predicted classes; the diagonal contains correct predictions. Example: The 64 in the raw neutral row and contradiction column means 64 neutral examples were predicted as contradictions. **Decisive coverage:** The proportion of cases on which the application gave a support/refute answer. Example: 22 / 450 = 4.9%; the remaining cases were unresolved. **Selective error:** The error rate only among decisive outputs, which must be read together with coverage. Example: 3 / 22 = 13.6% errors among the frozen policy's decisive results. **Held-out evaluation and leakage:** Evaluation should use examples that did not influence training or design choices. Prior exposure can make the measured result too optimistic. Example: FEVER-related data appears in the upstream model's training sources, so this experiment does not establish performance on fully unseen model data. **CER and WER:** Character or word edit errors divided by the number of reference characters or words. Substitutions, insertions and deletions count as errors. Example: These measure transcription errors, not whether a news claim is true. **Regression test:** A check that an existing behavior still works after a code change. Example: Checking that a long caption is rejected without deleting its final negation. ### What we did and why 1. **1. Know what the dataset contains** — Each sample has a claim, a supplied context passage and a label. We used a semantically augmented FEVER NLI adaptation, selecting 150 examples per class before inference. Its named test split derives from original FEVER labelled development data; it is not the official blind FEVER test set. Why: The passage is already supplied, so the experiment tests inference with that context, not whether the live search finds the right evidence. 2. **2. Read the raw-model confusion matrix** — Rows are actual labels and columns are predicted labels, in the same order. Correct predictions are on the diagonal. Here 135 + 123 + 51 = 309 correct predictions out of 450, giving 68.7%. The neutral class was hardest, with 51 of 150 correct. Output/example: 309 / 450 × 100 = 68.7% for this recorded supplied-context experiment. 3. **3. Explain why the frozen policy scored lower** — The stricter policy often refused to decide even when the raw model had a strong prediction. It returned 21 support outputs and one refute output; 19 were correct and three were wrong. It left 428 cases unresolved. For the three-class comparison, unresolved outputs were placed in the neutral column. Why: This gives 37.6% diagnostic accuracy and 0.248 macro F1, with only 4.9% decisive coverage. Lower error among a small chosen set comes at the cost of leaving most cases unanswered. Abstention itself is not proof that the gold label was neutral. 4. **4. Distinguish the versions** — The submitted report and presentation record the frozen v1.2.0-era experiment. The current application/source is v1.2.1. On the same 18 inspected development cases, the revised pipeline changed from one supported result to five supported and two refuted results, leaving eleven uncertain. Why: Those cases informed the fixes, so they are not an independent held-out test. Seven decisive results out of eighteen must not be called seven-out-of-eighteen accuracy. The old 450-pair metrics do not measure the revised policy. 5. **5. Explain component results** — On 18 controlled OCR fixtures, 12 transcriptions matched exactly; character error rate was 12.8% and word error rate was 17.1%. A small 12-image convenience diagnostic found three top-label errors for both q4 and full precision. Confident image mistakes were also observed in the browser. Why: Controlled fixtures and small convenience sets reveal failure cases, but do not establish representative screenshot or image-authenticity accuracy. 6. **6. Demonstrate and conclude** — Show the input, the selected claim, the source passage and the explanation. Recorded v1.2.1 checks supported 'The Earth revolves around the Sun' and contradicted 'The Eiffel Tower is not located in Paris'. Also explain an uncertain example, such as the recorded Moon-orbit case. Live results can vary with source availability. Why: If a model or network stage fails during the demonstration, describe the failure and show the recorded result. Do not present an old screenshot as a new live run. Output/example: Conclusion: the prototype connects NLP, retrieval and model inference; it still needs broader independent evaluation, better retrieval and improved coverage. ### Results and their meaning - **Raw NLI, frozen experiment: 68.7% accuracy; 0.661 macro F1** — 450 supplied-context pairs, 150 per class; not live-retrieval or general news accuracy. - **Frozen application policy: 22 decisive; 19 correct; 3 errors; 428 unresolved** — 4.9% decisive coverage. Mapping unresolved to neutral produces 37.6% accuracy and 0.248 macro F1. - **Controlled OCR: 12/18 exact; CER 12.8%; WER 17.1%** — Controlled image fixtures, not representative screenshot accuracy. - **v1.2.1 development replay: 5 supported, 2 refuted, 11 uncertain** — 18 inspected cases used during fixes; not an independent benchmark. - **Recorded v1.2.1 software verification: 168/168 tests passed; 39 public assets matched** — Recorded release checks establish software/deployment consistency, not model accuracy. No new production run is claimed in this study note. ### Viva questions **1. What is your project's accuracy?** There is no validated overall current-news accuracy figure. In the recorded 450-pair supplied-context experiment, the raw NLI model had 68.7% accuracy. The frozen policy had 37.6% under the stated neutral-mapping comparison, and it answered decisively on only 22 cases. These are bounded historical results. **2. Why is the policy's accuracy lower than the raw model's?** The policy requires suitable evidence as well as a model score, so it abstains on many cases. It left 428 of 450 unresolved. That reduces decisive coverage and three-class accuracy; it is a real limitation, not a result we should hide. **3. Can I say the app is 86.4% accurate because 19 of 22 decisions were correct?** No. That percentage describes correctness only within the small subset that received a decisive answer. It excludes 428 unresolved cases and is not overall news accuracy. Report the 19 correct, three errors and 4.9% coverage together. **4. Was FEVER used to train your model?** We did not train the deployed model. We used a FEVER NLI adaptation for this evaluation. The upstream model already lists FEVER-related training data, so we cannot claim these examples were completely unseen during its earlier development. **5. What do the 168 passing tests establish?** They check software behavior such as decision rules, API handling and failure paths. They do not mean 168 news stories were correctly detected or establish an accuracy percentage. **6. What are the main limitations?** Retrieval can miss relevant context; rules can reject valid paraphrases; NLI and OCR can make mistakes; and the image classifier is unreliable on some inputs. English, browser resources and current-event coverage also limit the prototype. **7. What would you improve next?** First evaluate live retrieval and current claims on an independent labelled dataset. Then improve evidence coverage using a separate validation set, measure precision, recall and abstention, and evaluate the image components independently. Custom multimodal training would be a separate research step. **8. What did you learn from this project?** Preserving meaning matters as much as choosing a model. Good retrieval, correct context and clear uncertainty are essential. We also learned to distinguish a model score, a software test and a measured evaluation result rather than calling all three accuracy. **9. What are the main advantages of your approach?** It shows source passages, separates different kinds of evidence and explains why a claim remains unresolved. Browser inference also keeps image processing on the device. The trade-offs are download size, imperfect retrieval and conservative rules that miss valid cases. ### Check yourself The frozen policy answers only 22 of 450 cases. Which additional metric is essential? - A. Decisive coverage - B. Only the number of files - C. Only the download size Answer: A. An error rate among answered cases hides the many unresolved cases unless coverage is also reported. What does 168/168 software tests passing mean? - A. Every future network request will succeed - B. The checked software behaviors passed - C. The app has 100% fake-news accuracy Answer: B. Tests validate their stated cases, not general-world accuracy or future service availability. Which result describes the current v1.2.1 policy on an independent broad news benchmark? - A. No such result has been established - B. The 18 development cases - C. The old 450-pair policy accuracy Answer: A. The old policy experiment is historical and the small development set informed the changes. ### Remember - Memory hook: WHAT data? WHICH version? WHAT metric? WHAT limit? - Remember three numbers first: 450 pairs, raw 68.7%, policy 22 decisive with 3 errors. - The frozen policy left 428 unresolved; coverage is 4.9%. - Current v1.2.1 fixes improved inspected examples, not a new proven general accuracy. - A strong viva answer explains a limitation and the next experiment. ### Avoid these mistakes - Do not claim 100% accuracy, '168 correct detections' or '7/18 accuracy'. - Do not present the old frozen-policy numbers as measurements of v1.2.1. - Do not call the dataset's adaptation test split the official blind FEVER test set. - The pretrained model's possible prior FEVER exposure limits an unseen-data claim. - Software tests, component measurements and a live demonstration prove different things. - Do not promise that every simple factual statement will receive a decisive answer. ### Source/result notes Numbers verified against the frozen 450-pair summary, OCR records, v1.2.1 final replay and release-checks.json. Report/deck evaluation is historical and was not rerun for the revised v1.2.1 policy. - EVALUATION.md - summary.json - ocr-results.json - text-final-replay-summary.json - release-checks.json - RELEASE_1.2.1.md - TruthLens_v1.2.1_Update_for_Manan.txt - [FEVER dataset paper](https://aclanthology.org/N18-1074/) - [Supplied-context evaluation dataset](https://huggingface.co/datasets/tommasobonomo/sem_augmented_fever_nli) - [Fakeddit: possible future multimodal benchmark](https://aclanthology.org/2020.lrec-1.755/) --- ## MAD — Start here: how an Android app works Think of a restaurant counter. The screen is the menu and order form; an event is the customer pressing a button; Java is the staff following instructions; storage is the order book. Android calls the relevant code when something happens. ### Say it in the viva An Android app is event driven. XML defines the screen, Java handles actions, and the manifest declares components and permissions. When a user taps a button, its listener reads input, validates it, performs the operation and updates the screen. Persistent data belongs in files or a database, not only in screen variables. ### Understand the concepts **Activity:** An Android component that usually hosts a user-facing screen. Example: The calculator screen. **XML layout:** A description of views and their arrangement; it is not the calculation logic. Example: Two EditTexts, a Button and a result TextView. **Java / Kotlin:** Programming languages used to express behaviour; these native lab solutions use Java. Example: Read input, add values and update a label. **Listener / callback:** Code registered to run when an event occurs. Registering it does not mean the event has happened. Example: A doorbell rings first; then the listener answers. **Intent:** A message requesting an operation or carrying small data between components. Example: Open the result screen and pass a sum. **State:** The values that describe the current screen or application. Example: Selected date, current input, selected restaurant. **Asynchronous:** An operation finishes later and reports through a callback. Example: Do not announce a cloud save before its completion callback. **Main thread:** The thread responsible for screen drawing and interaction. Example: Long network work here can freeze the screen. ### What we did and why 1. **See the screen** — Android creates an Activity. In these labs, onCreate loads an XML layout using setContentView and connects controls to listeners. Why: A button drawn in XML cannot perform a calculation by itself. 2. **Follow one action** — Input → click event → validation → operation → output. For 2 + 3, the fields provide text, Java converts it to numbers, adds them and displays 5. Why: This same five-part flow explains almost every lab. 3. **Remember the lifecycle** — First opening: onCreate → onStart → onResume. Leaving: onPause → onStop. Returning from stopped: onRestart → onStart → onResume. Destruction may call onDestroy. Why: Android can recreate screens; saving only in an Activity field is temporary. 4. **Choose storage** — A variable is a temporary sticky note. Preferences are a few settings. A file is a document. SQLite is a structured local register. Firebase is an account-accessible cloud store. Why: Choose storage by the shape and lifetime of the data. ### Results and their meaning - **Example: 2 + 3 → 5** — The result follows a click callback, not merely displaying a Button. - **Lifecycle memory aid: Create → Start → Resume; Pause → Stop; Restart when returning** — Learn the purpose of each stage rather than only its spelling. ### Viva questions **1. If I forget the whole lab, how do I start?** Say: The aim was __. The user enters __. On __ event, we validate __, perform __ and display/store __. The main concept is __. **2. What is the difference between Java and XML?** XML describes what the screen contains and how it is arranged. Java describes what happens when the user interacts with it. **3. What is an event-driven application?** It waits for events such as taps, picker confirmations and network results, then executes the corresponding callback. **4. Is MainActivity.java Android's main() function?** No. The Android framework starts declared components and calls lifecycle methods. MainActivity is a conventional class name; the manifest identifies the launcher. **5. What is a View versus a ViewGroup?** A View is a UI element such as a Button. A ViewGroup contains and arranges child views, such as a LinearLayout. **6. Why validate before processing?** Input arrives as text and may be empty, malformed or outside the permitted range. Validation prevents meaningless work and explains the problem to the user. **7. What happens during rotation?** Android may recreate the Activity. Small screen state should be restored; durable records should already be in storage. **8. What is the difference between a crash and an ANR?** A crash ends execution because of an error. An ANR means the application is not responding, often because work blocks the main thread. **9. What does a successful build prove?** It proves compilation and packaging passed. It does not prove permission handling, real cloud acceptance or every device interaction. **10. What is the safest 20-second explanation pattern?** Aim → input → action → concept → output. Add one reason for the chosen component and one meaningful validation. ### Check yourself A visible button does nothing. What is most likely missing? - A. A cloud database - B. A click listener or valid event path - C. A camera permission Answer: B. Layout creates appearance; an event handler supplies behaviour. Which should survive process death? - A. Only an Activity field - B. A saved SQLite record - C. A local loop variable Answer: B. SQLite is persistent local storage. ### Remember - XML = appearance; Java = behaviour; manifest = declarations; Gradle = build. - Every lab: input → event → validation → operation → result. - Memory disappears; persistent storage has a longer lifetime. ### Avoid these mistakes - Do not say onCreate runs only once for the whole app: it runs for each created Activity instance. - onDestroy is not guaranteed before every process death; do not save critical data only there. - A screenshot shows a screen state, not all behaviour or persistence. ### Source/result notes Foundations checked against the course policy, submitted labs and current Android lifecycle documentation. - MAD Course Policy.pdf - MAD Viva Guide.docx - [Android activity lifecycle](https://developer.android.com/guide/components/activities/activity-lifecycle) --- ## MAD — Lab 1: Android Studio and emulator Android Studio is the workshop, the SDK is the toolbox, Gradle is the assembly process, and the emulator is a test phone on the computer. An AVD describes which virtual phone the emulator should run. ### Say it in the viva In this lab I studied Android Studio installation and emulator configuration. The SDK provides Android tools and platform APIs, an AVD describes a virtual test device, and Gradle builds the application. The main Java file handles screen behaviour, while activity_main.xml defines its layout. The purpose is to prepare and understand the development environment. ### Understand the concepts **IDE:** An integrated development environment combining editing, building, debugging and project tools. Example: Android Studio. **SDK:** Software Development Kit: Android APIs, tools and platform components. Example: The tools needed to compile and debug an Android app. **AVD:** Android Virtual Device: configuration of simulated hardware and software. Example: A chosen phone size with a particular Android system image. **Emulator:** The software that runs a simulated Android device using the AVD. Example: Test the app without a physical phone. **Gradle:** The build system that resolves dependencies and compiles/packages the app. Example: A failed dependency download can stop a build before launch. **Logcat:** A stream of Android logs used to inspect exceptions and events. Example: Find the error when an app closes immediately. ### What we did and why 1. **Prepare the environment** — Install Android Studio and the required SDK platform/build tools through the setup wizard or SDK Manager. Why: The editor alone cannot compile Android-specific code. 2. **Create a test device** — Use Device Manager to choose a hardware profile and Android system image, creating an AVD. Why: A device profile lets us test screen size and Android-version behaviour. 3. **Build and launch** — Create the app, allow Gradle to sync, select an emulator/device and run. Why: Build, install and launch are separate stages; identify the failing stage. 4. **Identify project files** — Java sources contain behaviour; res/layout contains screen XML; res/values contains named resources; AndroidManifest.xml declares components. Why: Knowing the right file is more useful than changing files randomly. ### Results and their meaning - **Expected outcome: A project opens and can be built/launched on a selected test target** — This is the setup objective; this study session did not reinstall the environment. ### Viva questions **1. Lab assignment: what is Android SDK?** It is the collection of Android platform APIs and development tools needed to build, test and debug an application. **2. Lab assignment: what is Android AVD?** It is the configuration of the virtual device: hardware profile, screen and Android system image. The emulator runs that configuration. **3. Lab assignment: what does MainActivity.java do?** It implements the main screen's behaviour and lifecycle callbacks in this project. Android launches it because of the manifest declaration, not its filename alone. **4. Lab assignment: what does activity_main.xml do?** It describes the main screen's UI structure. setContentView connects that resource to the Activity. **5. Why use an emulator?** It allows repeatable testing on different Android versions and screen configurations without owning every physical device. **6. Does an emulator replace all real-device testing?** No. Real hardware, performance, sensors and vendor behaviour can differ. **7. What is an APK?** An Android application package containing compiled code, resources and declarations that can be installed on a device. **8. What is Gradle sync versus Run?** Sync reads build configuration and resolves the project model/dependencies. Run builds if needed, installs and launches the app. **9. The app compiles but does not open. What should I check?** Check the selected device, installation result, launcher declaration and Logcat exception. Compilation alone does not prove launch success. **10. Which minimum hardware numbers should I memorize?** Understand RAM, disk and virtualization requirements; the old manual's Windows and Studio version figures are historical, not current installation requirements. ### Check yourself Which defines the virtual phone? - A. TextView - B. AVD - C. Intent Answer: B. AVD stores the virtual device configuration. Where would you inspect a crash? - A. colors.xml - B. Logcat - C. The image gallery Answer: B. Logcat shows exceptions and diagnostic events. ### Remember - Studio = IDE; SDK = tools; AVD = device profile; emulator = simulated phone. - Java behaviour + XML layout + manifest declaration + Gradle build. ### Avoid these mistakes - Do not confuse AVD configuration with the running emulator. - The manual contains older installation versions; do not present those as current system requirements. - Do not describe MainActivity as a manually called Java main function. ### Source/result notes Verified against the local submitted lab report; sample outputs below explain its documented behaviour, not a new device run. - MAD Lab 1.docx - MAD Course Policy.pdf --- ## MAD — Lab 2: calculator and Know Your Number A calculator form is like a small machine: input fields receive values, an operation button selects the action, and a result label shows the answer. The same flow supports factorial and even/odd checks. ### Say it in the viva This lab connects basic Java logic to an Android interface. I used input fields, buttons and result labels for arithmetic, factorial and even/odd operations. A click listener reads and validates the values, performs the selected calculation and displays the answer. The main learning is how views, events, conditions and loops work together. ### Understand the concepts **EditText / TextView:** EditText collects editable input; TextView displays text. Example: Enter 5 in EditText; show 120 in TextView. **Click listener:** A callback executed when a button is tapped. Example: Tap Factorial to start the calculation. **Factorial:** The product of positive integers from 1 to n, with 0! = 1. Example: 5! = 5 × 4 × 3 × 2 × 1 = 120. **Modulo:** The remainder after division. Example: 8 % 2 = 0, so 8 is even; 7 % 2 = 1, so 7 is odd. **ConstraintLayout:** A layout that positions views using relationships to other views or parent edges. Example: Attach a result label below the input instead of letting them overlap. **Resource:** A named reusable value such as a colour or string. Example: @color/error_red can be reused across fields. ### What we did and why 1. **Read input** — EditText gives text. Trim it, check it is present, then convert it to the required numeric type. Why: A numeric keyboard helps entry but is not complete validation. 2. **Choose the operation** — Calculator: add, subtract, multiply or divide. Factorial: multiply 1 through n. Even/odd: inspect n modulo 2. Why: Each button connects a visible action to a specific piece of logic. 3. **Handle invalid cases** — Reject division by zero; reject negative/noninteger factorial input; avoid overflow for large n. Why: A plausible-looking result can still be numerically wrong. 4. **Display feedback** — Update a TextView for a lasting result; use field errors for invalid input and Toast for short feedback. Why: Put feedback where it helps the user understand the action. ### Results and their meaning - **Worked example: 8 ÷ 2 = 4; 5! = 120; 7 is odd** — Illustrative calculations matching the required operations. - **Invalid case: Division by 0 → show an input error** — A useful app explains why the operation cannot proceed. ### Viva questions **1. What did this experiment demonstrate?** How to collect input, respond to button events and show the output of simple arithmetic or decision logic. **2. Why do we convert input text to a number?** EditText returns text. Numeric operations require a numeric value; otherwise text concatenation can produce 23 instead of 5. **3. Why should division by zero be checked?** It has no valid ordinary arithmetic result. Depending on numeric type, unchecked code may throw an exception or produce a nonfinite value. **4. How does factorial work?** Start with result 1 and multiply by every integer from 1 through n. For n = 5, the answer is 120; for n = 0, it remains 1. **5. How do you determine even or odd?** Divide by 2 and inspect the remainder using modulo. Remainder zero means even. **6. Lab assignment: how does android:onClick work?** The XML attribute names a public Activity method returning void and accepting one View parameter. Android resolves and invokes it on a tap. A programmatic listener is another approach. **7. Lab assignment: what is a Toast?** A short, temporary message shown without requiring the user to dismiss it. makeText creates it and show displays it; LENGTH_SHORT and LENGTH_LONG are duration choices. **8. Lab assignment: can we position a Toast anywhere?** Older examples use setGravity. For text toasts in apps targeting Android 11 or later, it has no effect; do not promise identical placement on current devices. **9. Lab assignment: where is colors.xml and why use it?** It is under res/values. Named colour resources can be referenced from XML or Java and updated centrally; values-night can provide dark-theme alternatives. **10. Why use hint rather than text for an input example?** A hint is guidance shown when the field is empty. A prefilled text label may accidentally be parsed as the user's numeric input. **11. What does a correct output establish?** It establishes that calculation for that input. Also test empty input, zero, negative values, division by zero and large factorials. **12. Why can the button work but the answer be invisible?** The TextView may be covered or incorrectly constrained. Check both logic and layout instead of assuming the calculation failed. ### Check yourself Why is 0! equal to 1 in this loop? - A. Android supplies it - B. It is set to 0 - C. The result begins at 1 and no multiplication occurs Answer: C. The empty product is 1, matching the factorial definition. Which is a persistent on-screen result? - A. A Logcat line - B. A TextView - C. A brief Toast Answer: B. The TextView remains until its UI state changes. ### Remember - Read → validate → convert → calculate → display. - 5! = 120; 0! = 1; n % 2 tests parity. - A Toast is temporary; TextView keeps a visible result. ### Avoid these mistakes - Integer division may truncate a fractional result. - Factorial grows quickly and can overflow fixed-size integer types. - Do not assume inputType prevents every invalid value. ### Source/result notes Verified against the local submitted lab report; sample outputs below explain its documented behaviour, not a new device run. - MAD Lab 2.docx - [Android Toast API](https://developer.android.com/reference/android/widget/Toast) --- ## MAD — Lab 3: explicit and implicit Intents An explicit Intent is like writing a friend's exact address. An implicit Intent is like saying 'I need a taxi': Android chooses an eligible handler for the requested action. ### Say it in the viva The lab demonstrates screen communication through Intents. One task calculates the harmonic series and displays the result on a second Activity. Other tasks open a browser, map or dialer and show student details in a ListView. Explicit Intents select a known screen; implicit Intents describe an action that another capable app can handle. ### Understand the concepts **Explicit Intent:** Names the exact component to start. Example: Open ResultActivity. **Implicit Intent:** Describes an action/data without choosing an exact component. Example: Open a website using a suitable browser. **Extras:** Small typed key-value values carried with an Intent. Example: The sender and receiver both use the key result. **Activity versus Fragment:** An Activity is an Android component; a Fragment is a modular UI/controller hosted by an Activity. Example: One Activity can host different reusable fragments. **Back stack:** A stack of Activities used for navigation. Example: Back usually closes the top Activity and reveals the previous one. **Activity result:** A destination returns a result code and optional data to its caller. Example: Edit a value and return it to the original screen. ### What we did and why 1. **Series input** — Accept a positive integer n for 1 + 1/2 + ... + 1/n. Why: The task is a harmonic sum, not 1 + 2 + ... + n. 2. **Compute and transfer** — Use floating-point division for each term. Put the calculated result or required input into Intent extras and start the result Activity. Why: Integer 1/i would discard fractions for i greater than 1. 3. **Delegate an action** — Use an implicit action with appropriate data, such as a web URL, geo URI or tel URI. Why: The operating system finds a suitable installed handler. 4. **Display details** — A welcome-screen button opens the details Activity, which presents student fields through a ListView. Why: It combines explicit navigation with a list-based presentation. ### Results and their meaning - **Series example: n = 3 → approximately 1.833333** — Three reciprocal terms are included. - **Delegated action: Dialer opens with a number** — The user still decides whether to make the call. ### Viva questions **1. What was the series in the task?** The harmonic series: 1 + 1/2 + 1/3 + ... + 1/n. **2. For n = 3, what should the result mean?** It is the sum of three reciprocal terms: 1 + 0.5 + about 0.333333 = about 1.833333. **3. Why use 1.0/i rather than integer 1/i?** The decimal operand preserves fractional terms. Integer division would make terms such as 1/2 equal zero. **4. Explicit versus implicit Intent?** Explicit names a specific component; implicit specifies an action and data for Android to resolve. **5. Why can an implicit Intent fail?** No installed application may handle the action/data, or restrictions may prevent it. Handle the failure instead of assuming a browser or camera is always present. **6. Does ACTION_DIAL immediately place a call?** No. It opens the dialer with the number. ACTION_CALL attempts a call directly and involves additional permission requirements. **7. Lab assignment: Activity versus Fragment?** An Activity is managed as a component and declared in the manifest. A Fragment is a reusable part hosted inside an Activity and has its own lifecycle. **8. Lab assignment: explain the seven lifecycle callbacks.** Create initializes; Start makes visible; Resume enables foreground interaction; Pause loses foreground; Stop is hidden; Restart precedes returning from stopped; Destroy cleans up the instance when called. **9. Lab assignment: can an Intent return data?** Yes. The caller launches using an Activity Result contract. The destination sets a result with extras and finishes; the caller reads the returned result in its callback. **10. Why must extra keys and types match?** The receiving screen retrieves data by key and type. A mismatch produces missing data or a default instead of the intended value. **11. Is Intent used to query a ContentProvider directly?** A provider is normally accessed through ContentResolver. An Intent can ask another component to select or act on data, but it is not a SQL-style provider query. **12. Why is onCreate not a one-time whole-app method?** It runs when that Activity instance is created, including a replacement after some configuration changes. ### Check yourself Which asks Android to find a browser? - A. DatePicker - B. Implicit Intent - C. SQLite Cursor Answer: B. The Intent describes an action and URL. What does integer 1/2 produce in Java? - A. 0 - B. 2 - C. 0.5 Answer: A. Integer arithmetic truncates the fraction. ### Remember - Explicit = exact destination; implicit = requested action. - Extras carry small data; result callbacks return it. - n = 3 → 1.833333... ### Avoid these mistakes - Do not confuse a harmonic sum with an arithmetic sum. - Do not say the app sends a call merely because the dialer opens. - Returning data requires a result path, not only starting a new Activity. ### Source/result notes Verified against the local submitted lab report; sample outputs below explain its documented behaviour, not a new device run. - MAD Lab 3.docx --- ## MAD — Lab 4: grocery ListView and date picker Think of a shop's dated catalogue. The catalogue stores the vegetables, the adapter prepares one display row for each item, and ListView places those rows on screen. ### Say it in the viva The app displays vegetable names and images for a selected date. A DatePickerDialog lets the user choose the date, GroceryCatalog returns the sample records, and SimpleAdapter binds each name and image to a ListView row. On confirmation the list refreshes; an unrecorded date shows an empty state. This demonstrates adapters, list views and event-based UI updates. ### Understand the concepts **Adapter:** The bridge converting data records into displayed views. Example: Data → adapter → ListView. **SimpleAdapter:** Binds named keys in maps to corresponding child view IDs. Example: name → vegetableName; image → vegetableImage. **ArrayAdapter:** Often displays objects as simple text rows; it can be customized. Example: A list of vegetable names. **Custom adapter:** Provides custom row binding and behaviour. Example: A row with an image, name, price and button. **notifyDataSetChanged:** Tells an attached adapter view to refresh from its current data. Example: It does not itself fetch records. **Empty state:** A clear display when no items exist for the selected conditions. Example: No availability recorded for this date. ### What we did and why 1. **Load a date** — The initial documented date is 17 September 2026. Ask the sample catalogue for its records. Why: The date is an input to the data lookup. 2. **Bind records** — Each map contains name and image. SimpleAdapter maps those keys to the TextView and ImageView in the row XML. Why: The ListView does not know how a vegetable object should look by itself. 3. **Confirm a change** — DatePickerDialog returns a selected date after OK. Replace the row data and notify the adapter. Why: Cancel must preserve the last confirmed date and its rows. 4. **Handle no record** — Clear the old rows and show the no-availability-record message for an unrecorded date. Why: Leaving yesterday's vegetables visible would misrepresent today's data. ### Results and their meaning - **17 September: 5 rows: Potato, Onion, Tomato, Cabbage, Cauliflower** — Documented sample catalogue. - **18 September: 3 rows: Potato, Tomato, Cabbage** — Changing the date changes the data. - **19 September: No availability recorded** — An explicit empty state avoids stale rows. ### Viva questions **1. Lab assignment: what is an adapter's role?** It reads a data item and supplies the row view the UI should display. The source stores values; the adapter binds; the ListView arranges and scrolls. **2. Lab assignment: which adapter types were discussed?** BaseAdapter, ArrayAdapter, SimpleAdapter and customized ArrayAdapter/SimpleAdapter implementations. Custom versions are extensions, not separate mandatory built-in class names. **3. Why was SimpleAdapter suitable?** Each vegetable row has fixed named fields: a name and image resource. These map directly to two XML child views. **4. What happens on a different confirmed date?** The catalogue is queried for that date, the backing collection is replaced and the adapter is notified. **5. Does notifyDataSetChanged query the catalogue?** No. The app must change the adapter's data first; notification asks the UI to rebind it. **6. What happens if the picker is cancelled?** The previous confirmed date and displayed list should remain unchanged. **7. Why does the month sometimes need plus one?** Calendar and DatePicker use zero-based months. Human-readable numeric months usually run 1 through 12. **8. What does no availability recorded mean?** The sample catalogue has no record for that date. It does not prove a real shop is sold out. **9. Why separate GroceryCatalog from MainActivity?** One answers the data lookup; the other manages user interaction. Each is easier to understand and check. **10. Is this the Foodie Order assignment?** No. Grocery List is Lab4; Foodie Order is a separate integration assignment even though one old document reused an Experiment4 heading. ### Check yourself After modifying rows, what tells the view to refresh? - A. requestPermissions - B. setResult - C. notifyDataSetChanged Answer: C. It signals that the adapter's underlying data changed. Which layer knows the vegetable name/image fields? - A. The adapter binding - B. Only the emulator - C. The system clock Answer: A. The adapter maps values into the row views. ### Remember - Data → SimpleAdapter → row XML → ListView. - Picker OK commits; Cancel keeps old state. - No record means clear the old rows. ### Avoid these mistakes - Update the adapter's actual collection, not a disconnected list. - Keep from-keys and to-view IDs in matching order. - Sample inventory is not live shop stock. ### Source/result notes Verified against the local submitted lab report; sample outputs below explain its documented behaviour, not a new device run. - MAD Lab 4 Grocery.docx - MAD Experiment 4.pdf --- ## MAD — Lab 5: WebView, dialogs and regex A WebView is a small browser surface inside your app. Static HTML is like a printed notice already in the app; a remote page must be obtained from a server. ### Say it in the viva This experiment uses WebView to display static HTML and remote website content. The user enters an address, the app validates it and asks for confirmation before loading. A regular expression helps check text patterns, while WebViewClient handles navigation and loading errors. The key distinction is that a valid-looking URL is not proof that a website is available. ### Understand the concepts **WebView:** A view that renders web content within the native app. Example: A restaurant website inside the application's screen. **HTML versus XML:** HTML describes a web page; Android layout XML describes native screen controls. Example: The XML contains a WebView that renders the HTML. **Regex:** A pattern for checking or finding text shapes. Example: A followed by exactly three digits accepts A123, not A12. **matches versus find:** matches checks the whole input; find searches for a matching portion. Example: Digits can be found inside a longer sentence. **AlertDialog:** A focused prompt with information or actions requiring a decision. Example: Open this website or cancel. **WebViewClient:** Receives page/navigation callbacks for a WebView. Example: Keep navigation inside the app and report loading failure. ### What we did and why 1. **Choose local or remote** — The starting Activity opens the static-content screen or the remote-input screen. Why: Separate screens make the two content sources clear. 2. **Check the address** — Trim input, reject empty values/whitespace, parse the URI and apply the lab's HTTPS/domain policy. Why: Avoid starting a request for an obviously invalid value. 3. **Confirm then load** — An AlertDialog provides Open, Cancel or Help. After confirmation call loadUrl; static HTML uses loadData. Why: Confirmation must occur before the navigation. 4. **Handle the result** — Show loading/success/error states. Keep appropriate links in the WebView and cancel certificate-error requests. Why: Network access can fail after input validation passes. ### Results and their meaning - **Valid example: A bare example.com becomes https://example.com** — Illustrative normalization; actual page availability is separate. - **Invalid example: example com → field error** — Embedded whitespace fails the lab's policy. ### Viva questions **1. What did static and remote mean in this lab?** Static HTML is bundled text rendered locally. Remote content is fetched from a URL over a network. **2. Does local HTML require internet?** Not if all its content is local. Remote images, scripts or links inside it can still require network access. **3. Lab assignment: what is a regular expression?** A formal text pattern used for validation and pattern matching. Character classes choose allowed characters; quantifiers choose repetition; anchors mark boundaries. **4. Lab assignment: matches versus find?** matches requires the entire input to follow the pattern; find locates matching subsequences within a longer input. **5. Why validate and still handle network errors?** Validation checks format and app policy. It cannot guarantee DNS, connectivity, certificates or server responses. **6. Why use HTTPS?** It protects transport and checks server identity through certificates. It does not guarantee that all content is truthful or harmless. **7. What does the confirmation dialog add?** It lets the user inspect the normalized address and explicitly continue or cancel before navigation starts. **8. WebView versus ACTION_VIEW?** WebView renders in our screen; ACTION_VIEW delegates to an eligible outside app such as a browser. **9. Does this validator accept every valid URL?** No. It intentionally restricts schemes, domains, credentials and ports for this lab. Rejection can reflect policy rather than universal URL invalidity. **10. What should happen after a certificate error?** Cancel the request and explain the failure. Bypassing certificate checks is not a solution. **11. Why might Back stay inside the web page?** The app can go back through WebView history first, and finish the Activity when no history remains. **12. What result should you explain in a viva?** Valid input reaches confirmation and then loading; invalid input gets a field error; an unavailable server gets a separate loading error. ### Check yourself Which can work fully offline? - A. Firebase login - B. Bundled HTML without remote resources - C. Every remote website Answer: B. The app already holds the required local content. A URL passes regex but fails to load. Is that contradictory? - A. Yes; HTTPS guarantees availability - B. No; format and network availability are different - C. Yes; regex guarantees a server Answer: B. Different checks answer different questions. ### Remember - Local content: loadData; remote content: loadUrl. - Validate → confirm → request → handle result. - Regex matches whole input; find searches within it. ### Avoid these mistakes - A URL-format check is not a security or truth guarantee. - Do not confuse Android XML layouts with web HTML. - A reference webpage image is not evidence of a new live fetch. ### Source/result notes Verified against the local submitted lab report; sample outputs below explain its documented behaviour, not a new device run. - MAD Lab 5 Corrected.docx - MAD Experiment 5.pdf --- ## MAD — Lab 6: employee/project records Imagine filling the employee half of a paper form, passing it to the project desk, and saving one combined record. Intent extras carry the first half; a private text file keeps the completed record. ### Say it in the viva The first Activity collects employee details and a date of birth. An explicit Intent sends them to the project Activity, which collects the project name and submission time. The combined record is saved and read as an internal text file. The main concepts are picker callbacks, data transfer between screens and persistent file handling. ### Understand the concepts **DatePickerDialog:** A calendar selection dialog with a confirmation callback. Example: OnDateSetListener receives year, zero-based month and day. **TimePickerDialog:** A clock selection dialog with a time callback. Example: OnTimeSetListener receives hourOfDay and minute. **Internal storage:** An app-private storage area that needs no broad external-storage permission. Example: The employee record belongs to the app's sandbox. **Encoding:** A rule converting characters into bytes and back. Example: UTF-8 handles names beyond plain English letters. **Overwrite versus append:** Overwrite replaces file content; append adds after existing content. Example: This lab saves one combined record, not an accumulating register. **try-with-resources:** A Java mechanism that closes streams automatically. Example: The file writer closes even if a write fails. ### What we did and why 1. **Collect employee data** — Read name and department; choose a date through DatePickerDialog. Reject empty fields and a future date of birth. Why: The picker supplies a date but does not automatically know the business rule. 2. **Transfer to project screen** — Put the employee values into extras and read the matching keys in ProjectActivity. Why: The second screen needs the exact values, not a global variable. 3. **Choose time and save** — TimePickerDialog returns hourOfDay and minute. Combine both forms and write UTF-8 to employee_project_record.txt. Why: A saved file retains the record beyond the current screen instance. 4. **Read safely** — Open the same private filename and show the content; explain when no file exists. Why: Reading before saving is a normal situation that should not crash. ### Results and their meaning - **Example: Employee + department + DOB + project + time → one labelled record** — The saved text combines both screens' input. - **Second save: Latest record replaces the previous one** — Documented overwrite behaviour. ### Viva questions **1. Lab assignment: why DatePickerDialog and TimePickerDialog?** They constrain input to structured date/time selections, use less screen space than full picker widgets, and return values through callbacks. **2. What callbacks receive the selections?** OnDateSetListener receives the date; OnTimeSetListener receives the time. **3. Why is a future date of birth invalid?** It contradicts the field's meaning. Set a maximum date and/or validate the confirmed value. **4. What is hourOfDay?** The hour in a 24-hour range from 0 to 23. Display formatting may convert it to an AM/PM presentation. **5. What does the final time-picker boolean control?** Whether the picker displays a 24-hour clock. It does not change the returned hourOfDay range. **6. Why use Intent extras?** They explicitly carry the employee values to the second Activity. The keys and types must match on both sides. **7. Does this file require storage permission?** No. App-private internal files are accessed within the application's sandbox. **8. What happens when Save is pressed twice?** The supplied overwrite flow replaces the previous content with the latest combined record. It does not create two history entries. **9. Why use OutputStreamWriter and BufferedReader?** The writer converts text to bytes using an encoding; the reader reads text efficiently. Both should be closed after use. **10. What does trim prevent?** A field containing only spaces from being accepted as a meaningful name or project. **11. What if the file has never been saved?** Show a readable no-saved-record message instead of allowing a missing-file exception to crash the screen. **12. How would you support many searchable employee records?** Use a structured database with identifiers and queries. A single overwritten file is appropriate only for a simple record exercise. ### Check yourself Which requires broad storage permission here? - A. Neither private save nor private read - B. Every file write - C. Reading a local TextView Answer: A. The file is in the app's private internal storage. What should happen on Read before Save? - A. A helpful missing-record message - B. A crash - C. A fabricated record Answer: A. Handle the expected absence of a file. ### Remember - Screen1 → extras → Screen2 → UTF-8 private file. - Pickers return values; application validation gives them meaning. - One overwrite file is not a multi-record database. ### Avoid these mistakes - MODE_PRIVATE is about app-private access; the chosen write mode determines overwrite versus append. - January is month 0 in the picker callback. - Internal files are removed on uninstall or clearing app data. ### Source/result notes Verified against the local submitted lab report; sample outputs below explain its documented behaviour, not a new device run. - MAD Lab 6 Employee Project.docx --- ## MAD — Lab 7: SQLite student database SQLite is like a register with named columns and rules. Each row is a student. Unlike a loose text file, it can find, update or remove a particular row without treating the whole register as one long string. ### Say it in the viva The app manages student records using SQLite CRUD operations. It validates roll number, name, batch and marks, then inserts, selects, updates or deletes the targeted record. SQLiteOpenHelper creates and upgrades the schema, while a Cursor reads query results. The main learning is structured persistent storage with constraints and safe, targeted operations. ### Understand the concepts **CRUD:** Create, Read, Update, Delete: the four basic record operations. Example: Insert a student; view marks; correct marks; remove a record. **Primary key:** A unique row identifier. Example: The internal numeric ID identifies one row. **UNIQUE / CHECK:** Database rules rejecting duplicates or invalid values. Example: Duplicate roll numbers and marks above 100 are rejected. **Cursor:** A movable reader over query results. Example: moveToNext visits each returned row. **ContentValues:** A container of column names and values used for writes. Example: name → Asha; marks → 88. **SQLiteOpenHelper:** Manages opening, first creation and version-based schema upgrades. Example: onCreate for a new database; onUpgrade for a schema change. **Bound arguments:** Values passed separately from SQL command text. Example: Treat a quote-containing name as data, not SQL syntax. ### What we did and why 1. **Define the structure** — The students table has an internal ID, unique roll number, name, batch and marks from 0 to 100. Why: Constraints preserve valid records even if a different UI path writes to the table. 2. **Create and read** — ContentValues supplies column-value pairs for an insert. SELECT returns matching rows through a Cursor. Why: A Cursor must move onto a row before its values can be read. 3. **Update and delete** — Use the roll number in a WHERE condition with bound arguments. Ask for confirmation before deletion and check affected-row counts. Why: An omitted condition could change every record; zero affected rows means no match. 4. **Keep and evolve records** — Data remains after the database is closed and reopened. Version 2 adds marks to the older schema through a migration. Why: An upgrade should preserve existing data rather than blindly dropping the table. ### Results and their meaning - **Recorded sequence, anonymized: Student A: 88 → update to 93; Student B deleted** — The supplied host-SQLite checks retained only the updated Student A record. - **Duplicate insert: Rejected; existing marks remain unchanged** — The unique roll-number rule protects the original record. - **Unknown identifier: 0 rows changed** — The method ran, but there was no matching target. ### Viva questions **1. What does CRUD mean?** Create adds a record, Read retrieves it, Update modifies it and Delete removes it. **2. Lab assignment: how do you fetch data from a SQL table?** Use SELECT with a column list and table; optionally filter with WHERE and sort with ORDER BY. Android exposes the results through a Cursor. **3. Lab assignment: how is data deleted?** Use DELETE or SQLiteDatabase.delete with a specific WHERE condition. It removes matching rows while retaining the table structure. **4. Lab assignment: why is SQLiteOpenHelper important?** It centralizes database creation, opening and version migrations so each Activity does not recreate tables or repeat schema logic. **5. Cursor versus ContentValues?** Cursor reads results coming out of the database. ContentValues supplies named values going into an insert or update. **6. Why use both input validation and constraints?** Validation gives helpful immediate feedback; database constraints protect integrity regardless of which code path writes. **7. What does an update count of zero mean?** No matching record was changed. The app should say not found rather than claim a successful edit. **8. What is the risk of forgetting WHERE?** The operation can affect every row. Always target the intended identifier for a single-record update or delete. **9. Does onUpgrade update a student's marks?** No. It changes the schema when the database version increases. Updating one row is a normal UPDATE operation. **10. How does this migration preserve records?** It adds the marks column with a default instead of dropping and recreating the table. **11. Why not use SharedPreferences?** A multi-row student register needs queries, constraints and targeted updates. Preferences fit a few simple settings. **12. How do bound arguments help?** They keep user values separate from SQL structure, reducing SQL-injection risk and handling quotes correctly. **13. What is a transaction?** A group of changes committed together or rolled back together, avoiding partially completed multi-step operations. **14. What is the difference between Clear Form and Delete?** Clear Form resets visible inputs. Delete changes persistent database records. ### Check yourself Which reads returned rows? - A. ContentValues - B. Toast - C. Cursor Answer: C. Cursor traverses query results. Which changes the table structure? - A. Schema migration - B. Changing a TextView - C. Clear Form Answer: A. A migration modifies the schema and should preserve needed records. ### Remember - ContentValues writes; Cursor reads; helper manages schema. - Check affected rows and keep user values bound. - Schema migration changes structure; UPDATE changes a record. ### Avoid these mistakes - An empty WHERE can affect all rows. - NOT NULL alone does not reject a blank string. - A host SQLite check is not a complete Android device test. ### Source/result notes Verified against the local submitted lab report; sample outputs explain documented behaviour, not a new device run. - MAD Lab 7.docx - MAD Experiment 7.pdf --- ## MAD — Lab 8: Google Maps marker A map is the paper; a coordinate is the address of a point; a marker is the pin; the camera is the part of the paper currently visible. A place label only names the pin—it does not find the coordinates. ### Say it in the viva The lab displays a Google Map and places a marker at entered latitude and longitude. The app waits for onMapReady, validates the coordinates, adds the marker and moves the camera. The label is displayed at that point. This version demonstrates map integration and coordinate validation; device location tracking is a separate extension. ### Understand the concepts **Latitude / longitude:** Angular coordinates locating a point on Earth. Example: Latitude measures north/south; longitude measures east/west. **Marker:** A visual pin at a coordinate. Example: A title such as Selected Location labels it. **Map camera:** The map's current centre, zoom and orientation. Example: Zooming changes the view, not the stored coordinate. **Geocoding:** Converting a place name/address into coordinates. Example: Typing Mumbai alone is not geocoding in this lab. **API key:** Identifies the application project for API requests and restrictions. Example: It is not a user password or GPS reading. **Asynchronous readiness:** The map is supplied later through a callback. Example: Use onMapReady rather than assuming immediate availability. ### What we did and why 1. **Prepare the map** — The layout hosts SupportMapFragment. getMapAsync requests a callback; onMapReady gives a usable GoogleMap. Why: Do not operate on a map object before it exists. 2. **Read and validate coordinates** — Latitude must be between -90 and 90; longitude between -180 and 180. Reject missing, nonnumeric and nonfinite values. Why: Parsing a number successfully is not enough to establish a valid coordinate. 3. **Place and view the marker** — Build a LatLng, replace the old marker, set its label and move/zoom the camera. Why: Marker position and viewed area are related but different values. 4. **Explain configuration** — The Maps SDK dependency supplies code. An enabled API, correctly restricted key and provider configuration allow runtime requests. Why: A clean compilation does not prove map authentication or network access. ### Results and their meaning - **Valid sample: 19.0760, 72.8777 → marker at that coordinate** — The lab uses numeric input; no claim that the device is physically there. - **Invalid sample: Latitude 95 → error** — Outside the valid latitude range. ### Viva questions **1. What input decides the marker's position?** The numeric latitude and longitude. The title only labels the point. **2. Does typing a place name locate it here?** No. This implementation does not geocode names; it requires coordinates. **3. What is onMapReady?** A callback giving the Activity a GoogleMap instance ready for map operations. **4. Why must latitude 95 be rejected?** Valid latitude is limited to -90 through 90 degrees. **5. Marker versus camera?** A marker stores a point of interest; the camera defines the area and zoom currently displayed. **6. Do typed coordinates need location permission?** No. They are user-provided values. Reading the device's position is a separate permission-controlled operation. **7. Lab assignment: what is an API key and how is it created?** It identifies requests for a configured cloud project. In the provider console, enable the required SDK, create credentials, restrict the key to the intended Android application/signing certificate and API, and configure it in the app. **8. Why mention billing without turning anything on?** The current Google Maps SDK setup lists a billing-enabled project as a prerequisite. This is a study explanation; no account setup or paid service is enabled here. **9. Why could the map be blank?** Possible causes include network failure, missing/incorrect key, API configuration, application restrictions or provider requirements. **10. How is this different from Foodie Assignment2?** The lab uses the Google Maps SDK. The Foodie continuation uses an OpenStreetMap WebView and gets device coordinates separately. **11. What is last-known versus current location?** Last-known can be old or absent. A current request tries to obtain a suitable fresh fix and can still fail. **12. How should a changed location appear?** Replace the prior marker and move the camera, rather than accumulating unrelated stale markers unless that is the intended feature. ### Check yourself Changing only a marker title changes its location? - A. No - B. Only with Wi-Fi - C. Yes Answer: A. Coordinates set location; the title is a label. Which needs runtime location permission? - A. Changing a marker title - B. Typing a coordinate - C. Reading device location Answer: C. Device location is protected user data. ### Remember - Coordinate = where; title = label; marker = pin; camera = view. - Wait for onMapReady. - Typed pin, geocoding and live tracking are three different features. ### Avoid these mistakes - A visible map does not prove GPS is running. - Do not reverse latitude and longitude. - No new billing, API key or paid service is created for this study guide. ### Source/result notes Verified against the local submitted lab report; sample outputs explain documented behaviour, not a new device run. - MAD Lab 8.docx - MAD Experiment 8.pdf - [Google Maps SDK setup](https://developers.google.com/maps/documentation/android-sdk/start) --- ## MAD — Lab 9: camera Intent and MiniGram Instead of building a camera, the app asks the phone's camera app to take the photo. A content URI is a controlled ticket to an image, rather than copying the full image into a message. ### Say it in the viva The experiment delegates photo capture through an implicit Intent and also supports choosing an image. The image is previewed, combined with a caption and shown in a local feed. Image files hold the picture bytes, while SQLite stores post details and flags. This demonstrates Activity results, controlled file sharing and local persistence; it is not connected to a social network. ### Understand the concepts **Implicit camera Intent:** A request for another application to perform capture. Example: The camera app provides its own preview and capture controls. **URI:** An identifier for a resource accessed through the appropriate provider. Example: content://... is resolved through ContentResolver. **FileProvider:** Shares selected app files with temporary URI access. Example: Allow the camera to write one prepared output file. **Thumbnail versus full image:** A thumbnail is a small preview; a full image needs an output destination or equivalent capture contract. Example: TakePicturePreview suits a banner preview, not a full-resolution archive. **ViewModel:** Holds UI-related state across configuration changes. Example: Keep an in-progress screen selection during rotation. **Persistence:** Retaining the record beyond an Activity instance or process. Example: SQLite plus private image files keep local posts. **RecyclerView / adapter:** A scrolling container and the component that binds post data into reusable cards. Example: Always replace image and text when a card is rebound. ### What we did and why 1. **Prepare capture** — Create an output file and a FileProvider content URI, then launch the camera with temporary access to that URI. Why: A full-size output file is different from a returned thumbnail. 2. **Receive and validate** — Handle the result and check that a usable image actually exists. A cancelled capture should not publish a post. Why: A callback alone does not guarantee valid image content. 3. **Import from gallery** — Receive a picker URI and copy the chosen image into the app's private storage. Why: A content URI is not necessarily a normal filesystem path or permanent access grant. 4. **Create the post** — Validate the image and caption; save image bytes in a file and caption/file reference/flags in SQLite; bind feed cards through an adapter. Why: Separating large image data from record metadata makes storage easier to manage. 5. **Like, save or delete** — Toggle local flags. Deleting a post removes its record and private image copy, not the user's original gallery file. Why: The app owns the imported copy, not every source image. ### Results and their meaning - **Post example: Selected image + caption → a local feed card** — Expected flow from the supplied solution. - **Persistence scope: Local application records and private images** — No restaurant or social-network backend is involved. ### Viva questions **1. Why use a camera Intent?** The camera app already provides capture UI. Delegating it is simpler than managing direct camera preview and capture APIs. **2. Why use a URI instead of a Bitmap in extras?** Large bitmaps consume memory and can exceed message limits. A URI identifies the image without copying all bytes through the Intent. **3. Why FileProvider rather than a raw file path?** It exposes controlled access to a selected file and supports temporary permissions to the receiving app. **4. Does RESULT_OK always contain the full photo in data extras?** No. Full-size capture commonly writes to the prepared output URI. The app must validate that file instead of assuming a returned bitmap. **5. What happens if capture is cancelled?** Preserve a valid previous state where appropriate and avoid creating an incomplete new post. **6. Why copy a gallery image into private storage?** The app gains a durable local copy, independent of a temporary selection grant, and can safely remove its own copy later. **7. What is stored in SQLite?** Post metadata such as caption, filename and like/saved flags. The image bytes remain in a separate file. **8. Does a ViewModel keep posts forever?** No. It helps with UI state across configuration changes; durable storage is still needed for process death and later launches. **9. Why resize very large images?** Decoded images can use much more memory than their compressed files. A controlled maximum dimension reduces memory use. **10. What happens when a post is liked twice?** In this solution the like is a Boolean toggle, so the second tap removes the like rather than endlessly increasing a counter. **11. Does deleting a post delete the original gallery image?** It should remove only the app's private copy and metadata, leaving the user's original untouched. **12. Is this Instagram?** No. It is a local educational feed without a social backend, followers, messages or real remote publishing. ### Check yourself Where should a full-size image usually be passed between components? - A. In a Toast - B. As a huge text extra - C. Through a controlled URI Answer: C. A URI avoids copying the full image into the message. Why are two storage forms used? - A. SQLite cannot hold any text - B. A ViewModel is a cloud server - C. Files hold image bytes; SQLite holds structured metadata Answer: C. Each storage form fits a different part of the post. ### Remember - Prepare URI → delegate capture → result → validate → preview → local post. - File = picture bytes; database = caption and record flags. - ViewModel is screen state, not permanent storage. ### Avoid these mistakes - A content URI is not always a filesystem path. - Do not assume image data is always returned in Intent extras. - Do not claim local feed actions upload to a social service. ### Source/result notes Verified against the local submitted lab report; sample outputs explain documented behaviour, not a new device run. - MAD Lab 9 Camera Intent.docx - [Android photo capture delegation](https://developer.android.com/media/camera/camera-deprecated/photobasics) --- ## MAD — Lab 10: broadcast sender and receiver A broadcast is like a public announcement. The action is the announcement type; extras contain the details; an IntentFilter says which announcements a receiver wants. ### Say it in the viva The app demonstrates receiving airplane-mode changes and a custom app event. It registers matching BroadcastReceivers, reads the action and extras in onReceive, and updates a status/log display. It also reads the initial state separately. The key concepts are event delivery, receiver lifetime and restricting private broadcasts to the application. ### Understand the concepts **Broadcast:** A message announcing an event to eligible receivers. Example: Airplane mode changed. **BroadcastReceiver:** A component that handles received events in onReceive. Example: Read state and refresh the screen. **IntentFilter:** Describes which action types a receiver accepts. Example: Listen for the airplane-mode action. **Dynamic registration:** Registering in code for a chosen lifetime. Example: The supplied lab pairs onCreate registration with onDestroy removal. **Static registration:** Declaring a receiver in AndroidManifest.xml. Example: Availability is still subject to action and Android background restrictions. **Exported:** Whether a receiver can receive externally originating events under applicable rules. Example: Keep an app-private custom receiver not exported. **Event log:** A record of events processed by the app. Example: A timestamp indicates handling order, not every event while unregistered. ### What we did and why 1. **Register interest** — Create a receiver and IntentFilter for the required action, with an appropriate export flag. Why: Only eligible receivers whose registration matches can receive that event. 2. **Observe system changes** — Open system settings for the user to change airplane mode. Receive the resulting event and read its Boolean state extra. Why: A normal app does not directly control airplane mode. 3. **Send a custom event** — The button sends an app-scoped action plus a message. Android delivers it to the private custom receiver. Why: This is event delivery, not a direct button call to onReceive. 4. **Process briefly and clean up** — Update the UI/log quickly and unregister when the chosen owner lifetime ends. Why: A receiver must not block on a long operation or retain an obsolete screen. ### Results and their meaning - **System event: state = true → show airplane mode ON** — Meaning of the received Boolean extra. - **Custom event: Message appears in the app's event history** — Shows sender → system delivery → receiver → UI flow. ### Viva questions **1. What does this lab demonstrate?** That an app can react to system or app events without the user directly invoking the receiver method. **2. Lab assignment: how is a receiver declared statically?** Add a receiver element in AndroidManifest.xml, name its BroadcastReceiver subclass, declare relevant intent filters and set its intended exported status. Delivery still depends on current platform restrictions. **3. Lab assignment: why is onReceive important?** It is the callback Android invokes when an eligible matching broadcast arrives. Read and validate the Intent there, perform brief work and finish quickly. **4. Does the Send button call onReceive directly?** No. It sends the broadcast; Android matches and delivers it, then invokes the receiver. **5. Why read airplane state on launch?** Listening for changes does not necessarily provide the current value immediately. The screen needs an initial state. **6. Can this app silently enable airplane mode?** No. It opens system settings so the user can change it, then observes the event. **7. Why separate system and custom receivers?** Their trust and export requirements differ. The private custom action should not become a general command endpoint for other apps. **8. What does setPackage accomplish?** It narrows the custom Intent's delivery to the intended application package. **9. Why unregister a dynamic receiver?** To stop callbacks and release the registered Context when the intended lifetime ends, preventing leaks or duplicate behaviour. **10. Why not always use onStart/onStop?** That pair limits listening to visibility. The supplied lab uses onCreate/onDestroy so it can remain registered while the user visits settings. Choose the lifetime for the scenario. **11. Can onReceive perform a long download?** It should remain short. Long work belongs in an appropriate background-work mechanism; keeping the callback blocked is unreliable. **12. Are manifest receivers guaranteed to work whenever the app is closed?** No. Modern Android restricts many implicit/background broadcasts. The action and component eligibility matter. ### Check yourself Who normally invokes onReceive? - A. SQLiteOpenHelper - B. The XML layout - C. Android after matching delivery Answer: C. The receiver is a framework callback. Why should a private custom receiver be not exported? - A. To make the screen bigger - B. To enable GPS - C. To reduce unintended access from other apps Answer: C. Its event is intended only for the app. ### Remember - Action = what happened; extras = details; filter = interest. - System event and private event need different exposure choices. - Receive quickly; unregister at the matching lifetime boundary. ### Avoid these mistakes - Broadcasts announce events; Activity Intents open screens. - Export flags are a delivery/security choice, not a performance option. - Do not claim a log captured changes while the receiver was inactive. ### Source/result notes Verified against the local submitted lab report; sample outputs explain documented behaviour, not a new device run. - MAD Lab 10 Broadcast Receiver.docx - [Android broadcasts](https://developer.android.com/develop/background-work/background-tasks/broadcasts) --- ## MAD — Lab 11: Flutter BMI and food list Flutter builds a screen as a tree of widgets. Think of nested boxes: an app contains a page, the page contains a column, and the column contains input fields, buttons and text. Changing state asks Flutter to rebuild the affected description. ### Say it in the viva The Flutter exercises demonstrate two ideas. The BMI calculator validates height and weight, converts centimetres to metres and updates a result using state. The food-list app displays products and opens a detail screen with the selected price. Both use Dart widgets instead of Android XML layouts; the main concepts are widget trees, state and navigation. ### Understand the concepts **Widget:** An immutable description of part of the UI. Example: Text, Column, ListView and Scaffold are widgets. **StatefulWidget:** A widget with a separate State object for changing UI data. Example: BMI inputs/result change during use. **StatelessWidget:** Describes UI from its supplied configuration without its own mutable State object. Example: A detail page displaying a selected immutable Food. **setState:** Records a synchronous state change and schedules rebuilding. Example: A computed BMI becomes visible in the result area. **Navigator:** Manages routes/screens in a stack. Example: push opens details; pop returns to the list. **TextEditingController:** Reads and controls a text input and holds listeners/resources. Example: Dispose it when the owning state is removed. **pubspec.yaml:** Declares project configuration, dependencies and assets. Example: Each of the two small Flutter projects has its own file. **Expanded:** Gives a child the remaining bounded space in a Row or Column. Example: A scrolling ListView inside a Column needs a bounded height. ### What we did and why 1. **Build the widget tree** — main calls runApp. MaterialApp supplies app-level theme/navigation; Scaffold supplies the page structure. Why: Each widget has a small role in the screen hierarchy. 2. **Validate and calculate BMI** — Read centimetres and kilograms, reject invalid/nonpositive values, convert height to metres and calculate weight divided by height squared. Why: Units and validation matter before showing a numeric answer. 3. **Update the screen** — Store the result in State and call setState. Clear stale results when input changes; dispose text controllers when no longer needed. Why: The visible result must correspond to the current input. 4. **Open food details** — ListView.separated displays Food objects; a tap uses Navigator.push and passes the selected object to the detail page; pop returns. Why: Keeping name and price together avoids mismatched parallel arrays. 5. **Explain the scene graph** — BMI: MaterialApp → Scaffold → ScrollView → Form → Column → fields/buttons/result. Food: MaterialApp → Scaffold → Column → Expanded → ListView → ListTile. Why: A scene graph here is the parent-child widget hierarchy and how layout space is assigned. ### Results and their meaning - **BMI sample: 170 cm, 65 kg → 22.49** — Worked calculation from the supplied solution. - **Food detail: Cupcake → Price: 10; Pizza → Price: 12** — Teacher-reference numeric values; currency unspecified. - **Widget tree: Scaffold → Column → Expanded → ListView → ListTile** — The list receives bounded height and creates rows. ### Viva questions **1. What were the two Flutter tasks?** A BMI calculator from height/weight and a food list that opens a selected item's price on another screen. **2. How is Flutter different from these native Android labs?** These Flutter tasks use Dart and widgets for UI. The earlier native tasks use Java and XML layouts. **3. What is the BMI formula used here?** Weight in kilograms divided by height in metres squared. Convert centimetres to metres first. **4. What is the result for 170 cm and 65 kg?** Height is 1.70 metres. 65 divided by 1.70 squared is approximately 22.49. **5. Why is the BMI page stateful?** The input, validation and result change during interaction and must be reflected on the screen. **6. What does setState do?** It signals that state used by the UI changed and schedules a rebuild. It does not manually draw individual pixels. **7. What is a validator's valid return?** A null result means valid; an error string explains invalid input. **8. Why clear the result after editing input?** Otherwise the old number might appear to describe the new values even though it has not been recalculated. **9. How does a food tap reach the detail screen?** The tap pushes a route carrying the selected Food object, and the detail page reads its name and price. **10. Lab question: what is the scene graph?** For this lab, draw and explain the widget/layout hierarchy: MaterialApp, Scaffold, layout containers, then input or list widgets. Parent constraints determine available space. Flutter also maintains element and rendering structures; the teaching diagram focuses on widgets. **11. Why use Expanded around ListView?** It supplies a bounded remaining height inside Column, allowing the list to scroll without an unbounded-height layout error. **12. Hot reload versus hot restart?** Hot reload applies supported code changes while preserving current state; hot restart reruns the Dart app and resets state. **13. Does this app diagnose health or prescribe a category?** No. The supplied implementation is a formula exercise displaying a number, without medical category assignment. **14. What currency do the food prices use?** The teacher reference does not specify one. Keep the numeric prices without inventing a currency. ### Check yourself What if height is entered as 170 but never converted? - A. The BMI is off by a factor of 10000 - B. Only text colour changes - C. Nothing changes Answer: A. The formula requires metres squared. Which opens a Flutter detail page? - A. SQLite SELECT - B. Navigator.push - C. startActivity in XML Answer: B. Flutter navigation uses routes managed by Navigator. ### Remember - Widget tree = nested UI; State = changing data. - 170 cm → 1.70 m → 65/2.89 → 22.49. - List tap → push detail → selected object → pop back. ### Avoid these mistakes - Use metres in the denominator; squaring centimetres gives the wrong scale. - Do not launch side effects repeatedly from build. - The source sheet names Flutter but does not print an experiment number; Lab11 is its course sequence label. ### Source/result notes Both aims and the scene-graph question verified directly in the local teacher Flutter PDF and completed report. The sheet has no printed experiment number. - Experiment on Flutter.pdf - Flutter Completed.docx - [Flutter interactivity and state](https://docs.flutter.dev/ui/interactivity) --- ## MAD — Assignment 1: Foodie Order interface Foodie Order uses a familiar restaurant menu to combine several lab concepts. Selecting food changes the visible item; Add creates a demonstration order; menus expose actions; Intents open other screens or external apps. ### Say it in the viva The first Foodie Order assignment combines Android UI components in an ordering scenario. Spinners select restaurant and food, a custom adapter displays image/name/price rows, and a swipe changes the highlighted item. Menus and Intents provide navigation, website viewing and sharing. Orders in this first version are in memory, so it demonstrates interaction rather than permanent restaurant transactions. ### Understand the concepts **Spinner:** A compact one-of-many selection control. Example: Select one restaurant. **Model object:** Groups related fields into one value. Example: FoodItem keeps its name, price and image together. **Custom adapter:** Binds a record to a more detailed row layout and actions. Example: The Add button refers to the food in its own row. **ViewHolder / row reuse:** Caches references and rebinds an old row instead of rebuilding every child lookup. Example: Always replace all fields when a row is reused. **Touch gesture:** Interprets movement between touch-down and touch-up. Example: A horizontal movement beyond a threshold counts as a swipe. **Options / popup / context menu:** Screen-wide, anchored-view and selected-item action menus. Example: Settings; More Options; Delete this order. **In-memory collection:** A list existing only inside the running process. Example: It can disappear when the process ends. ### What we did and why 1. **Select and browse** — Choose a restaurant and food through Spinners. The highlighted image and food selection remain synchronized. Why: Different controls must represent the same selected object. 2. **Bind food rows** — FoodAdapter reads a FoodItem and fills a custom row with image, name, price and Add action. Why: A single text-only row is insufficient for the richer item layout. 3. **Add and review orders** — The Add callback passes the current item to the Activity/repository; OrdersActivity displays its order records and total. Why: Keep data flow explicit: item → callback → record → order list. 4. **Use three menu types** — Toolbar Options: Home, My Orders, Profile, Settings and Exit. The More Options popup acts on the selected food: Add to Cart, Buy Now and Add to Wishlist. Long-press an order for the Context Menu: Reorder, Cancel Order and Share Order. Why: The correct menu explains what an action applies to. 5. **Open or return information** — Edit Location, Website Activity and My Orders open through explicit Intents. Website Activity contains a WebView; Share Restaurant uses an implicit share Intent. The Edit Location screen returns its saved value through an Activity result. Why: Launching a chooser is not the same as completing an external send. ### Results and their meaning - **Order flow: Select food → Add → record appears in the order list** — Educational local state, not restaurant fulfilment. - **Share flow: Chooser opens with prepared text** — The receiving app/user controls whether sending occurs. ### Viva questions **1. What is the assignment's purpose?** To combine UI widgets, adapters, gestures, menus and Intents in one understandable application flow. **2. Why use a model class?** Name, price and image belong to one food. Keeping them in one object prevents mismatches across lists, images and actions. **3. Why use a custom ListView adapter?** Each row needs multiple values and a button. The adapter supplies both presentation and the correct item callback. **4. Why is row reuse useful?** It reduces repeated view creation and lookup during scrolling. Every reused row must still be fully rebound to the current item. **5. How does a swipe select the next item?** Compare horizontal touch-down and touch-up positions, apply a threshold and move the selected index. Wrap the index at the list ends. **6. Why have a swipe threshold?** Small finger movement during a normal tap should not accidentally change the item. **7. Options menu versus popup menu versus context menu?** Options actions belong to the screen; a popup is anchored to a particular view; a context menu applies to a selected item, commonly after long press. **8. What happens when Add is tapped?** The selected food becomes a local demonstration order record and the order display/total can be refreshed. **9. Does Add place a real restaurant order?** No. It does not contact a restaurant or process payment. **10. Why can the first version lose newly added orders?** Its collection is held in process memory. It is not a database or file; the continuation adds persistence. **11. What does PriceCalculator do?** It totals the relevant order prices. The displayed total should be calculated from records rather than a hardcoded label. **12. Does opening Share prove a message was sent?** No. It prepares content and opens a chooser. The user still selects the target and completes the send there. **13. How does editing location return a value?** The destination screen sets a result containing the edited value and finishes. The caller handles that result. **14. Was the original hamburger action a complete drawer?** No. The first version contained a simple home-message placeholder; the continuation implements actual drawer navigation. **15. Why was a local repository sufficient?** The first assignment's aim is to demonstrate UI and navigation using sample data, not live restaurant integration. **16. How would you explain one improvement?** Persist order records in SQLite so they survive process death. That is implemented in the second assignment. **17. Lab assignment: why use Intent.createChooser for sharing?** It asks the user to select an eligible receiving application instead of silently assuming one destination. The chooser does not prove a message was ultimately sent. **18. Lab assignment: what is WebViewClient for?** It receives WebView navigation/loading callbacks and can keep appropriate page navigation inside the app. INTERNET permission and separate error handling are still needed. **19. What happens after Cancel Order in the first assignment?** The selected in-memory order is removed, the adapter is notified, and order count/total are recalculated. The second assignment instead retains history using a cancelled status. **20. What happens after Reorder?** A new order for the same food, restaurant and price is added with the current date and Confirmed status. It is another local demonstration record. **21. What is the initial sample order total?** The supplied first-stage data adds 249 + 199 + 299 = 747 INR. It is a sample total, not a real purchase. **22. Does finishAffinity kill the application process?** No. It finishes the current Activity and other relevant Activities below it in the same task with the same affinity. It is navigation/task cleanup, not a guaranteed process-kill command. **23. How does Assignment 1 (Partly) relate to this topic?** Its faculty reference shows the same Foodie Order interface: selectors, swipe, custom menu rows, three menu types, location/website/order screens and sharing. This topic combines that initial reference with the full local implementation. **24. What is the difference between the food popup and the order context menu?** The More Options popup applies to the currently selected food: Add to Cart, Buy Now or Wishlist. The long-press context menu applies to an existing order: Reorder, Cancel or Share. Their selected objects are different. ### Check yourself Which menu fits deleting one long-pressed order? - A. Entire phone settings - B. DatePicker - C. Context menu Answer: C. The action applies to the selected item. What survives process death in Assignment1's static list? - A. Nothing is guaranteed - B. All orders permanently - C. A Firebase backup automatically Answer: A. A static collection is still process memory. ### Remember - Spinner selection → model → custom row → Add callback → order list. - Options = screen; popup = anchored control; context = selected item. - Assignment1 memory; Assignment2 persistence. ### Avoid these mistakes - Do not describe a Toast placeholder as a working screen. - Do not confuse an overflow/options menu with a navigation drawer. - The Foodie report's reused Experiment4 heading does not make it the Grocery lab. ### Source/result notes Verified against the live faculty reference image for Assignment 1 (Partly), the full local Foodie Order submission and its implementation guide. The partial and full entries share this UI task; this walkthrough does not claim a new device run. - Foodie Order Assignment 1.docx - Foodie Order Assignment 1 Source.zip - Foodie Order Assignment 1 Faculty Reference.png --- ## MAD — Assignment 2: Foodie Order continuation The continuation turns the temporary order list into a small local record system. SQLite is the device's order book; Firebase is an optional account-linked remote store; TXT/CSV are exported documents; preferences are small personal settings. ### Say it in the viva The second assignment extends Foodie Order with persistent SQLite records, TXT/CSV files, authentication and user-scoped Firebase data. It also adds location, profile/settings and navigation screens. A user action is validated, saved locally and then optionally synchronized. Local saving and cloud saving are separate results, and demo mode demonstrates local features without pretending to be a cloud account. ### Understand the concepts **Authentication:** Establishes who the user is. Example: Firebase sign-in supplies an authenticated UID. **Authorisation:** Decides which data that user may access. Example: Rules allow a user to read/write only their own path. **SQLite versus Firebase:** SQLite is a local relational store; Firebase Realtime Database is a remote JSON-style tree. Example: Offline local order versus account-linked cloud record. **Owner scoping:** Associate records with a stable user/session identifier and filter reads/writes. Example: Two users on one phone should not see each other's orders. **Snapshot:** A saved representation of data at a particular time. Example: A TXT file does not change until exported again. **CSV escaping:** Preserve commas, quotes and line breaks inside field values using proper quoting. Example: A restaurant name containing a comma must remain one column. **Callback:** A result supplied after asynchronous work finishes. Example: Show cloud success only in the completion path. **Idempotent key choice:** Use the same record key when repeating the same logical sync. Example: Stable local order keys avoid duplicate cloud records. **SharedPreferences:** Small persistent key-value settings. Example: Theme, profile display label or feature toggles. **Fused location provider:** Combines available device location sources. Example: It returns coordinates and accuracy; it does not draw map tiles. ### What we did and why 1. **Start with a session** — The splash screen routes to authentication or the home screen. Email/Google sign-in produces a Firebase identity; local demo is explicitly separate. Why: Identity determines which user-scoped records should be read. 2. **Keep local structured data** — SQLite stores restaurants, orders and wishlist entries. Orders/wishlist are filtered by owner; cancellation changes status rather than deleting history. Why: Persistence must preserve ownership and meaningful historical records. 3. **Save an order** — FoodAdapter → MainActivity → DataRepository → DatabaseHelper → SQLite. Reload OrdersActivity and recalculate applicable totals. Why: A visible Add button is only the beginning; explain the full data path. 4. **Use TXT and CSV** — Generate readable text or structured comma-separated rows. Save privately; use the system document picker for export and a FileProvider URI for sharing. Why: An export is a snapshot, not the live database itself. 5. **Synchronize with Firebase** — Write beneath the authenticated user's UID, use stable keys for repeated local sync, and wait for completion. Listen for cloud changes and handle cancellation/errors. Why: Starting a network request is not server-confirmed success. 6. **Use location and preferences** — A fused location provider obtains coordinates with permission; an OpenStreetMap WebView displays them. Preferences store small settings. Stop recurring location/listener work when no longer needed. Why: Map rendering, device location, settings and cloud records are distinct responsibilities. ### Results and their meaning - **Offline path: Local demo/order operations can use SQLite** — Cloud authentication/sync requires its own availability and setup. - **Sync failure: Keep the valid local record and report/retry cloud failure** — Two outcomes must not be collapsed into one success claim. - **Export: Saved TXT/CSV reflects the records when generated** — Reading an old export is not automatically reading today's database. ### Viva questions **1. What changed from Assignment1?** Orders now persist in SQLite, with file export, authentication, cloud records, location and functional navigation/settings modules. **2. Why retain SQLite when Firebase exists?** Local access is fast and supports local/demo use. Cloud storage supports account-linked remote persistence. They solve different needs. **3. Which tables are used and why?** Restaurants hold catalogue records, orders hold history and status, and wishlist stores saved foods by owner. Different record types have different responsibilities. **4. What happens when a restaurant is deleted?** Historical order records preserve the restaurant name as a snapshot; deleting the catalogue entry should not erase past orders. **5. How is cancellation different from deletion?** Cancellation changes status and keeps history. Totals exclude cancelled entries while history can still list them. **6. How are monetary values represented?** The supplied whole-rupee sample menu uses integers. If smaller units were required, storing an integer count of the smallest unit avoids floating-point rounding issues. **7. What is authentication versus authorisation?** Authentication identifies the user. Authorisation enforces which records that identity may access. A sign-in screen alone does not secure the database. **8. What is the difference between demo and login?** Demo uses an explicitly local session and sample data. It does not silently impersonate a Firebase user or guarantee cloud access. **9. Why might Google selection succeed but Firebase login fail?** Choosing an account is one step. Firebase still needs a valid token and correct provider, client ID, app package and signing-certificate configuration. **10. Should the app store the password locally?** No. The authentication SDK manages its session; plaintext passwords do not belong in preferences or SQLite. **11. Where are Firebase orders stored conceptually?** Under an orders branch, then the signed-in user's UID, then the order identifier. Rules should enforce that ownership. **12. setValue versus updateChildren?** setValue replaces the value at its target; updateChildren changes specified child values while keeping other children. Use the smallest intended scope. **13. Why use a stable local order key for sync?** Repeating synchronization updates the same logical cloud record rather than creating a fresh duplicate each time. **14. Does local save success prove cloud success?** No. Local persistence and network synchronization are separate. A cloud failure should not erase a valid local record. **15. What does ValueEventListener do?** It supplies changes from the subscribed database path. The screen rebuilds its cloud list from those results and removes an unnecessary screen-level listener when it stops. **16. What is TXT versus CSV?** TXT is human-readable labelled text; CSV is tabular interchange with consistent columns. Neither is the relational database itself. **17. Why not simply join CSV values with commas?** A field can contain a comma or quote. Proper escaping keeps it in one column; spreadsheet formula prefixes also need appropriate handling. **18. Why use the system document picker for export?** It lets the user choose a destination and provides scoped access without requesting unrestricted storage access. **19. Which map and location components are used?** OpenStreetMap in a WebView renders the map. FusedLocationProviderClient independently obtains device coordinates after permission checks. **20. Can the map work while location fails?** Yes. Network map rendering and obtaining a current fix are separate operations, each with its own failure cases. **21. What happens on logout?** The active session ends and the old navigation stack is cleared. Stored local records remain owned by their account and can be selected when that user returns. **22. Where are dark mode/profile settings stored?** Small settings use SharedPreferences; theme resources support dark mode. They are not a substitute for multi-row order tables. **23. What is the local notification?** It is device feedback for an order action, not a real restaurant delivery update or Firebase Cloud Messaging service. **24. What is outside the scope?** Real restaurant fulfilment, payments, dispatch, background tracking and production conflict resolution. These should be explained as possible extensions. **25. How would you explain the architecture in one sentence?** Activities coordinate interaction, model objects group values, helpers manage storage/services, and XML defines views. ### Check yourself What actually secures user-specific Firebase data? - A. Hiding a button - B. Server-side rules tied to authenticated identity - C. Renaming MainActivity Answer: B. UI visibility is not an access-control boundary. A cloud request starts. When should success be shown? - A. Immediately before sending - B. Whenever the screen is visible - C. After the successful completion callback Answer: C. Network operations complete asynchronously. Which stores a theme switch? - A. An IntentFilter - B. SharedPreferences - C. A full-size image file Answer: B. A theme preference is a small key-value setting. ### Remember - UI → validate → local store → optional sync → feedback. - Identity says who; rules say what access. - SQLite register; TXT/CSV snapshots; preferences settings; Firebase remote records. - Map renderer ≠ location provider. ### Avoid these mistakes - Never solve permission-denied by making the whole cloud database public. - A loaded map does not prove a location fix; a login chooser does not prove Firebase acceptance. - Local demo, local save and cloud synchronization are separate states. - Do not claim sample orders are real purchases. ### Source/result notes Verified against the local Foodie Order submission and its implementation guide; this is a source walkthrough, not a new device or cloud run. - Foodie Order Assignment 2 Complete.docx - Foodie Order Assignment 2 Source.zip - MAD Viva Guide.docx - [Firebase Realtime Database reads and writes](https://firebase.google.com/docs/database/android/read-and-write) --- ## MAD — Faculty case: blood-donor registration The faculty case combines familiar pieces: a form, a local register, a list, three menu types, files and a banner image. Study it by following a single donor through input → database → list → optional export. ### Say it in the viva The proposed app registers donor details in SQLite and displays them using a custom ListView adapter. Add, Edit and Delete update the selected record, with deletion confirmation. Menus support TXT/CSV append/read actions, while the banner chooses a gallery image or camera preview. It combines CRUD, adapters, dialogs, menus and Activity results in one screen. ### Understand the concepts **DBHelper:** A SQLiteOpenHelper subclass that owns schema creation and database access. Example: Insert, query, update and delete donors. **Boolean result:** A true/false summary of a write outcome. Example: Return true when the intended insert/update/delete succeeds. **Custom BaseAdapter:** Converts donor records into row.xml views. Example: Name plus Age | Blood Group and an image. **Three menu roles:** Options: file actions; context: selected donor; popup: banner source. Example: One UI can legitimately use all three. **Append:** Adds content at the end of a file. Example: Saving another donor retains earlier exported entries. **Header row:** The CSV row naming columns. Example: id,name,age,bloodGroup is not a donor record. **Activity Result contract:** Defines the launched operation and returned value. Example: GetContent returns a URI; TakePicturePreview returns a thumbnail bitmap. ### What we did and why 1. **Collect a donor** — Use EditTexts for ID, name and age, plus a Spinner of the eight supplied blood-group labels. Validate missing or malformed input. Why: A fixed selection list avoids inconsistent blood-group spelling; this is record entry, not a medical eligibility decision. 2. **Write and display** — DBHelper creates donors(id, name, age, bloodGroup). CRUD methods target the identifier. Read the Cursor into display data and refresh CustomAdapter. Why: Database data and visible list data must stay synchronized. 3. **Select a record** — Tap a donor to fill the form. Long press opens Edit/Delete/Details; Details uses an AlertDialog. Why: A row action needs the selected donor's ID, not a stale row position. 4. **Append and read files** — Save the form donor to donors.txt or donors.csv. Write the CSV header once for a new file; skip it on reading and count O-negative rows. Why: Appending retains existing entries; repeated saves can intentionally create repeated export rows. 5. **Choose the banner** — A PopupMenu offers Gallery/GetContent or Camera/TakePicturePreview. Show the returned URI or thumbnail bitmap. Why: The faculty task says the banner is not stored in the donors database. ### Results and their meaning - **Illustrative input: Donor 7, Asha, 22, O-negative** — A fictional teaching example, not personal medical data. - **Illustrative count: Two O-negative export rows → count 2** — Counts rows; duplicate exports require an explicit deduplication rule if unique-donor counts are desired. ### Viva questions **1. What is the table schema in the task?** donors has id as INTEGER PRIMARY KEY, name as TEXT, age as INTEGER and bloodGroup as TEXT. **2. Why use the donor ID?** It uniquely identifies which record should be edited or deleted, even if two donors share a name. **3. Why does getAllDonors return a Cursor?** A query can produce multiple rows. Cursor provides a way to traverse and read those results. **4. Why refresh after each CRUD operation?** The database changed, but the adapter's displayed collection may still contain the old rows. **5. What does tapping versus long-pressing a donor do?** A tap fills the form. A long press opens donor-specific Edit, Delete and Details actions. **6. Why confirm Delete?** Deleting a record is easy to trigger accidentally; confirmation allows cancellation before the database mutation. **7. Which menu contains Save/Read TXT/CSV?** The screen's Options Menu. These are file actions rather than actions on the banner source. **8. Why write the CSV header only once?** Repeated headers would be mixed with data. The reader should skip the one header before processing donor rows. **9. How is the O-negative count obtained?** Read data rows, parse the blood-group field and increment a counter for O-negative entries. State whether it counts exported rows or unique donors. **10. Is a naive split on comma always correct?** It matches the simple classroom format only if values cannot contain embedded commas/quotes. A robust extension needs proper CSV parsing/escaping. **11. How is the banner handled?** Gallery returns a content URI for setImageURI; TakePicturePreview returns a small bitmap for setImageBitmap. **12. Is the banner photo saved with a donor?** No. The faculty scenario explicitly excludes it from the database. **13. Does Delete Files delete SQLite donors?** No. File exports and the database are separate storage; deleting export files should not silently delete donor records. **14. What do the output and count prove?** They demonstrate record processing for the entered data. This classroom app does not decide medical eligibility or actual blood availability. ### Check yourself Where does the banner belong in the specified schema? - A. In the primary key - B. Nowhere; it is not stored in the database - C. In the age column Answer: B. The task says the banner photo is not saved in the database. Which menu should Edit a long-pressed donor? - A. Banner popup - B. Context menu - C. System notification shade Answer: B. It operates on the selected list item. ### Remember - Form → SQLite → Cursor → adapter → list. - Options=file; context=donor; popup=banner. - TXT/CSV exports and database are separate. ### Avoid these mistakes - This is a faculty practice scenario, not a claimed completed/submitted app. - Do not count the CSV header as a donor. - Banner preview is not persistent full-resolution donor photography. - No medical donor-eligibility rules were supplied; do not invent them. ### Source/result notes Verified against the faculty's 30 September Teams task. Included as an integration practice case, not a claimed submitted implementation. - MAD Faculty Blood Donor Task.txt --- ## MAD — Faculty extension: current-location tracking The typed-marker lab is like placing a pin using an address you already know. Location tracking first asks the device where it is, then places or moves the pin using the returned coordinates. ### Say it in the viva The faculty extension connects the Google Map to device location updates. The app waits for the map, requests/checks location permission and subscribes through the fused location provider. Each callback supplies coordinates used to update the marker and camera. The main concepts are asynchronous services, runtime permission and stopping updates when the feature no longer needs them. ### Understand the concepts **Location fix:** An estimated device position with timestamp and accuracy. Example: It may be absent, old or imprecise. **Runtime permission:** A user grant checked while the app runs. Example: The user can deny or later revoke location access. **Fused provider:** Combines suitable device location sources. Example: Coordinates can come from more than GPS alone. **Location callback:** Receives asynchronous location results. Example: Do not assume a coordinate is available immediately after requesting it. **Map rendering:** Displays tiles, markers and camera view. Example: It can fail separately from location acquisition. **Foreground scope:** Do work while the relevant user-facing feature needs it. Example: Stop unnecessary recurring updates after leaving the screen. ### What we did and why 1. **Wait for map readiness** — Obtain GoogleMap through its callback before updating its content. Why: A permission result or location callback can arrive when the map is not ready yet. 2. **Check runtime permission** — Declare the needed permission and request it at runtime when required. Handle denial and approximate access. Why: A manifest declaration alone does not establish a current user grant. 3. **Receive updates** — Request recurring locations; read each returned latitude/longitude and update the marker/camera. Why: The requested interval is a preference, not a promise of exact delivery. 4. **Stop the subscription** — Remove updates when the appropriate lifecycle scope ends. For a foreground-only feature, stop when it is no longer needed. Why: Repeated high-accuracy work can consume battery and retain screen references. ### Results and their meaning - **Illustrative result: A new device fix moves the marker to the returned coordinate** — A conceptual flow, not a location measurement taken in this session. - **Failure state: No permission or no fix → explain unavailability** — Do not replace missing data with a fake current position. ### Viva questions **1. How is this different from Lab8's typed input?** Here the device supplies coordinates asynchronously. In the earlier submitted lab the user types them. **2. Does a map dependency supply location automatically?** No. The map renders; a separate location API supplies device coordinates. **3. Why request permission at runtime?** Location is protected user data, and the current grant can be denied or revoked. **4. What if permission is denied?** Explain that the tracking feature is unavailable while keeping unrelated app functions usable. **5. What is approximate location?** The system provides a less precise position. The app should handle it instead of assuming every grant gives exact coordinates. **6. Why update the marker in the callback?** That is when actual coordinates arrive. Code immediately after the request may run before any location is available. **7. Can the device always obtain a fresh location?** No. Indoors, emulator configuration, disabled services or other conditions can prevent a suitable fix. **8. Why remove updates?** To avoid unnecessary work, battery use and callbacks outliving the screen's intended lifetime. **9. Does a shorter interval always mean better output?** It can cost more resources and is not guaranteed to improve useful accuracy. Choose the frequency needed by the task. **10. What does a moving marker demonstrate?** That received location results are being rendered. It does not alone prove perfect accuracy or background tracking. ### Check yourself Why can a map be visible but tracking unavailable? - A. Map rendering and location are separate - B. Visible maps always mean GPS works - C. The marker title is missing Answer: A. The app still needs permission and a suitable location result. When should a marker use a new device coordinate? - A. Only after restarting Android - B. After the location callback supplies it - C. Before requesting permission Answer: B. The callback is the source of the asynchronous result. ### Remember - Permission + provider callback + map readiness are separate requirements. - Coordinates first, marker second. - Subscribe while needed; remove updates afterward. ### Avoid these mistakes - The faculty sample removes updates in onDestroy; a foreground-only design should align subscriptions with when the feature is needed, not rely on process termination. - Do not fabricate a current location when no fix exists. - Do not publish API keys from the supplied example. ### Source/result notes Verified against the faculty's 12 October Teams location-tracking example. This extension is distinct from the typed-coordinate Lab8 submission. - MAD Faculty Google Maps Task.txt --- ## MAD — Nimble: why we built it and what the user does Suppose you need an Android developer for a college project. Normally you search profiles, decide who seems suitable, write an introduction and exchange messages to find a meeting time. Nimble brings those steps into one voice-first conversation, with the user controlling the actions. ### Say it in the viva Nimble is a native Android app for professional networking, built with Java and XML. A user describes the person or opportunity they need, reviews suitable profiles and approves an introduction or meeting proposal. We use local SQLite storage, Firebase for accounts and shared records, and voice and AI integrations. A labelled demo uses fictional people, while live interactions require real participants and their approvals. ### Understand the concepts **Native Android app:** An app built using Android platform components and packaged for Android devices. Example: Nimble uses Java Activities and XML layouts, rather than showing a website as its whole interface. **Java and XML:** Java defines behavior and logic. XML defines screen structures and resources such as strings, colors and themes. Example: XML declares a Talk button; Java handles what happens when it is pressed. **Agent:** In this project, a system that uses a goal and constraints to prepare a sequence of actions. It is not an unrestricted autonomous person. Example: Find a suitable profile, prepare an introduction and wait for approval. **Brief:** Structured information about the user's goals, limits and permissions. Example: Looking for a remote collaborator, within the declared time and compensation limits. **State:** The current stage of a process; allowed next actions depend on it. Example: An introduction can be pending, accepted, declined or blocked. **Demo versus live flow:** Demo data demonstrates behavior with fictional examples. Live data represents real users and needs actual permissions and responses. Example: The sample person's acceptance is simulated and labelled. ### What we did and why 1. **1. State the user's problem** — A student may know the skill needed but not whom to approach. Search, introduction and scheduling are separate tasks. Nimble tries to make that sequence easier. Why: The goal is purposeful connections, not simply collecting more contacts. 2. **2. Create a profile and an agent brief** — The profile describes who you are: skills, experience, education and interests. The brief describes what you offer, seek and allow, including time, compensation, location and availability. Output/example: Profile = facts about the person. Brief = goals and limits for the task. 3. **3. Search and review** — The user speaks or types a request such as 'Find an Android developer in Mumbai'. The app shows profiles with matching reasons. The user should inspect those reasons and the profile before taking the next step. Why: A suggestion is useful only when the user can see why that person was suggested. 4. **4. Approve the next action** — The app can prepare an introduction locally. Sending it or proposing a meeting is an outward action, so the normal flow has an approval checkpoint. A recipient must accept a real introduction before direct messaging is available. Explicit autopilot options allow only particular actions within the stored brief. Output/example: Drafting is not the same as sending; proposing is not the same as both people accepting. 5. **5. Explain the navigation** — Chat is the home screen. A left drawer opens chat history, Deals, Messages, Meetings, Your agent and the account entry. The Talk button opens voice conversation; typing is available through a keyboard button. Why: The current app is chat-first. There is no bottom-navigation requirement or public posts feed in this scope. 6. **Two-minute project story** — Nimble helps people move from a networking goal to a useful conversation. We chose this problem because finding a suitable person is only the first step; users also need an introduction, an agreed purpose and a meeting time. The application is native Android, with Java for behavior and XML for screens and resources. A user signs in or enters a clearly labelled demo, creates a profile and describes a goal by voice or text. The app finds suitable profiles and explains the matching signals. It can draft an introduction, but sending and meeting actions follow approval rules. Agent briefs record the person's goals, limits and allowed automation. SQLite keeps private history, drafts and saved people on the phone. Firebase handles identity and shared member records. A small backend protects LinkedIn and voice-provider secrets. Cloud voice has a phone-speech fallback, although device and network conditions still matter. The final report records a successful offline build and fifteen JVM tests. Sample discovery and the guided meeting flow were observed, but current external login, continuous voice and real two-user flows still need further testing. We keep that distinction clear: the eighteen demo profiles and simulated acceptance do not represent real people or actual meetings. ### Results and their meaning - **App form: Native Java/XML Android app, version 1.2.2** — The current source and final report agree on the version. - **Demo network: 18 fictional profiles** — Useful for a controlled demonstration, not a real-world user base. ### Viva questions **1. What is Nimble?** Nimble is a voice-first professional networking Android app. It helps users find suitable people, review introductions and arrange meetings, with clear approval steps. **2. Why did you build it?** Finding a useful connection involves searching, understanding the person's fit, writing a message and scheduling. We wanted one guided flow that makes these steps easier and gives the user control. **3. What makes it different from a simple contact list?** A contact list stores people. Nimble also uses a request and a personal brief to suggest relevant profiles, explain the match and prepare next actions such as an introduction or meeting. **4. Why use voice?** A user can describe a goal naturally without filling many search filters. Voice also suits a short guided conversation. Typing remains available when speaking is inconvenient or speech recognition fails. **5. Does the agent send messages without asking?** The normal flow asks for approval before an outward action. Any autopilot action must be specifically enabled in the person's brief and remain within its limits. The app should never pretend that a draft has already been sent. **6. Are the profiles LinkedIn search results?** No. Live suggestions come from registered Nimble members, while the demo contains eighteen fictional profiles. LinkedIn sign-in supplies basic identity; full career information comes from the user's own imported document or text. **7. Why Java and XML instead of a web app?** The mini-project required a native Android implementation. Java and XML let us use Android screen lifecycles, permissions, speech, local storage, notifications and intents directly. **8. Give the project flow in one sentence.** Describe a goal, review a suitable person, approve the introduction, discuss the opportunity and confirm a meeting. ### Check yourself A draft introduction exists locally. Has it been sent? - A. Only if the profile has a photo - B. No, sending is a separate approved action - C. Yes Answer: B. Creating a draft and communicating it externally are different steps. What does the agent brief contain? - A. Goals, limits, availability and permissions - B. A list of every LinkedIn user - C. Only the person's photo Answer: A. The brief constrains what the agent should attempt. ### Remember - Memory hook: GOAL → PERSON → INTRODUCTION → AGREEMENT → MEETING. - Profile tells who; brief tells what they want and allow. - Nimble tagline: People move opportunities. - Demo is labelled; real actions need real participants. ### Avoid these mistakes - Do not claim the sample people or simulated acceptances are live users. - Do not call basic LinkedIn identity a full career-data import. - Do not describe an approval checkpoint as an already completed outward action. - Do not claim a user study proved increased networking success; no such study is reported. ### Source/result notes Checked against the current Java/XML repository, README and final 19-page MAD report. Current source version is 1.2.2. Live integrations are described with the report's partial/pending limits. - README.md - I053_Nimble_MAD_Project.pdf - experience-plan.md - MainActivity.java - Planner.java - DemoCatalog.java - [Nimble source repository](https://github.com/dhruvpaleja/nimble) - [Nimble release downloads](https://github.com/dhruvpaleja/nimble/releases/latest) --- ## MAD — Nimble: how the Android app is organized Think of the app as three connected parts: screens, decisions and data. A screen collects a request. Logic decides what to do. A repository reads or writes the required records. Keeping those roles separate makes the application easier to understand and maintain. ### Say it in the viva Nimble uses Activities for major screens and Fragments for destinations such as chat and messages. XML defines layouts and resources; adapters display changing lists. AppRepository separates screens from data access. SQLite stores private local history and drafts, while Firebase Auth and Firestore handle identity and shared member records. Intents connect the app to internal screens and external functions such as maps, sharing or a calendar. ### Understand the concepts **Activity:** An Android component that provides a screen and participates in the Android lifecycle. Example: VoiceActivity coordinates the full-screen spoken conversation. **Fragment:** A reusable piece of UI and behavior hosted within an Activity. Example: ChatFragment or MessagesFragment inside the app's navigation. **Lifecycle:** The stages a component goes through as Android creates, shows, pauses and destroys it. Example: Audio or screen resources must be managed when the screen is no longer active. **Intent:** A message asking Android to start an action. An explicit intent names a target component; an implicit intent describes an action for a suitable app. Example: Open a Nimble screen explicitly; ask a maps app to display a location implicitly. **RecyclerView and adapter:** RecyclerView reuses item views; the adapter connects data to those views. Example: Binding each chat message to the correct message-card appearance. **SQLite:** A relational database stored on the device, with tables, rows and queries. Example: Chat history remains available after restarting the app. **Firestore:** A cloud document database used here for records shared among authenticated participants. Example: A connection's status or a meeting proposal visible to its participants. **Repository:** A layer that coordinates data operations so screen code need not handle each database detail. Example: The screen asks AppRepository for information instead of constructing every local or cloud request itself. **Transaction:** A group of database changes treated as one unit. Example: Delete a conversation and its messages together. **SharedPreferences:** Storage for small key-value settings rather than large structured histories. Example: Remembering appearance settings or a notification fingerprint. **Authentication versus authorization:** Authentication asks 'Who are you?' Authorization asks 'What may you access or change?' Example: Signing in establishes an account; participant rules restrict which messages it can read. ### What we did and why 1. **1. Understand the screen components** — An Activity represents an Android screen with a lifecycle. A Fragment is a reusable section hosted inside an Activity. MainActivity hosts the main chat-and-drawer experience, while screens such as voice or profile editing have their own Activities. Why: Lifecycle separation helps manage what must be created, displayed, paused or released as the user moves between screens. 2. **2. Separate appearance from behavior** — XML layout files arrange views. Resource files store text, colors, styles and drawables. Java connects views to actions and data. Shared resources help maintain a consistent design and support light/dark themes. Output/example: Change a shared color resource to update the intended related screens consistently. 3. **3. Display lists efficiently** — RecyclerView displays a list without creating a permanent view for every item. An adapter binds records to reusable item views. Nimble's MessageAdapter displays messages, profile cards and plan steps. Why: Reusing views is more suitable for a scrolling conversation than continually adding an unlimited number of independent views. 4. **4. Explain the data boundary** — LocalStore uses SQLiteOpenHelper and the nimble.db database. Local tables hold chats, AI messages, serialized records, saved people and drafts. Firestore stores shared profiles, connections, messages, briefs, deals and meetings. AppRepository coordinates the chosen data source. Why: Local data survives closing the app and supports local/demo work. Shared records need cloud access so the relevant participants can see updates. 5. **5. Preserve database consistency** — A chat and its messages belong together. Deleting them inside one transaction means the operations commit together or roll back. Parameterized selection arguments identify records without building SQL by concatenating user input. Output/example: A primary key uniquely identifies a record; chat_id links messages to their conversation. 6. **6. Explain authentication and permission** — Authentication establishes who is signed in. Authorization decides which records that identity may access. Firestore rules check owners, participants and allowed state changes; hiding a button in the app is not sufficient protection. Android permissions separately control device capabilities such as microphone access. Why: A signed-in user must not automatically gain access to another person's private conversations. ### Results and their meaning - **Android targets: minSdk 26; compileSdk 35; targetSdk 35; Java 17 source** — The source supports Android 8.0 or newer; these are build settings, not evidence that every device was tested. - **Local database: nimble.db, schema version 2** — Uses SQLiteOpenHelper with related chat/message records and account-scoped local data. ### Viva questions **1. What is the architecture of your app?** The UI uses Activities, Fragments and XML resources. Domain classes handle matching and planning. AppRepository coordinates SQLite and Firebase data, while a separate backend protects external integration secrets. **2. Why use both SQLite and Firestore?** SQLite is for private local history, drafts and sample data. Firestore is for records that real members need to share. They serve different needs rather than duplicating every operation. **3. What data is stored in SQLite?** The main local tables store chats, AI messages, serialized records, saved people and drafts. Chat IDs connect messages to conversations, and owner fields separate local account records. **4. Why delete a chat using a transaction?** The chat row and its message rows should disappear together. A transaction prevents a partial change from leaving an inconsistent local conversation. **5. What is the difference between Activity and Fragment?** An Activity is a major screen component managed by Android. A Fragment is a reusable UI section hosted in an Activity. We use separate Activities for major flows and Fragments for destinations in the main interface. **6. Where are intents used?** Explicit intents open our own screens. Implicit intents request another app for actions such as viewing a map, sharing text, opening a link or adding a calendar event. **7. Why is checking permission only in the UI insufficient?** A UI control only affects the normal screen flow. The data service also needs authorization rules so an invalid request cannot read or change another user's records. **8. What is the difference between an APK and source code?** Source code is the Java, XML and configuration used to build the application. An APK is the packaged Android application produced by the build and installed on a device. ### Check yourself Which storage best fits a private persistent chat draft on this device in Nimble? - A. A public webpage - B. A temporary button label - C. SQLite Answer: C. Nimble's local database stores drafts and history. Why have Firestore rules if the app already hides forbidden buttons? - A. To train a language model - B. Backend authorization must also reject forbidden requests - C. Rules are only visual styling Answer: B. Client UI state is not a sufficient data-access boundary. ### Remember - Memory hook: SCREEN → LOGIC → REPOSITORY → DATA. - Activity = screen; Fragment = hosted section; Intent = action request. - SQLite = this device. Firestore = shared member records. - Authentication identifies; authorization restricts access. ### Avoid these mistakes - Do not call SQLite a cloud database. - Do not claim that a successful login grants access to all Firestore data. - Do not describe RecyclerView as the database. - Do not say XML performs the business logic. - Do not claim Android will keep an Activity alive permanently in the background. ### Source/result notes Checked in AndroidManifest.xml, build.gradle, LocalStore.java, repository structure, README and final report. No new app execution is claimed. - AndroidManifest.xml - build.gradle - LocalStore.java - AppRepository.java - MainActivity.java - MessageAdapter.java - firestore.rules - I053_Nimble_MAD_Project.pdf - [Android Activities](https://developer.android.com/guide/components/activities/intro-activities) - [Android intents](https://developer.android.com/guide/components/intents-filters) - [Android SQLite storage](https://developer.android.com/training/data-storage/sqlite) - [Firestore security rules](https://firebase.google.com/docs/firestore/security/get-started) --- ## MAD — Nimble: matching, voice, approvals and actual results Not every feature labelled 'agent' is a trained AI model. Nimble combines explicit matching and scheduling rules with cloud AI for conversation and profile extraction. Rules control the available actions and approvals. This separation makes the workflow easier to explain and constrain. ### Say it in the viva The app uses weighted profile-term matching and explicit planning, negotiation and scheduling rules. Cloud AI helps generate conversational replies and extract profile information, but outputs must map to allowed actions and existing profiles. Voice uses backend-protected cloud speech with Android speech facilities as a fallback. The recorded evaluation includes a successful build, fifteen JVM tests and observed demo flows; live provider and real two-user testing remain incomplete. ### Understand the concepts **Weighted matching:** Adding different importance values for matching terms in different profile fields. Example: A relevant role or skill can contribute more than a casual mention in an about field. **Stemming:** Reducing related word forms to a common form for matching. Example: The light matcher aims to connect developer and developers; it is not full understanding of every phrase. **Rule-based versus learned:** Rule-based logic follows explicitly written rules. A learned model uses parameters obtained from data. An application can combine both. Example: Nimble scheduling is explicit logic, while conversational generation uses a cloud AI model. **STT versus TTS:** Speech-to-text converts speech into written text. Text-to-speech converts written text into spoken audio. Example: The user's request goes through STT; the reply can be spoken through TTS. **OAuth:** An authorization flow where a provider grants limited access without giving the app the user's password. Example: LinkedIn sign-in uses a protected backend flow; identity verification is separate from importing career documents. **WorkManager:** Android's facility for deferrable work that should run when its constraints permit. Example: Network-connected periodic agent updates, not exact-to-the-minute meeting alarms. **Unit test versus integration test:** A unit test checks focused logic. An integration test checks connected components and services working together. Example: Testing schedule intersection is different from a real two-user meeting flow through Firebase. **Fallback:** An alternative available path when the preferred path cannot run. Example: Phone speech facilities or typed input when cloud voice is unavailable; availability still depends on the device. ### What we did and why 1. **1. Explain matching with a simple example** — For 'Android developer in Mumbai', the matcher removes unhelpful request words and compares meaningful terms with profile fields. It normalizes aliases such as Bombay/Mumbai and uses light stemming. Matches in different fields contribute different weights; the app can explain which profile fact matched. Why: This is transparent rule-based ranking, not a newly trained recommendation model or a guarantee that someone will be a successful collaborator. 2. **2. Turn a goal into a plan** — Planner represents steps as done, ready or waiting. Local actions such as saving a profile or preparing a draft can be completed locally. An introduction remains waiting for approval or recipient acceptance. Cloud AI helps with language, while validation checks its suggested action and profile identity. Output/example: The visible plan should reflect work actually completed, not invented agent activity. 3. **3. Negotiate and find a meeting slot** — A brief defines constraints. Negotiation checks whether proposed terms fit them. Scheduler intersects both people's declared availability windows and searches for a suitable future time. It does not inspect every event in the person's calendar. Why: A matching declared window is a candidate time, not proof that there are no real-world calendar conflicts. 4. **4. Handle voice and service failure** — Speech-to-text turns spoken audio into text; text-to-speech reads a reply aloud. Cloud voice goes through the backend, so the provider secret is outside the APK. Android recognition and TTS provide a fallback, depending on the device and available services. Typed input is also available. Output/example: No paid fallback is enabled. Free quota or network failure should produce a clear error or retry state. 5. **5. Explain background work** — WorkManager schedules network-constrained periodic agent work. The configured interval is fifteen minutes, but Android decides actual execution based on constraints and system conditions. The worker notifies when there is a meaningful pending action, subject to notification permission. Why: Periodic background work is not an exact-time alarm or a promise that the app runs continuously. 6. **6. State the actual testing evidence** — The final report records an offline assembleDebug success and fifteen JVM tests for matching, negotiation and scheduling on 18 October 2026. The report includes earlier observed screenshots of sample discovery and a guided simulated meeting. Latest external login, continuous voice and real two-user flows are marked pending or partial. Output/example: A successful build means the application packaged successfully. Unit tests validate their checked logic. Neither proves every cloud integration works on every device. 7. **7. Discuss future improvements** — The next steps are fresh external-provider checks, two-account integration tests, voice interruption and recovery tests on different devices, and better measurement of match usefulness. Keep demonstration data clearly labelled and preserve user approval boundaries. Why: These tests target the main remaining practical uncertainties instead of inventing a universal accuracy score. ### Results and their meaning - **Recorded build: assembleDebug --offline: successful** — Recorded on 18 October 2026; no fresh build is claimed by these study notes. - **Recorded JVM checks: 15 passed** — Matching, negotiation and scheduling logic, not general product success or user outcomes. - **Observed demonstration: Sample discovery and simulated meeting flow** — The profiles and peer acceptance are fictional and labelled. - **Further testing: Current external sign-in, continuous voice and real two-user flows** — Marked partial/pending in the final report; treat that as the viva evidence boundary. ### Viva questions **1. Is your people-matching algorithm a trained machine-learning model?** The inspected matcher uses weighted terms, aliases and light stemming across profile fields. It is explicit rule-based ranking. Cloud AI supports conversation, but we do not claim that this matcher was trained on a recommendation dataset. **2. Where is AI used, and where are rules used?** AI Logic supports conversational replies and profile extraction. Matching, planning, permissions, negotiation and scheduling also contain explicit rules. Model outputs are checked against allowed actions and available profiles. **3. How does the app choose a meeting time?** It compares the two briefs' declared availability windows and looks for a shared future slot. The current scheduler does not check a complete personal calendar, so the proposed time still needs appropriate confirmation. **4. What is the difference between speech recognition and TTS?** Speech recognition turns the user's spoken words into text. TTS turns the application's reply text into spoken audio. They are opposite directions in the voice conversation. **5. Will the agent definitely run every fifteen minutes?** No. Fifteen minutes is the configured periodic interval. WorkManager execution depends on Android scheduling and constraints such as connectivity. It is not an exact-time guarantee. **6. What did you test?** The final report records a successful offline debug build and fifteen JVM tests for matching, negotiation and scheduling. It also shows observed demo flows. Latest external login, continuous voice and live two-user interaction still need further testing. **7. What are the main advantages and limitations?** The app connects discovery, introductions and meetings with clear approval and persistent history. Limits include network and free quotas, device-dependent speech, rule-based match quality and incomplete live multi-user validation. **8. What would you do next?** Run fresh tests with two real accounts, test provider login and voice recovery on multiple devices, and evaluate whether the suggested matches are actually useful. I would improve those areas before claiming broad reliability. ### Check yourself Fifteen JVM tests pass. What can you conclude? - A. The app guarantees perfect networking matches - B. Every live service works indefinitely - C. The checked logic passed those tests Answer: C. The scope of a test result is limited to what was actually checked. Both users list the same evening window. What does Scheduler establish? - A. Both people have already accepted - B. No possible calendar conflict exists - C. A candidate time within their declared overlap Answer: C. Availability-window intersection is not complete calendar access or consent. ### Remember - Memory hook: MATCH → PLAN → APPROVE → ACT → VERIFY. - Rules control action; AI helps language. - STT listens into text; TTS speaks text. - Remember the honest result: build passed, 15 logic tests passed, live flows still need checks. ### Avoid these mistakes - Do not say 'all features tested end-to-end' when the report marks live flows pending. - Do not invent a matching accuracy percentage. - Do not call WorkManager an exact fifteen-minute timer. - Do not claim the calendar is completely conflict-free merely because declared windows overlap. - Do not claim phone speech is guaranteed offline on every device. - Do not describe a model-generated plan as authorization to send or accept something. ### Source/result notes Checked against PeopleMatcher, Planner, Autopilot, AgentWorker, AgentService, backend documentation and the final report's 18 October 2026 testing section. No new live provider or multi-user check performed. - PeopleMatcher.java - Planner.java - Autopilot.java - AgentWorker.java - AgentService.java - Scheduler.java - README.md - I053_Nimble_MAD_Project.pdf - [Android WorkManager](https://developer.android.com/topic/libraries/architecture/workmanager) - [Firebase Authentication](https://firebase.google.com/docs/auth) - [Firebase AI Logic](https://firebase.google.com/docs/ai-logic) - [Android SpeechRecognizer](https://developer.android.com/reference/android/speech/SpeechRecognizer) ---