# Q3c · Question answering system Subject: Natural Language Processing Exam: TEE Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-r25/q3c Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r25--q3c/ Previous examination · complete worked answers **Question. Explain a question-answering system with its architecture and an example.** [10 marks] ### Answer **Question answering (QA) returns an answer to a natural-language question using available evidence or knowledge.** Information retrieval usually returns ranked documents; QA must identify the requested information and deliver the answer. A relevant document is useful evidence but is not automatically the answer. ### Question types and answer types | Question | Expected answer type | Required operation | | --- | --- | --- | | Who founded the company? | Person or organisation | Find the founder relation | | When did the museum open? | Date/time | Locate the opening date | | Where is the campus? | Location | Locate the campus-location relation | | How many books were issued? | Quantity | Read or compute the stated count | | Why was the train cancelled? | Reason/explanation | Find the causal evidence | | Is the office open on Sunday? | Yes/no with evidence | Check the relevant schedule statement | **Closed-domain QA** operates within a restricted area such as a university handbook. **Open-domain QA** addresses a wider range of subjects and usually needs broad retrieval. **Extractive QA** selects a span from a passage. **Generative QA** constructs answer text; it still must preserve the evidence and avoid unsupported details. Question analysis → Evidence retrieval → Passage/record understanding → Answer selection or composition → Support check → Present answer ### Explain each module | Module | What happens | Why it matters | | --- | --- | --- | | Question analysis | Identify topic, intent, constraints and answer type. | When asks for a time, not a place. | | Retrieval | Find a relevant passage or structured fact. | Reduces the search space. | | Evidence understanding | Parse relations, resolve references and ambiguous words. | The correct date must belong to the correct event. | | Answer formation | Extract a span or compose a short supported response. | Turns a document into an actual answer. | | Verification | Check whether every claimed detail follows from evidence. | Prevents filling gaps by guessing. | | Presentation | Return a concise answer with suitable context. | Avoids dumping unrelated text. | ### Worked application 1 · extract the correct fact **Question. Given “The City Museum opened in 1998 in Pune. It was renovated in 2016”, answer “When did the museum open?” Show the QA stages.** #### Given A two-sentence passage. Dates 1998 and 2016 refer to different events. #### To find The opening date, not the renovation date. #### Method and why it applies Use extractive QA: identify the requested event and expected date type, then choose the span linked to that event. ### Solution #### Step 1. Analyse the question When requires a date/time answer. The event is opening; the entity is the City Museum. #### Step 2. Retrieve the relevant sentence The first sentence mentions the museum and its opening. The second sentence discusses renovation. #### Step 3. Identify event–date relations | Event | Date | | --- | --- | | opened | 1998 | | renovated | 2016 | #### Step 4. Select the answer span 1998 is attached to opened, so select the four-character year span 1998. #### Step 5. Reject the distractor 2016 is also a date, but it answers a different question: when was the museum renovated? #### Step 6. Present the answer “The museum opened in 1998.” **Answer.** 1998. Matching only the answer type date would be insufficient without matching the event. ### Worked application 2 · resolve a pronoun before answering **Question. Given “Mira joined Orion Labs in 2022. She became its research lead in 2024”, answer “Who became Orion Labs' research lead, and when?”** #### Given The passage contains the name Mira, pronoun She, organisation Orion Labs and possessive its. Assume the normal reading that She refers to Mira and its to Orion Labs. #### To find The person and the date of becoming research lead. #### Method and why it applies Combine reference resolution with relation extraction. The question asks for two fields, both tied to the promotion/role event. ### Solution #### Step 1. Identify requested fields Who → person; when → date. The relation is became research lead of Orion Labs. #### Step 2. Resolve mentions | Mention | Referent | | --- | --- | | She | Mira | | its | Orion Labs | #### Step 3. Read the target event After resolution, the second sentence states that Mira became Orion Labs' research lead in 2024. #### Step 4. Distinguish the dates 2022 belongs to joining the organisation. 2024 belongs to becoming research lead. #### Step 5. Compose a supported answer “Mira became Orion Labs' research lead in 2024.” #### Step 6. Verify completeness The answer supplies both requested fields and does not add an unsupported department, qualification or reason. **Answer.** Mira, in 2024. Reference resolution connects the second sentence to the named person and organisation. ### Worked application 3 · detect insufficient evidence **Question. A handbook excerpt states “The library opens at 8 a.m. on weekdays.” A user asks “At what time does it close on Sunday?” Explain the correct QA response.** #### Given Only the opening time for weekdays is supplied. No closing time or Sunday schedule is present. #### To find Whether an answer is supported and how to handle the missing information. #### Method and why it applies Compare every question constraint against the evidence. Topic overlap with library is not sufficient: time type and day must also match. ### Solution #### Step 1. Extract the question constraints | Constraint | Requested | | --- | --- | | Entity | Library | | Event | Closing | | Day | Sunday | | Answer type | Time | #### Step 2. Extract the supplied fact Entity=library; event=opening; day=weekdays; time=8 a.m. #### Step 3. Identify mismatches Opening is not closing. Weekdays do not supply a Sunday schedule. #### Step 4. Avoid the unsupported substitution Do not answer 8 a.m. just because it is the only time in the passage. #### Step 5. Return an evidence-limited response “The excerpt does not state the library's Sunday closing time.” A system with further retrieval can search a relevant schedule. #### Step 6. State the benefit The system avoids converting a relevant-looking number into an incorrect answer. **Answer.** No Sunday closing time is supported by the excerpt. Report insufficient evidence or retrieve the missing schedule. ### Advantages, limits and evaluation | Design choice | Benefit | Limitation | | --- | --- | --- | | Extractive answer | The returned wording comes directly from evidence. | The answer may require combining sentences or may not be present as one span. | | Generative answer | Can combine supported facts into a readable response. | Can introduce unsupported information unless carefully checked. | | Closed-domain collection | Controlled terminology and relevant resources. | Cannot answer facts outside its knowledge source. | | Broad retrieval | Wider question coverage. | Irrelevant, outdated or conflicting sources can be retrieved. | Evaluate **exact match** when a canonical answer string is expected, **token overlap/F1** when wording may vary, and **factual correctness and completeness** when answers are composed. An answer can be grammatically fluent but wrong. Questions with no answer in the evidence should be included in evaluation so the system's restraint is also tested. For a longer exam answer, define QA, distinguish it from retrieval, explain answer types and system variants, draw the pipeline, work through a passage including distractors, and discuss ambiguity, missing evidence and evaluation. A list of chatbot names is not a QA design. **Yaad rakho:** Question ka entity + event + constraint match karo. Passage mein koi bhi number mil gaya matlab answer nahi.