Natural Language Processing · TEE · papers

Q3c · Question answering system

2025-02-21 · re-exam · recovered AI paper

Previous examination · complete worked answers

Question. Explain a question-answering system with its architecture and an example. [10 marks]

Answer

Question answering (QA) returns an answer to a natural-language question using available evidence or knowledge. Information retrieval usually returns ranked documents; QA must identify the requested information and deliver the answer. A relevant document is useful evidence but is not automatically the answer.

Question types and answer types

Question	Expected answer type	Required operation
Who founded the company?	Person or organisation	Find the founder relation
When did the museum open?	Date/time	Locate the opening date
Where is the campus?	Location	Locate the campus-location relation
How many books were issued?	Quantity	Read or compute the stated count
Why was the train cancelled?	Reason/explanation	Find the causal evidence
Is the office open on Sunday?	Yes/no with evidence	Check the relevant schedule statement

Closed-domain QA operates within a restricted area such as a university handbook. Open-domain QA addresses a wider range of subjects and usually needs broad retrieval. Extractive QA selects a span from a passage. Generative QA constructs answer text; it still must preserve the evidence and avoid unsupported details.

Question analysis

Evidence retrieval

Passage/record understanding

Answer selection or composition

Support check

Present answer

Explain each module

Module	What happens	Why it matters
Question analysis	Identify topic, intent, constraints and answer type.	When asks for a time, not a place.
Retrieval	Find a relevant passage or structured fact.	Reduces the search space.
Evidence understanding	Parse relations, resolve references and ambiguous words.	The correct date must belong to the correct event.
Answer formation	Extract a span or compose a short supported response.	Turns a document into an actual answer.
Verification	Check whether every claimed detail follows from evidence.	Prevents filling gaps by guessing.
Presentation	Return a concise answer with suitable context.	Avoids dumping unrelated text.

Worked application 1 · extract the correct fact

Question. Given “The City Museum opened in 1998 in Pune. It was renovated in 2016”, answer “When did the museum open?” Show the QA stages.

Given

A two-sentence passage. Dates 1998 and 2016 refer to different events.

To find

The opening date, not the renovation date.

Method and why it applies

Use extractive QA: identify the requested event and expected date type, then choose the span linked to that event.

Solution

Step 1. Analyse the question

When requires a date/time answer. The event is opening; the entity is the City Museum.

Step 2. Retrieve the relevant sentence

The first sentence mentions the museum and its opening. The second sentence discusses renovation.

Step 3. Identify event–date relations

Event	Date
opened	1998
renovated	2016

Step 4. Select the answer span

1998 is attached to opened, so select the four-character year span 1998.

Step 5. Reject the distractor

2016 is also a date, but it answers a different question: when was the museum renovated?

Step 6. Present the answer

“The museum opened in 1998.”

Answer. 1998. Matching only the answer type date would be insufficient without matching the event.

Worked application 2 · resolve a pronoun before answering

Question. Given “Mira joined Orion Labs in 2022. She became its research lead in 2024”, answer “Who became Orion Labs' research lead, and when?”

Given

The passage contains the name Mira, pronoun She, organisation Orion Labs and possessive its. Assume the normal reading that She refers to Mira and its to Orion Labs.

To find

The person and the date of becoming research lead.

Method and why it applies

Combine reference resolution with relation extraction. The question asks for two fields, both tied to the promotion/role event.

Solution

Step 1. Identify requested fields

Who → person; when → date. The relation is became research lead of Orion Labs.

Step 2. Resolve mentions

Mention	Referent
She	Mira
its	Orion Labs

Step 3. Read the target event

After resolution, the second sentence states that Mira became Orion Labs' research lead in 2024.

Step 4. Distinguish the dates

2022 belongs to joining the organisation. 2024 belongs to becoming research lead.

Step 5. Compose a supported answer

“Mira became Orion Labs' research lead in 2024.”

Step 6. Verify completeness

The answer supplies both requested fields and does not add an unsupported department, qualification or reason.

Answer. Mira, in 2024. Reference resolution connects the second sentence to the named person and organisation.

Worked application 3 · detect insufficient evidence

Question. A handbook excerpt states “The library opens at 8 a.m. on weekdays.” A user asks “At what time does it close on Sunday?” Explain the correct QA response.

Given

Only the opening time for weekdays is supplied. No closing time or Sunday schedule is present.

To find

Whether an answer is supported and how to handle the missing information.

Method and why it applies

Compare every question constraint against the evidence. Topic overlap with library is not sufficient: time type and day must also match.

Solution

Step 1. Extract the question constraints

Constraint	Requested
Entity	Library
Event	Closing
Day	Sunday
Answer type	Time

Step 2. Extract the supplied fact

Entity=library; event=opening; day=weekdays; time=8 a.m.

Step 3. Identify mismatches

Opening is not closing. Weekdays do not supply a Sunday schedule.

Step 4. Avoid the unsupported substitution

Do not answer 8 a.m. just because it is the only time in the passage.

Step 5. Return an evidence-limited response

“The excerpt does not state the library's Sunday closing time.” A system with further retrieval can search a relevant schedule.

Step 6. State the benefit

The system avoids converting a relevant-looking number into an incorrect answer.

Answer. No Sunday closing time is supported by the excerpt. Report insufficient evidence or retrieve the missing schedule.

Advantages, limits and evaluation

Design choice	Benefit	Limitation
Extractive answer	The returned wording comes directly from evidence.	The answer may require combining sentences or may not be present as one span.
Generative answer	Can combine supported facts into a readable response.	Can introduce unsupported information unless carefully checked.
Closed-domain collection	Controlled terminology and relevant resources.	Cannot answer facts outside its knowledge source.
Broad retrieval	Wider question coverage.	Irrelevant, outdated or conflicting sources can be retrieved.

Evaluate exact match when a canonical answer string is expected, token overlap/F1 when wording may vary, and factual correctness and completeness when answers are composed. An answer can be grammatically fluent but wrong. Questions with no answer in the evidence should be included in evaluation so the system's restraint is also tested.

For a longer exam answer, define QA, distinguish it from retrieval, explain answer types and system variants, draw the pipeline, work through a passage including distractors, and discuss ambiguity, missing evidence and evaluation. A list of chatbot names is not a QA design.

Yaad rakho: Question ka entity + event + constraint match karo. Passage mein koi bhi number mil gaya matlab answer nahi.