Natural Language Processing · TEE · formulas

Information retrieval

Application calculations

Formulas explained through full worked examples

Information retrieval (IR) locates and ranks items relevant to an information need, usually expressed as a query. The collection may contain documents, product descriptions or scientific papers. Retrieval estimates relevance; it does not itself establish that every retrieved claim is true.

Documents → cleaning
Build index
Query → same representation
Match & rank
Return relevant results

The inverted index

An inverted index maps a term to the documents containing it, often with positions or counts. For D1 “cats drink milk” and D2 “dogs drink water”, the posting list for drink is {D1,D2}, while milk maps only to D1. This avoids rescanning every document for every query.

TermPosting list
catsD1
drinkD1, D2
milkD1
dogsD2
waterD2

Matching and ranking

Boolean retrieval combines term membership using AND, OR and NOT. Vector-space retrieval represents documents and queries as weighted vectors, for example with TF-IDF, and compares them with cosine similarity. Learned retrieval can use dense embeddings for meaning-based matching. The representation used at query time must be compatible with the document index.

Search engines and recommendation

A search engine receives an explicit query and ranks matching indexed pages. A recommender may derive an implicit query from a user’s liked items or profile and retrieve similar candidates. Both require candidate retrieval and ranking, but the source of the information need differs.

Evaluate the results

For retrieval, TP is the number of retrieved relevant documents, FP retrieved irrelevant documents, and FN relevant documents not retrieved. Precision and recall use the same formulas as classification. Ranking measures also consider the positions of relevant items; a relevant result at rank 1 is more useful than one far below.

Retrieval evaluation

A search system retrieves 8 documents; 6 are relevant. There are 10 relevant documents in the collection. Find precision, recall and F1.

Given

Retrieved=8; relevant retrieved=6; all relevant=10.

To find

P, R and F1.

Formula and why it applies

P=\frac{\text{relevant retrieved}}{\text{retrieved}},\quad R=\frac{\text{relevant retrieved}}{\text{all relevant}}LaTeX: P=\frac{\text{relevant retrieved}}{\text{retrieved}},\quad R=\frac{\text{relevant retrieved}}{\text{all relevant}}
F_1=\frac{2PR}{P+R}LaTeX: F_1=\frac{2PR}{P+R}

The denominator is 8 for precision and 10 for recall because they ask different questions.

Solution

Step 1. Identify the confusion counts

TP=6; FP=8−6=2; FN=10−6=4.

Step 2. Precision

P=\frac68=0.75LaTeX: P=\frac68=0.75

Step 3. Recall

R=\frac6{10}=0.6LaTeX: R=\frac6{10}=0.6

Step 4. F1

F_1=\frac{2(0.75)(0.6)}{0.75+0.6}=\frac{0.9}{1.35}=\frac23\approx0.666667LaTeX: F_1=\frac{2(0.75)(0.6)}{0.75+0.6}=\frac{0.9}{1.35}=\frac23\approx0.666667
Precision=0.75, recall=0.60, F1≈0.666667.

Yaad rakho: Precision: jo laaye, unmein kitne sahi? Recall: jo sahi the, unmein kitne laaye?