# Bag of words and bag of n-grams

Subject: Natural Language Processing

Exam: TEE

Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/formulas/f-vectors/u2-bow

Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/formulas/f-vectors--u2-bow/

Formulas explained through full worked examples

**Bag of words (BoW) represents a document by the counts of vocabulary terms, discarding their positions.** A binary BoW variant stores only presence/absence; a count BoW stores frequencies. State which is used.



$$
x_{d,j}=C(v_j,d).
$$





$$
x_{d,j}
$$

 is document 

$$
d
$$

's coordinate for vocabulary term 

$$
v_j
$$

. 

$$
C(v_j,d)
$$

 is its occurrence count.

A **bag of n-grams** uses short contiguous token sequences as features. Bigrams can distinguish “not good” from “good”, but increasing n enlarges the vocabulary and creates more rare features.

**Question. Build count and binary BoW for D1=“cat cat dog” and D2=“dog fish”. Use vocabulary [cat,dog,fish] and no stopword removal.**

#### Given

Two tokenized documents, fixed vocabulary order.

#### To find

Count vectors and binary vectors.

#### Formula and why it applies

Count BoW uses 

$$
C(v_j,d)
$$

. Binary BoW is 1 when that count is positive and 0 otherwise.

#### Step 1. D1 counts

cat=2, dog=1, fish=0 → [2,1,0].

#### Step 2. D2 counts

cat=0, dog=1, fish=1 → [0,1,1].

#### Step 3. Convert to presence

D1 [1,1,0]; D2 [0,1,1].

**Answer.** Count rows [2,1,0], [0,1,1]; binary rows [1,1,0], [0,1,1].

**Question. After lowercasing and removing {is,a,means}, obtain BoW vectors for D1=“Continuous learning is learning lifetime”, D2=“Continuous learning means lifetime learning”, D3=“Learning is a good practice”. Use [continuous,good,learning,lifetime,practice].**

#### Given

Three documents and exact stopword set from the supplied slides.

#### To find

The document–term count matrix.

#### Formula and why it applies

Remove only the listed stopwords, then count every remaining token in the fixed column order.

#### Step 1. Clean D1

[continuous,learning,learning,lifetime] → [1,0,2,1,0].

#### Step 2. Clean D2

[continuous,learning,lifetime,learning] → [1,0,2,1,0].

#### Step 3. Clean D3

[learning,good,practice] → [0,1,1,0,1].

#### Step 4. Compare D1 and D2

They have identical BoW vectors despite different original wording/order.

**Answer.** Rows [1,0,2,1,0], [1,0,2,1,0], [0,1,1,0,1].

**Question. Build a bag of bigrams for “not good but not bad” with vocabulary [not good,good but,but not,not bad]. Do not add sentence boundary tokens.**

#### Given

Five tokens; use adjacent pairs, stride 1.

#### To find

The bigram count vector.

#### Formula and why it applies

A sequence of 

$$
L
$$

 tokens has 

$$
L-1
$$

 adjacent bigrams when no boundaries are added.

#### Step 1. Enumerate pairs

(not,good), (good,but), (but,not), (not,bad).

#### Step 2. Count each vocabulary bigram

Each appears once.

#### Step 3. Write vector

[1,1,1,1].

**Answer.** Four bigram features, each with count 1.

BoW: order bhool, word-count rakh. Bag of bigrams: neighbouring word-pair count.