Natural Language Processing · TEE · papers

Q7b · Co-occurrence, dot products and plot

2024-12-03 · final · recovered AI paper

Previous examination · complete worked answers

Question. Define target words, context window, context words and co-occurrence matrix. For the printed boy/ball/dog/sun corpus below, use one whole sentence as the window; contexts={kicked,field,tree,ran}. Find boy·dog and boy·sun, plot on kicked/field dimensions and comment. [10 marks]

Answer

Target is the word whose row we represent. Context is an observed neighbouring word whose column we count. A window defines which tokens count as neighbours; here all other tokens in the same sentence. A co-occurrence matrix records those counts.

Given

The boy kicked the ball into the field. The dog barked at the boy. The sun was shining over the field. The boy ran after the dog. The ball rolled to the tree.

To find

Matrix, two dot products and the requested 2D coordinates.

Formula: which one and why

u\cdot v=\sum_i u_iv_iLaTeX: u\cdot v=\sum_i u_iv_i

Use lowercase token-pair counts within each sentence; do not cross sentence boundaries.

Step 1. List contributions and build the matrix

SentenceTarget occurrencesContext occurrences
1boy, ballkicked, field
2dog, boy
3sunfield
4boy, dogran
5balltree
Target / contextkickedfieldtreeran
boy1101
ball1110
dog0001
sun0100

Step 2. Dot boy and dog

[1, 1, 0, 1] · [0, 0, 0, 1] = 1×0 + 1×0 + 0×0 + 1×1 = 1

Step 3. Dot boy and sun

[1, 1, 0, 1] · [0, 1, 0, 0] = 1×0 + 1×1 + 0×0 + 1×0 = 1

Step 4. Plot coordinates: kicked on x, field on y

Target(x,y)
boy(1, 1)
ball(1, 1)
dog(0, 0)
sun(0, 1)
kickedfielddog (0,0)sun (0,1)boy, ball (1,1)
Two coordinates omit tree and ran, so boy and ball overlap here despite different full vectors.

Final answer

boy·dog=1; boy·sun=1. Both share one counted context contribution. Dot-product similarity reflects this corpus and magnitude, not a claim that boy is equally semantically related to dog and sun in general.