Natural Language Processing · TEE · papers

Q1b · Good corpus characteristics

2023-01-30 · re-exam · recovered AI paper

Previous examination · complete worked answers

Answer

A useful corpus is suited to its task and population: representative genres/languages, adequate size and diversity, balanced or intentionally documented sampling, reliable consistent annotations, clean encoding/tokenization, metadata and provenance, legal/ethical permission, documented train/test splits, and versioned reproducibility. “Large” alone does not imply representative. Example: a medical-tagging corpus needs clinical language and negation annotation; only movie reviews would be unsuitable.