F23 Q7b

Answer — step by step

Bias is error caused by restrictive assumptions or insufficient fit; variance is sensitivity to the particular training sample. A high-bias model may miss useful structure and perform poorly even on training data. A high-variance model can fit training data very well but perform much worse on comparable validation data.

Diagram: Validation behavior distinguishes useful fitting from prolonged fitting to training details. Labels: Loss | Epochs | Validation | Training | Best validation region | Save the best validation checkpoint; do not choose by test loss.

Increasing model capacity can reduce bias but may increase variance. Stronger regularization usually increases bias and reduces variance. More representative training data often reduces variance without changing the model's approximation bias much. These are typical trends, not unconditional numerical laws.

Underfitting remedies include improving features, using a more expressive appropriate model, training sufficiently and reducing excessive regularization. Overfitting remedies include representative extra data, data augmentation, appropriate L1/L2 or dropout, early stopping and sometimes reducing model size. Keep an untouched test set for final evaluation. Repeatedly tuning on test results turns that set into another validation set and requires a fresh test set for an unbiased final estimate.

Training and validation errors alone do not determine a diagnosis without task difficulty, evaluation consistency and learning curves. A small gap with both errors high suggests underfitting; a large gap with very low training error suggests overfitting.

These are the conventional exam trends, assuming other conditions are comparable. They are not guarantees for every training run.