F22 Q3a

Answer — step by step

Bias is error caused by restrictive assumptions or insufficient fit; variance is sensitivity to the particular training sample. A high-bias model may miss useful structure and perform poorly even on training data. A high-variance model can fit training data very well but perform much worse on comparable validation data.

Diagram: Validation behavior distinguishes useful fitting from prolonged fitting to training details. Labels: Loss | Epochs | Validation | Training | Best validation region | Save the best validation checkpoint; do not choose by test loss.

Increasing model capacity can reduce bias but may increase variance. Stronger regularization usually increases bias and reduces variance. More representative training data often reduces variance without changing the model's approximation bias much. These are typical trends, not unconditional numerical laws.

Underfitting remedies include improving features, using a more expressive appropriate model, training sufficiently and reducing excessive regularization. Overfitting remedies include representative extra data, data augmentation, appropriate L1/L2 or dropout, early stopping and sometimes reducing model size. Keep an untouched test set for final evaluation. Repeatedly tuning on test results turns that set into another validation set and requires a fresh test set for an unbiased final estimate.

Training and validation errors alone do not determine a diagnosis without task difficulty, evaluation consistency and learning curves. A small gap with both errors high suggests underfitting; a large gap with very low training error suggests overfitting.

Classifier: four mutually exclusive grades require 4 output scores, softmax probabilities and categorical cross-entropy. A reject class is still a labeled class in this formulation.

Diagnosis: a modest 3 percentage-point gap with both errors appreciable suggests underfitting/high bias if a much better attainable baseline exists. Confirm learning curves and label quality. Improve relevant features or model capacity, train sufficiently, tune the optimizer and reduce excessive regularization. The percentages alone do not prove the cause.