# R23 Q1d

Subject: Deep Learning

Exam: MT2

Reader: https://deep-learning-study-room.vercel.app/#mt2/read/16/mt2-c16-s2

Canonical: https://deep-learning-study-room.vercel.app/read/dl/mt2/section/mt2-c16-s2/

## Question

**Question (4 marks).** Evaluate how mini-batch size affects optimization and parallel processing.

## Answer — step by step

A mini-batch forms one gradient from B examples. Smaller B usually gives noisier gradients, lower activation memory and more updates per epoch. Larger B averages more examples, often uses vectorized hardware more efficiently up to resource limits, but needs more memory and gives fewer updates per epoch. N examples produce ceil(N/B) updates if the last partial batch is kept. Too-large B does not guarantee faster wall-clock progress or better generalization. Tune batch size with learning rate and resource constraints. Computation within a batch can be parallelized; successive optimizer steps remain dependent on the updated parameters.