Deep Learning · TEE · section
R25 Q1a · ML versus DL and architecture choice
February 2025 re-exam · complete solved paper
Question
Question. Compare ML and DL. Choose architectures for five-class diabetic-retinopathy retinal images, alphabet-image recognition, and weather prediction from N days of history. (5 marks)
Answer — step by step
Definition. Deep learning is a subfield of machine learning that learns representations through multiple trainable processing layers. Each layer transforms the preceding representation, allowing the complete model to learn a composition of functions from data.
Position within artificial intelligence
Artificial intelligence studies systems that perform tasks such as perception, reasoning and decision-making. Machine learning develops models whose parameters or behavior are estimated from data. Deep learning uses layered neural models within machine learning.
LaTeX: \text{Deep learning}\subset\text{Machine learning}\subset\text{Artificial intelligence}Motivation: why use multiple learned layers?
Representation learning. Hand-designing every useful image or speech feature is difficult. A deep model can learn intermediate representations together with the final prediction objective. In a CNN, early layers may respond to local edges, while later layers combine information over larger regions. These patterns depend on training and are not guaranteed to have a particular human meaning.
Nonlinear relationships. Real tasks often depend on combinations of input variables. Composing affine transformations with nonlinear activations can represent interactions that a single linear decision boundary cannot. Depth supplies repeated transformations; increasing only the number of neurons in one layer changes width instead.
End-to-end optimization. The final task loss supplies gradients to the preceding trainable layers. The feature extractor and prediction head can therefore be adjusted for the same task. Preprocessing and human decisions about architecture, data and evaluation still remain necessary.
Factors that made deep learning practical
| Factor | How it contributes |
|---|---|
| Larger datasets | Provide more examples of the variations a complex model must learn |
| GPUs and parallel computation | Accelerate the large matrix and convolution operations used in training |
| Improved training methods | Backpropagation, suitable activations, initialization and optimizers make fitting deep models more practical |
| Pretrained models | Reuse learned features so a new task may require less labeled data and training effort |
Applications in different domains
An application answer should identify the input, the prediction and the role of the learned representation.
| Domain | Example input | Required output | Role of deep learning |
|---|---|---|---|
| Computer vision | Photograph or video frame | Image class, object locations or pixel labels | Learn visual features with a CNN and an appropriate task head |
| Language | Review, sentence or conversation | Sentiment label, translated words or a response | Learn representations of text and its context |
| Speech | Audio sequence | Words or a speaker label | Model patterns in acoustic features over time |
| Healthcare | Medical image or patient measurements | A task-specific class or numerical estimate | Learn associations from labeled examples; assess results on suitable held-out data |
| Autonomous systems and robotics | Camera and other sensor measurements | Perception estimates used by a larger control system | Recognize objects and relevant environmental features |
| Financial services | Transaction features or a time-ordered record | Fraud score, risk estimate or forecast | Model patterns and interactions among observations |
Advantages, limitations and comparison with traditional ML
| Aspect | Traditional machine learning often uses | Deep learning often uses |
|---|---|---|
| Representation | Selected or engineered input features | Features learned jointly across layers |
| Useful architectures | Linear models, decision trees, SVMs | MLPs, CNNs, RNNs and gated recurrent models |
| Training from scratch | Can be effective with modest structured datasets | Large models often need substantial data and computation |
| Interpretation | Some models have directly inspectable rules or coefficients | Learned internal representations can be harder to interpret |
Feature learning, flexible nonlinear modeling and reuse of pretrained representations are major advantages. Limitations include computation and memory costs, overfitting with insufficient representative data, difficulty interpreting predictions, and reduced reliability when deployment data differs from training data. High training accuracy does not establish generalization.
These are tendencies, not absolute divisions. Traditional ML can also learn representations; deep models can process tabular data. A pretrained deep model may be useful even with a small target dataset. Select the model according to the task, available data, resources and validation results.
Question. Explain the motivation for deep learning and describe applications in at least three domains.
Answer. Deep learning is motivated by the need to learn useful hierarchical representations and nonlinear relationships directly from complex data. Multiple trainable layers compose feature transformations, and the task loss can train those transformations jointly with the predictor. Larger datasets, GPU computation and improved optimization methods have made this practical. In computer vision, CNNs learn visual features for image classification and object localization. In language processing, neural sequence models use word representations and context for sentiment analysis or translation. In speech processing, neural models map acoustic patterns to words. The benefits must be balanced against data requirements, computation, overfitting and interpretability limitations.
| Task | Suitable architecture | Why / output |
|---|---|---|
| Retinal images,5classes | CNN | Spatial image features;5-way softmax for exclusive classes |
| Alphabet images | CNN | Edges/shapes compose into characters; class-count softmax |
| Weather from a temporal history | RNN/LSTM or GRU | Ordered history; regression output for numeric weather quantities, suitable classification head for categorical weather |
Architecture choice is justified by data structure. A fully connected model can technically take flattened images, but ignores spatial locality; sequential models preserve temporal order.