Q1(b) Applications of PSO [5]

Definition (1 line)

Particle Swarm Optimization is a population-based metaheuristic, inspired by bird flocking and fish schooling, in which particles move through the search space using their own best position (pbest) and the swarm's best position (gbest) to find optimal or near-optimal solutions.

Applications

1. Function optimisation: maximising or minimising complex non-linear, multimodal functions where derivatives are unavailable.
2. Neural-network training: optimising weights and biases (your Lab 8 uses the pyswarms library for this).
3. Hyper-parameter tuning of ML models (learning rate, number of hidden units, SVM C and γ).
4. Feature selection: choosing the most useful subset of features (binary PSO).
5. Image processing: threshold selection for segmentation, edge detection, image enhancement.
6. Path planning for robots and drones: shortest collision-free path.
7. Power systems: economic load dispatch, optimal placement of capacitors.
8. Control engineering: tuning PID controller gains.
9. Scheduling and wireless sensor networks: job scheduling, sensor node placement and clustering.

One application in detail: training a neural network with PSO

1. Encoding. Flatten all weights and biases of the network into one vector. Each particle is one complete set of weights; a network with 50 weights gives a 50-dimensional search space.
2. Fitness. Run the training data through the network with that particle's weights and compute the error (for example mean squared error or cross-entropy). Lower error = better particle (minimisation).
3. Search. Apply the velocity and position updates. Each particle is pulled towards the weight set that gave its own lowest error (pbest) and towards the lowest-error weights found by the swarm (gbest).
4. Stop after a fixed number of iterations or when the error stops improving; gbest is the trained network.
5. Why PSO here: it needs no gradients, so it works with non-differentiable activations and can explore candidate weights without differentiating the objective. It can still stagnate; comparison with gradient methods depends on the problem. Limitation: large networks create a high-dimensional search space, and evaluating many complete candidate networks can be computationally expensive. Gradient-based training is often more efficient when useful gradients are available; there is no universal speed ordering for every problem and implementation.

Advantages and disadvantages (add for 10 marks)