Advanced Optimization Techniques

Abstract

Many difficulties such as multi-modality, dimensionality and differentiability are associated with the optimization of large-scale problems. Traditional techniques such as steepest decent, linear programing and dynamic programing generally fail to solve such large-scale problems especially with nonlinear objective functions. Most of the traditional techniques require gradient information and hence it is not possible to solve non-differentiable functions with the help of such traditional techniques. Moreover, such techniques often fail to solve optimization problems that have many local optima. To overcome these problems, there is a need to develop more powerful optimization techniques and research is going on to find effective optimization techniques since last three decades. Some of the well-known population-based optimization techniques developed during last three decades are: Genetic Algorithms (GA) [16] which works on the principle of the Darwinian theory of the survival-of-the fittest and the theory of evolution of the living beings; Artificial Immune Algorithms (AIA) [14] which works on the principle of immune system of the human being; Ant Colony Optimization (ACO) [10] which works on the principle of foraging behavior of the ant for the food; Particle Swarm Optimization (PSO) [20] which works on the principle of foraging behavior of the swarm of birds; Differential Evolution (DE) [35] which is similar to GA with specialized crossover and selection method; Harmony Search (HS) [15] which works on the principle of music improvisation in a music player; Bacteria Foraging Optimization (BFO) [27] which works on the principle of behavior of bacteria; Shuffled Frog Leaping (SFL) [12] which works on the principle of communication among the frogs, Artificial Bee Colony (ABC) [18] which works on the principle of foraging behavior of a honey bee; Biogeography-Based Optimization (BBO) [34] which works on the principle of immigration and emigration of the species from one place to the other; Gravitational Search Algorithm (GSA) [29] which works on the principle of gravitational force acting between the bodies and Grenade Explosion Method (GEM) [1] which works on the principle of explosion of