On the Use of Run Time Distributions to Evaluate and Compare Stochastic Local Search Algorithms

A closed form index is derived that gives the probability that one of them finds a solution at least as good as a given target value in a smaller computation time than the other in the case of general run time distributions.

Abstract

Run time distributions or time-to-target plots are very useful tools to characterize the running times of stochastic algorithms for combinatorial optimization. We further explore run time distributions and describe a new tool to compare two algorithms based on stochastic local search. For the case where the running times of both algorithms fit exponential distributions, we derive a closed form index that gives the probability that one of them finds a solution at least as good as a given target value in a smaller computation time than the other. This result is extended to the case of general run time distributions and a numerical iterative procedure is described for the computation of the above probability value. Numerical examples illustrate the application of this tool in the comparison of different algorithms for three different problems.