On the convergence of multiattribute weighting methods

This is the first experiment where the subjects created the alternatives and attributes themselves and suggests that the resulting weights are different because the methods explicitly or implicitly lead the decision makers to choose their responses from a limited set of numbers.

Abstract

The convergent validity of five multiattribute weighting methods is studied in an Internet experiment. This is the first experiment where the subjects created the alternatives and attributes themselves. Each subject used five methods to assess attribute weights – one version of the analytic hierarchy process (AHP), direct point allocation, simple multiattribute rating technique (SMART), swing weighting, and tradeoff weighting. They can all be used following the principles of multiattribute value theory. Furthermore, SMART, swing, and AHP ask the decision makers to give directly the numerical estimates of weight ratios although the elicitation questions are different. In earlier studies these methods have yielded different weights. Our results suggest that the resulting weights are different because the methods explicitly or implicitly lead the decision makers to choose their responses from a limited set of numbers. The other consequences from this are that the spread of weights and the inconsistency between the preference statements depend on the number of attributes that a decision maker considers simultaneously.