Studying a set of properties of inconsistency indices for pairwise comparisons1footnote 11footnote 1This is a preprint of the paper: Brunelli M., Studying a set of properties of inconsistency indices for pairwise comparisons, Annals of Operations Research, doi:10.1007/s10479-016-2166-8

Studying a set of properties of inconsistency indices for pairwise comparisons111This is a preprint of the paper: Brunelli M., Studying a set of properties of inconsistency indices for pairwise comparisons, Annals of Operations Research, doi:10.1007/s10479-016-2166-8

Matteo Brunelli
Systems Analysis Laboratory, Department of Mathematics and Systems Analysis
Aalto University
   P.O. Box 11100, FI-00076 Aalto, Finland
e–mail: matteo.brunelli@aalto.fi

Abstract

Pairwise comparisons between alternatives are a well-established tool to decompose decision problems into smaller and more easily tractable sub-problems. However, due to our limited rationality, the subjective preferences expressed by decision makers over pairs of alternatives can hardly ever be consistent. Therefore, several inconsistency indices have been proposed in the literature to quantify the extent of the deviation from complete consistency. Only recently, a set of properties has been proposed to define a family of functions representing inconsistency indices. The scope of this paper is twofold. Firstly, it expands the set of properties by adding and justifying a new one. Secondly, it continues the study of inconsistency indices to check whether or not they satisfy the above mentioned properties. Out of the four indices considered in this paper, in its present form, two fail to satisfy some properties. An adjusted version of one index is proposed so that it fulfills them.

Keywords: pairwise comparisons, consistency, inconsistency indices, analytic hierarchy process.

In decision making problems it is often common practice to use pairwise comparisons between alternatives as a basis to assign scores to the same alternatives. Pairwise comparisons allow the decision maker to decompose the problem of assigning scores to alternatives into smaller problems, where only two alternatives are considered at a time.

Another reason for using pairwise comparisons is that their use allows an estimation of the inconsistency of the preferences of a decision maker. In the literature, consistency of preferences is commonly related with the rationality of a decision maker and his ability in discriminating between alternatives (Irwin, 1958). Consider, for sake of illustration, three stones (alternatives) . If, for instance, is reputed twice as heavy as , and twice as heavy as , then it is reasonable to assume that should be four times as heavy as . This situation is called consistent, as the pairwise comparisons of the decision maker respect a principle of transitivity/rationality, and is depicted in Figure 1(a). An example of inconsistent pairwise comparisons is illustrated in Figure 1(b).

(a) Consistent triad of pairwise comparisons.
(b) Inconsistent triad of pairwise comparisons.
Figure 1: Example of consistent and inconsistent triads of pairwise comparisons on .

There is a meeting of minds on accepting preferences which are not consistent, but not too inconsistent either. In this paper, with the term inconsistency we mean a deviation from the condition of full consistency. In the theory of the AHP, Saaty (1993, 2013) required pairwise comparisons to be near consistent, i.e. not too inconsistent. As recalled by Gass (2005), Luce and Raiffa (1957) shared the same opinion in accepting inconsistencies and wrote “No matter how intransitivities arise, we must recognize that they exist, and we can take a little comfort in the thought that they are an anathema to most of what constitutes theory in the behavioral sciences today”. On a similar note, Fishburn (1999) wrote that “Transitivity is obviously a great practical convenience and a nice thing to have for mathematical purposes, but long ago this author ceased to understand why it should be a cornerstone of normative decision theory”.

It is in this context—where consistency is an auspicable but hardly ever achievable condition—that it becomes crucial to quantify inconsistency. Such quantification is indeed possible, since it is natural to envision that the notion of inconsistency is a matter or degree. Consequently, a wealth of inconsistency indices has been proposed in the literature; for instance the Consistency Index (Saaty, 2013), the Harmonic Consistency Index (Stein and Mizzi, 2007), the Geometric Consistency Index (Aguarón and Moreno-Jiménez, 2003), the statistical index by Lin et al. (2013), and the index by Kułakowski (2015), just to cite few.

It is worth noting that the study of inconsistency of preferences is not limited to the single mathematical methods employing pairwise comparisons, as for instance the AHP. It is the case to remark that the study of inconsistency is immune from many of the criticisms moved against specific mathematical methods employing them. For instance, one of the critical points of the Analytic Hierarchy Process (AHP) is the rank reversal, which was discovered by Belton and Gear (1983) and recently surveyed by Maleki and Zahir (2013). Similarly, already Watson and Freeling (1982, 1983) questioned the interpretation of the weights in the AHP and their use in the aggregation of different priority vectors. In part, also the criticisms by Dyer (1990a, b) were triggered by the interpretation of the weights. Nevertheless, even though the above mentioned criticisms are to be taken into account, they are connected with the aggregation and interpretation of priority vectors proposed for the AHP, and therefore they will not affect the subject matter of inconsistency evaluation. Further support to the use of pairwise comparison matrices and their interpretation comes from the fact that pairwise comparison matrices as defined in this paper are group isomorphic (Cavallo and D’Apuzzo, 2009)—and thus structurally identical—to the probabilistic preference relations studied by Luce and Suppes (1965). Such a strict connection between these two representations of valued preferences does not only make them mutually supportive, but increases the relevance of studying one of them—as it is going to be done in this paper—since abstract results are then extendible to the other one.

The use of the notion of inconsistency has gone beyond its mere quantification. One prominent use of inconsistency indices is that of localizing the inconsistency and detect what comparisons are the most contradictory (Ergu et al., 2011) and guide the decision maker when he tries to obtain sufficiently consistent preferences (Pereira and Costa, 2015). This process was also advocated by Fishburn (1968) in a discussion on decision theory: “If the individual’s preferences appear to violate a “rational” preference assumption, the theory suggests that he reexamine and revise one or more preference judgments to eliminate the inconsistency.”. Another use of inconsistency indices regards pairwise comparison matrices with missing entries. In these situations, inconsistency indices have been used as objective functions to be minimized to find the most plausible values of the missing comparisons with respect to the elicited ones (Koczkodaj et al., 1999; Lamata and Pelaez, 2002; Shiraishi et al., 1999; Chen et al., 2015). All this can be seen as evidence on the role played by inconsistency indices in the decision process, and consequently on the importance of having realiable indices.

Inconsistency of preferences has been studied empirically (Bozóki et al, 2013), and existing studies on inconsistency indices compared them numerically (Brunelli et al, 2013a) and showed that some indices are very different and therefore can lead to very different evaluations of the inconsistency of preferences. Conversely, it was proven that some of them are in fact proportional to each other (Brunelli et al., 2013b). Recently, Brunelli and Fedrizzi (2015a) and Koczkodaj and Szwarc (2014) proposed two formal approaches. Brunelli and Fedrizzi (2015a) proposed five properties in the form of axioms to formalize the concept of inconsistency index and then tested on some well-known indices.

In the pursuit of a formal treatment of inconsistency quantification, this paper presents some developments concerning the aforementioned set of properties. Firstly, in Section 2, a new property, of invariance under inversion of preferences, is introduced and its role is discussed. Secondly, Section 3 contains further results on the satisfaction of the properties by some known inconsistency indices. More specifically, we shall study four indices and discover that, in its present form, two do not fully satisfy the set of properties. An adjustment of one index is then proposed so that it satisfies them. Finally, Section 4 offers a concise discussion on the role of inconsistency quantification and on the results obtained in this paper.

1 Pairwise comparison matrices and inconsistency indices

Given a set of alternatives, a pairwise comparison matrix is a positive square matrix such that , where is the subjective assessment of the relative importance of the th alternative with respect to the th one. A pairwise comparison matrix can be seen as a convenient mathematical structure into which valued pairwise comparisons between alternatives are collected. Its general and its simplified (thanks to ) forms are the following,

The rest of the paper will follow the usual interpretation of entries in terms of ratios between quantities expressible on a ratio scale with a zero element. The classical example is that of and being stones and being the numerical estimation of the ratio between their weights. Note that this approach considers entries taking values from an unbounded scale and complies with the formal treatment given by Herman and Koczkodaj (1996) and Koczkodaj and Szwarc (2014). Furthermore, with this interpretation, a pairwise comparison matrix is consistent if and only if

(1)

which means that each direct comparison is exactly backed up by all indirect comparisons . For notational convenience, the set of all pairwise comparison matrices is defined as

The set of all consistent pairwise comparison matrices is defined accordingly,

An inconsistency index is a function which evaluates the intensity of deviation of a pairwise comparison matrix from its consistent form (1). In other words, the value is an estimation of how much irrational the preferences collected in are. Up to now, various inconsistency indices have been introduced heuristically, and an open question relates to what set of properties should be used to characterize them. That is, all the reasonable properties for a function to fairly capture inconsistency could be used for various purposes; for example to check the validity of already proposed indices (Brunelli and Fedrizzi, 2015a), devise new ones, and derive further properties (Brunelli and Fedrizzi, 2015b). Figure 2 offers a snapshot of the meaning of the set of properties.

Figure 2: The set of properties can be used to define a family of functions which can be used to estimate inconsistency, and discards functions which do not make sense if used as inconsistency indices, e.g. the trace and the determinant of .

Brunelli and Fedrizzi (2015a) proposed five properties to characterize inconsistency indices. Since these properties were already justified and defined in the original work, they are here only briefly recalled. Note that they were organized in the form of an axiomatic systems, meaning that the soundness of single properties implies the soundness of the entire set of properties, i.e. the “logical intersection” of the properties.

P1:

There exists a unique representing the situation of full consistency, i.e.

P2:

Changing the order of the alternatives does not affect the inconsistency of preferences. That is,

for any permutation matrix .

P3:

If preferences in are intensified, then the inconsistency cannot decrease. More formally, since the power is the only meaningful function to intensify preferences, we defined . Then, the property is as follows,

P4:

Given a consistent pairwise comparison matrix and considering an arbitrary non-diagonal element (and its reciprocal ) such that , then, as we push its value far from its original one, the inconsistency of the matrix should not decrease. More formally, given a consistent matrix , let be the inconsistent matrix obtained from A by replacing the entry with , where . Necessarily, must be replaced by in order to preserve reciprocity. Let be the inconsistent matrix obtained from A by replacing entries and with and respectively. The property can then be formulated as

(2)

for all , , and .

P5:

Function is continuous with respect to the entries of .

2 A new property of invariance under inversion of preferences

Preferences expressed in the form of a pairwise comparison matrix can be inverted by taking its transpose . For instance, if in is inverted into we have that the intensity of preference is the same, but the direction is inverted. Clearly, by inverting all the preferences we change their polarity, but leave their structure unchanged. Thus, it is reasonable to expect a structural property of preferences—as inconsistency is—to be invariant under inversion. This can be formalized in the following property of invariance under inversion of preferences (P6).

Property 6 (P6).

An inconsistency index satisfies P6, if and only if .

The previous justification of this property can be transposed into an example. Consider the following matrix and its transpose .

(3)

One can equivalently express the structure of the preferences by means of directed weighted graphs with nodes and values of the edges . Figure 3 represents these graphs for and , respectively.

(a) Graph of .
(b) Graph of .
Figure 3: Graphs of and .

The two graphs are identical, with the only exception of the directions of the arrows. Now, if we do not impose P6, we might end up with inconsistency indices which consider the violation of the condition of consistency in the direction more or less (but not equally) important than the violation in the direction , although both directions equivalently reflect the same structure of preferences.

The justification of P6 comes from theoretical intuition, but in some decision processes both a matrix ad its transpose can actually appear. Examples are applications in group decision making where pairwise comparison matrices are used in surveys on customers’ needs and users’ satisfactions, as for instance done by Nikou and Mezei (2013) and Nikou et al. (2015). In these contexts there are as many pairwise comparison matrices as responding customers (usually a large number) and therefore it is not completely unlikely to find both preferences represented by and , especially when there are few alternatives and intensities of preference are weak. i.e., values of entries are close to 1.

Note that, in general, an inversion of preferences cannot be obtained by row-column permutations. For example, given the matrices and in (3), there does not exist a permutation matrix such that .

One natural question is whether or not this new property, P6, is implied by a conjoint application of the others (independence) and if, when added to the set P1–P5, does not make it contradictory (logical consistency). The following theorem claims the independence and the logical consistency of the properties P1–P6.

Theorem 1.

Properties P1–P6 are independent and form a logically consistent axiomatic system.

Proof.

See Appendix. ∎

In light of the previously offered justification and Theorem 1, one concludes that P6 is another interesting property of inconsistency indices, and is independent from P1–P5.

3 Extending the analysis of the satisfaction of the axioms

Previous research (Brunelli and Fedrizzi, 2015a; Cavallo and D’Apuzzo, 2012) has made the effort of proving whether or not some known inconsistency indices satisfy the set of properties. This section continues the investigation on the satisfaction of the set of properties by testing four indices proposed in the literature and used in real-world decision making problems. For each index we shall recall the definition and highlight its relevance in both theory and practice.

3.1 Index by Koczkodaj

The following index, , was introduced by Koczkodaj (1993) and extended by Duszak and Koczkodaj (1994).

Definition 1 (Index (Duszak and Koczkodaj, 1994)).

Given a pairwise comparison matrix , the index is

(4)

This index has been used to estimate missing entries of incomplete pairwise comparisons (Koczkodaj et al., 1999) and in real-world applications in problems such as the evaluation of research institutions in Poland (Koczkodaj et al., 2014) and medical diagnosis (Kakiashvili et al., 2012). It was also compared to Saaty’s Consistency Index (Bozóki and Rapcsák, 2008) and on occasions even claimed superior to it (Koczkodaj and Szwarc, 2014). Given its theoretical and practical relevance, it is therefore important to check what properties it satisfies. Here we show that index satisfies the six properties P1–P6.

Proposition 1.

Index satisfies the properties P1–P6.

Proof.

It is straightforward, and thus omitted, to show that properties P1, P2, P5, and P6 are satisfied. For P3 we need to show that the local inconsistency for the generic transitivity ,

(5)

is non-decreasing for . We can do it by proving that . With , we study the two quantities

If the triple is consistent, then and P3 is satisfied. If the triple is not consistent, then and positive, and the derivatives of I and II in are:

Given , if , then and are positive, which proves that (5) is a non-decreasing function for . It follows that also is a non-decreasing function of .
To prove the satisfaction of P4 we start considering

(6)

with . By setting we can rewrite it as

and show that it is a non-decreasing function for and a non-increasing function for . We then need to study the following quantities:

and their derivatives in

By studying their sign we can derive that

Similarly, P4 can be proven also in the case when the exponent is at the denominator of in (6). ∎

3.2 Index by Salo and Hämäläinen

Salo and Hämäläinen (1995, 1997) proposed their inconsistency index, , which stands for ambiguity index. Their inconsistency index has been implemented in the online decision making platform Web-HIPRE (Mustajoki and Hämäläinen, 2000) and has been used, for instance, in the analysis of a real-world governmental decision on energy production alternatives (Salo and Hämäläinen, 1995) and in traffic planning (Hämäläinen and Pöyhönen, 1996).

Definition 2.

Given a pairwise comparison matrix and an auxiliary matrix with , then the index is

(7)

The interpretation of is original and different from those of other indices. Consider that is not a real number but, instead, the set of possible values of as could be deduced from indirect comparisons .
For example, given the matrix

we have

from which we obtain and .
It is possible to build an interval-valued matrix

such that the ‘true value’ of the comparison between and shall lie in the interval . The larger the intervals are, the more inconsistent the matrix, and in fact is a normalized sum of the lengths of the intervals . The following shows that satisfies all properties except P3.

Proposition 2.

Index satisfies P1, P2 and P4–P6, but not P3.

Proof.

We shall prove all the properties separately.

P1

: Assuming , then we should prove .
(): As all the terms of the sum in (7) are non-negative, if , then they must all be equal to zero. Such terms equal zero only when all the numerators equal zero, i.e. when , which implies that .
(): If , then all the elements are singletons and therefore , implying that the numerators in (7) equals zero and

P2

: Straightforward.

P3

: It is sufficient to consider the following matrix and its derived and

(8)

and observe that and . Hence and P3 is not satisfied.

P4

: Given , a consistent pairwise comparison matrix of order can be equivalently written as

(9)

or, more compactly, as where

Then, each element of the auxiliary matrix is as follows

Now, to test P4, without loss of generality, we fix the pair and replace and with and , respectively. Consequently, and are replaced by and . Hence, for all

Considering the definition of we reckon that the terms associated with for equals zero. Therefore, we shall prove that all the other terms are non-decreasing functions of . We can rewrite

and with , it boils down to prove that

(10)

is a non-decreasing function of when also . Now we should examine the two cases (i) and (ii) . We start with and, considering that

and that therefore, for the case , is always an increasing function of . Hence, we can substitute with and (10) can be replaced by

(11)

Considering that we are in the case with , we simplify (11), and obtain

(12)

So now we shall prove that is positive for all .

This last quantity is always positive for . A very similar result can be derived for the case (ii) and thus satisfies P4.

It can be checked that also satisfies properties P5 and P6. ∎

Although in its present form index does not satisfy P3, the underlying idea is ingenious and it is sufficient to adjust it, i.e. discard the normalization at the denominator, to make it satisfy P3.

Proposition 3.

The inconsistency index

satisfies properties P1–P6.

Proof.

It follows from the proof of Proposition 2 that satisfies P1, P2, P4, P5, and P6. To show that P3 is satisfied, it is sufficient to take the arguments of the sum and consider that they are all non-negative, since . Consequently, the terms are monotone non-decreasing functions with respect to , and P3 is satisfied. ∎

Example 1.

Consider the pairwise comparison matrix in (8) and its associated . Figure 4 contains the plots of and for as functions of and shows their different behaviors.

(a) Index can be decreasing w.r.t , and even tend to , when .
(b) Index is monotone non-decreasing w.r.t. .
Figure 4: Comparison between and with respect to P3.

3.3 Index by Wu and Xu

Wu and Xu (2012) defined their inconsistency index using some properties of the Hadamard product of positive matrices.

Definition 3 (Index by Wu and Xu (2012)).

The index defined by Wu and Xu is

where .

Note that the matrix can be interpreted as a consistent approximation of . In the original paper was used in a mathematical model to manage consistency and consensus at once. Until now, no formal or numerical analysis has been made on and there is no information on its properties. However, with the following proposition we show that it satisfies P1–P6.

Proposition 4.

Index satisfies the properties P1–P6.

Proof.

We shall show that P1 is satisfied, with . First we need to prove that .

Now it can be seen that each function attains its global minimum, equal to , when , which is a restatement of the consistency condition. In this case, to receive the hint that , it is enough to simplify the sum,

Now in the other direction, , it suffices to expand

Since consistency implies we have

It is simple, and thus omitted, to show that P2, P5 and P6 hold. To prove P3, we shall call . By expanding ,

Since , by summing and we obtain

which is an increasing function for . Since this holds for the general pair of indices , the index satisfies P3.
To prove P4, assume, without loss of generality, that the element to be modified is . For sake of simplicity, we can modify it and its reciprocal by multiplying them by . All the with will be equal to 1. For the entries with one index equal to either or we have

meaning that . As we know that , by summing and and simplifying, one obtains

which is a strictly convex function for with minimum in . Similarly, for and , it is

which shares the same property. ∎

3.4 Cosine Consistency Index and other indices

Many times it is not easy to prove whether an index satisfies some properties, but numerical tests and counterexamples can always be used to show that the index does not. This was the case with the Cosine Consistency Index.

Definition 4 (Cosine Consistency Index (Kou and Liu, 2014)).

The Cosine Consistency Index is

where .

Note that and its interpretation is reversed, meaning that the greater its value the less inconsistent is. It is simple, and it can also be found in the original paper, to show that satisfies P1, P2, P5, and P6. For instance, in the case of P1, the proof comes directly from Equation 6 and Theorem 3 in the paper by Kou and Liu (2014). However, the following counterexample suffices to show that does not satisfy P3.

Example 2.

Consider the matrix

(13)

and its associated . The plot of is reported in Figure 5 and shows that does not satisfy P3.

Figure 5: For the matrix in (13), an intensification of preferences decreases the inconsistency.

Often, although in their present forms they do not satisfy P1–P6, ideas behind indices are valid and slight modifications are sufficient to make them satisfy a set of properties. One example is the index proposed by Ramík and Korviny (2010) which was later studied (Brunelli, 2011). Another concrete example is the Relative Error index by Barzilai (1998) which does not satisfy P4 and P5. In its original formulation such index is

where and , and where the denominator acts as a normalization factor. Here it can be proved that, if we discard the denominator, we obtain

which, unlike , satisfies all the properties.

Proposition 5.

Index satisfies all the properties P1–P6.

Proof.

It is easy to show, and therefore omitted, that P1, P2, P5 and P6 are satisfied. We shall prove P3 and P4 separately. To prove the satisfaction of P3 we consider and note that applying the logarithmic transformation to its entries, we obtain . Hence,

which implies that is monotone non-decreasing for .
To show that P4 holds, let us consider the matrix , and its associated . P4 can equivalently be restated as the property that, if we take an entry and its reciprocal and substitute them with and , respectively, then the inconsistency index is a quasi-convex function of with minimum in . From the proof of Proposition 5 in (Brunelli and Fedrizzi, 2015a) one recovers that, by introducing , it is

Thus one obtains,

which is a decreasing function of for and an increasing function for . ∎

4 Discussion and conclusions

Choosing the most suitable inconsistency index is of considerable importance, yet formal studies had not been undertaken until very recently (Brunelli and Fedrizzi, 2015a; Koczkodaj and Szwarc, 2014). This is in contrast with the existence of long-standing studies on other aspects of pairwise comparisons. One of these is the choice of the method for deriving the priority vector, for which axiomatic studies have been proposed in the literature already in the Eighties (Cook and Kress, 1988; Fichtner, 1986) and in the Nineties (Barzilai, 1997). Nevertheless, it has been shown by numerical studies (Ishizaka and Lusti, 2006) that, excepts for some particular cases, such differences can be negligible and that therefore, in most of the cases, choosing one method or another does not really influence the final outcome.

In light of the recently proposed five properties for inconsistency indices, the contribution of this research is at least twofold:

  • Firstly, it introduces and justifies a sixth property (P6) and shows that, together with the other five, it forms an an independent and logically consistent set of properties.

  • Secondly, the paper further analyzes the satisfaction of the properties P1–P6. Four inconsistency indices have been considered from the literature and it was found that two of them fail to fully satisfy the set of properties P1–P6. A simple adjustment of one of these indices was proposed to make it fit P1–P6.

Table 1 presents a summary of the findings of this research and shows how they expanded the original set of properties for inconsistency indices (Brunelli and Fedrizzi, 2015a). It is remarkable that, in the form in which they were originally introduced in the literature, the majority of the indices satisfy only some of them. This seems to indicate that the definition of the properties and the analysis of their satisfaction is not a mere theoretical exercise.

P1 P2 P3 P4 P5 P6
\cdashline1-6 (Definition 1)
(Definition 2)
(Definition 3)
(Definition 4)
Table 1: Summary of propositions: ✓=property is satisfied, ✗=property is not satisfied, ‘—’=unknown. The original results presented in this research are separated from previous ones (Brunelli and Fedrizzi, 2015a) by the dashed lines.

The properties were here, and in previous research (Brunelli and Fedrizzi, 2015a), justified. Nevertheless, clearly, this should not prevent anyone from criticizing and improving them: it is indeed desirable that a set of properties be openly discussed within a community. In this direction, if the system P1–P6 is considered too restrictive, it is worth noting that Theorem 1 implies that any subset of the properties P1–P6 also forms an independent and logically consistent set or properties (just a more relaxed one) which, indeed, can be used for the same purposes of P1–P6. In conclusion, it is the author’s belief that a systematic study of inconsistency and inconsistency indices may bring new insights and more formal order into the evergreen topic of rational decision making. Furthermore, in the future, it should be possible to extend the set of properties to other types of numerical representations of preferences as, for instance, reciprocal preference relations (Tanino, 1984) and skew-symmetric additive representations (Fishburn, 1999).

Acknowledgements

The author is grateful to the reviewers and the Associate Editor for their precious comments. The manuscript benefited from the author’s discussions with Michele Fedrizzi and Ragnar Freij. A special mention goes to Sándor Bozóki who also had the intuition that a property might have been missing. This research has been financially supported by the Academy of Finland.

plus 0.3ex

References

  • Aguarón and Moreno-Jiménez (2003) Aguarón, J., & Moreno-Jiménez, J. M. (2003). The geometric consistency index: Approximated thresholds. European Journal of Operational Research, 147(1), 137–145.
  • Barzilai (1997) Barzilai, J. (1997). Deriving weights from pairwise comparison matrices. The Journal of the Operational Research Society, 48(12), 1226–1232.
  • Barzilai (1998) Barzilai, J. (1998). Consistency measures for pairwise comparison matrices. Journal of Multi-Criteria Decision Analysis, 7(3), 123–132.
  • Belton and Gear (1983) Belton, V., & Gear, T., (1983). On a short-coming of Saaty’s method of analytic hierarchies. Omega, 11(3), 228–230.
  • Bozóki and Rapcsák (2008) Bozóki, S., & Rapcsák, T. (2008). On Saaty’s and Koczkodaj’s inconsistencies of pairwise comparison matrices. Journal of Global Optimization, 42(2), 157–175.
  • Bozóki et al (2013) Bozóki, S., Dezső, L., Poesz, A., & Temesi, J. (2013). Analysis of pairwise comparison matrices: an empirical research. Annals of Operations Research, 211(1), 511–528.
  • Brunelli (2011) Brunelli, M. (2011). A note on the article “Inconsistency of pair-wise comparison matrix with fuzzy elements based on geometric mean” [Fuzzy Sets and Systems 161 (2010) 1604-1613]. Fuzzy Sets and Systems, 176(1), 76–78.
  • Brunelli et al (2013a) Brunelli, M., Canal, L., & Fedrizzi M. (2013a). Inconsistency indices for pairwise comparison matrices: a numerical study. Annals of Operations Research, 211(1), 493–509.
  • Brunelli et al. (2013b) Brunelli, M., Critch, A., & Fedrizzi M. (2013b). A note on the proportionality between some consistency indices in the AHP. Applied Mathematics and Computation, 219(14), 7901–7906.
  • Brunelli and Fedrizzi (2015a) Brunelli, M., & Fedrizzi, M. (2015a). Axiomatic properties of inconsistency indices for pairwise comparisons. Journal of the Operational Research Society, 66(1), 1–15.
  • Brunelli and Fedrizzi (2015b) Brunelli, M., & Fedrizzi, M. (2015b). Boundary properties of the inconsistency of pairwise comparisons in group decisions. European Journal of Operational Research, 230(3), 765–773.
  • Cavallo and D’Apuzzo (2009) Cavallo, B., & D’Apuzzo, L. (2009). A general unified framework for pairwise comparison matrices in multicriterial methods. International Journal of Intelligent Systems, 24(4), 377–398.
  • Cavallo and D’Apuzzo (2012) Cavallo, B., & D’Apuzzo, L. (2012). Investigating properties of the -consistency index. In: Advances in Computational Intelligence. Communications in Computer and Information Science, Vol. 4, pp. 315–327.
  • Chen et al. (2015) Chen, K., Kou, G., Tarn, J.M., & Song, Y. (2015). Bridging the gap between missing and inconsistent values in eliciting preference from pairwise comparison matrices. Annals of Operations Research, 235(1), 155–175.
  • Cook and Kress (1988) Cook, W. D., & Kress, M (1988). Deriving weights from pairwise comparison ratio matrices: An axiomatic approach. European Journal of Operational Research, 37(3), 355–362.
  • Duszak and Koczkodaj (1994) Duszak, Z., & Koczkodaj, W. W. (1994). Generalization of a new definition of consistency for pairwise comparisons. Information Processing Letters, 52(5), 273–276.
  • Dyer (1990a) Dyer, J. S. (1990a). Remarks on the analytic hierarchy process. Management Science, 36(3), 249–258.
  • Dyer (1990b) Dyer, J. S. (1990b). A clarification of “Remarks on the analytic hierarchy process”. Management Science, 36(3), 274–275.
  • Ergu et al. (2011) Ergu, D., Kou, G, Peng, Y, & Shi, Y. (2011). A simple method to improve the consistency ratio of the pair-wise comparison matrix in ANP. European Journal of Operational Research, 213(1), 246–259.
  • Fichtner (1986) Fichtner, J. (1986). On deriving priority vectors from matrices of pairwise comparisons. Socio-Economic Planning Sciences, 20(6), 341–345.
  • Fishburn (1968) Fishburn, P. C. (1968). Utility theory. Management Science, 14(5), 335–378.
  • Fishburn (1999) Fishburn, P. C. (1999). Preference relations and their numerical representations. Theoretical Computer Science, 217(2), 359–383.
  • Gass (2005) Gass, S. I. (2005). Model world: The great debate — MAUT versus AHP. Interfaces, 35(4), 308–312.
  • Hämäläinen and Pöyhönen (1996) Hämäläinen, R. P., & Pöyhönen, M. (1996). On-line group decision support by preference programming in traffic planning. Group Decision and Negotiation, 5(4–6), 485–500.
  • Herman and Koczkodaj (1996) Herman, M. W., Koczkodaj, W. W. (1996). A Monte Carlo study of pairwise comparison. Information Processing Letters, 57(1), 25–29.
  • Irwin (1958) Irwin, F. W. (1958). An analysis of the concepts of discrimination and preference. The American Journal of Psychology, 71(1), 152–163.
  • Ishizaka and Lusti (2006) Ishizaka, A., & Lusti, M. (2006). How to derive priorities in AHP: a comparative study. Central European Journal of Operations Research, 14(4), 387–400.
  • Kakiashvili et al. (2012) Kakiashvili, T., Koczkodaj, W. W., & Woodbury-Smith, M. (2012). Improving the medical scale predictability by the pairwise comparisons method: Evidence from a clinical data study. Computer Methods and Programs in Biomedicine, 105(3):210–216.
  • Koczkodaj and Szwarc (2014) Koczkodaj, W., & Szwarc, R. (2014). On axiomatization of inconsistency indicators in pairwise comparisons. Fundamenta Informaticae, 132(4), 485–500.
  • Koczkodaj (1993) Koczkodaj, W. W. (1993). A new definition of consistency of pairwise comparisons. Mathematical and Computer Modelling, 18(7), 79–84.
  • Koczkodaj et al. (1999) Koczkodaj, W. W., Herman, M. W., & Orlowski, M. (1999). Managing null entries in pairwise comparisons. Knowledge and Information Systems, 1(1), 119–125.
  • Koczkodaj et al. (2014) Koczkodaj, W. W., Kulakowski, K., & Ligeza, A. (2014). On the quality evaluation of scientific entities in Poland supported by consistency-driven pairwise comparisons method. Scientometrics, 99(3), 911–926.
  • Kou and Liu (2014) Kou, G., & Lin, C. (2014). A cosine maximization method for the priority vector derivation in AHP. European Journal of Operational Research, 235(1), 225–232.
  • Kułakowski (2015) Kułakowski, K. (2015). Notes on order preservation and consistency in AHP. European Journal of Operational Research, 245(1), 333–337.
  • Lamata and Pelaez (2002) Lamata, M. T., & Peláez, J. I. (2002). A method for improving the consistency of judgements. International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems, 10(6), 677–686.
  • Lin et al. (2013) Lin, C., Kou, G., & Daji, E. (2013). An improved statistical approach for consistency test in AHP. Annals of Operations Research, 211(1), 289–299.
  • Luce and Suppes (1965) Luce, R. D., & Suppes, P. (1965). Preference, utility and subjective probability. In: R.D. Luce, R.R. Bush,  E.H. Galanter (Eds.), Handbook of Mathematical Psychology, pp 249–410.
  • Luce and Raiffa (1957) Luce, R. D., & Raiffa, H. (1957). Games and Decisions. John Wiley and Sons.
  • Maleki and Zahir (2013) Maleki, H., Zahir, S. (2013). A comprehensive literature review of the rank reversal phenomenon in the analytic hierarchy process. Journal of Multi-Criteria Decision Analysis, 20(3-4), 141–155.
  • Mustajoki and Hämäläinen (2000) Mustajoki, J., & Hämäläinen, R. P. (2000). Web-HIPRE: global decision support by value tree and AHP analysis. INFOR Journal, 38(3), 208–220.
  • Nikou and Mezei (2013) Nikou, S., & Mezei, J. (2013). Evaluation of mobile services and substantial adoption factors with Analytic Hierarchy Process (AHP) analysis. Telecommunications Policy, 37(10), 915–929.
  • Nikou et al. (2015) Nikou, S., Mezei, J., & Sarlin, P. (2015). A process view to evaluate and understand preference elicitation. Journal of Multi-Criteria Decision Analysis, 22(5–6), 305–329.
  • Pereira and Costa (2015) Pereira, V., & Costa, H. G. (2015). Nonlinear programming applied to the reduction of inconsistency in the AHP method. Annals of Operations Research, 229(1), 635–655.
  • Ramík and Korviny (2010) Ramík, J., & Korviny, P. (2010). Inconsistency of pair-wise comparison matrix with fuzzy elements based on geometric mean. Fuzzy Sets and Systems, 161(11), 1604–1613.
  • Saaty (1993) Saaty, T. L. (1993). What is relative measurement? The ratio scale phantom. Mathematical and Computer Modelling, 17(4), 1–12.
  • Saaty (2013) Saaty, T. L. (2013). The modern science of multicriteria decision making and its practical applications: The AHP/ANP approach. Operations Research, 61(5), 1101–1118.
  • Salo and Hämäläinen (1995) Salo, A. A., & Hämäläinen, R. P. (1995). Preference programming through approximate ratio comparisons. European Journal of Operational Research, 82(3), 458–475.
  • Salo and Hämäläinen (1997) Salo, A. A., & Hämäläinen, R. P. (1997). On the measurement of preferences in the analytic hierarchy process. Journal of Multi-Criteria Decision Analysis, 6(6), 309–319.
  • Shiraishi et al. (1999) Shiraishi, S., Obata, T., Daigo, M., & Nakajima, N. (1999). Assessment for an incomplete comparison matrix and improvement of an inconsistent comparison: computational experiments. In ISAHP 1999.
  • Stein and Mizzi (2007) Stein, W. E., & Mizzi, P. J. (2007). The harmonic consistency index for the analytic hierarchy process. European Journal of Operational Research, 177(1):488–497, 2007.
  • Tanino (1984) Tanino, T. (1984). Fuzzy preference orderings in group decision making. Fuzzy Sets and Systems, 12(2), 117–131.
  • Watson and Freeling (1982) Watson, S. R., & Freeling, A. N. S. (1982). Assessing attribute weights. Omega, 10(6), 582–583.
  • Watson and Freeling (1983) Watson, S. R., & Freeling, A. N. S. (1983). Comment on: assessing attribute weights by ratios. Omega, 11(1), 13.
  • Wu and Xu (2012) Wu, Z., & Xu, J. (2012). A consistency and consensus based decision support model for group decision making with multiplicative preference relations. Decision Support Systems, 52(3), 757–767.
  • (55)

Appendix: Proof of Theorem 1

To prove logical consistency, it is sufficient to find an instance of which satisfies all the properties P1–P6. One such instance is the following function

(14)

To prove the independence of P1–P6, it is sufficient to find a function satisfying all properties except one, for all the properties. The examples of inconsistency indices proposed by Brunelli and Fedrizzi (2015a) to prove the independence of the system P1–P5 are invariant under transposition. If follows that P1–P5 are logically independent within the system P1–P6. It remains to show that P6 does not depend on P1–P5. Consider that, if has one row, say , whose non-diagonal elements are all greater than one, i.e. , then this property is shared by any matrix , where is any permutation matrix, but not by its transpose . Taking into account the inconsistency index in (14), and defining as the row with the greatest non-diagonal element, then the function

(15)

is invariant under row-column permutation but not under transposition. Hence, satisfies AP but not P6. To prove the independence of P6, it remains to show that (15) satisfies P1 and P3–P5. It is easy, and thus omitted, to show that P1, P3, and P5 are satisfied. To prove it for P4, we note that any can be rewritten as