Axiomatic properties of inconsistency indices for pairwise comparisons

# Axiomatic properties of inconsistency indices for pairwise comparisons

Matteo Brunelli
Systems Analysis Laboratory, Department of Mathematics and Systems Analysis
Aalto University
P.O. box 11100, FIN-00076 Aalto, Finland
e–mail: matteo.brunelli@aalto.fi
Michele Fedrizzi
Department of Industrial Engineering
University of Trento
Via Mesiano 77, I-38123 Trento, Italy
e–mail: michele.fedrizzi@unitn.it

Abstract

Pairwise comparisons are a well-known method for the representation of the subjective preferences of a decision maker. Evaluating their inconsistency has been a widely studied and discussed topic and several indices have been proposed in the literature to perform this task. Since an acceptable level of consistency is closely related with the reliability of preferences, a suitable choice of an inconsistency index is a crucial phase in decision making processes. The use of different methods for measuring consistency must be carefully evaluated, as it can affect the decision outcome in practical applications. In this paper, we present five axioms aimed at characterizing inconsistency indices. In addition, we prove that some of the indices proposed in the literature satisfy these axioms, while others do not, and therefore, in our view, they may fail to correctly evaluate inconsistency.

Keywords: Pairwise comparisons, inconsistency indices, axiomatic properties, analytic hierarchy process.

## 1 Introduction

Pairwise comparisons have been used in some operations research methods to represent the preferences of experts and decision makers over sets of alternatives, criteria, features and so on. For simplicity, in this paper we shall speak of alternatives only, bearing in mind that it is a reductive view. The main advantage in using pairwise comparisons is that they allow the decision maker to compare two alternatives at a time, thus reducing the complexity of a decision making problem, especially when the set under consideration is large, and serve as a starting point to derive a priority vector which is the final rating of the alternatives. Pairwise comparisons have been used in well-known decision analysis methods as, for instance, the Analytic Hierarchy Process (AHP) by Saaty (1977) (see Ishizaka and Labib (2011) for an updated discussion), and its generalizations, which have been proved effective in solving many decision problems (Ishizaka et al., 2011).

In the literature, and in practice, it is assumed that the dependability of the decision is related to the consistency of his/her pairwise judgments. That is, the more rational the judgments are, the more likely it is that the decision maker is a good expert with a deep insight into the problem and pays due attention in eliciting his/her preferences. Similarly, if judgments are very intransitive and irrational, it is more plausible that the expert expressed them with scarce competence, since he/she would lack the ability to rationally discriminate between different alternatives. This is summarized by Irwin’s thesis claiming that “…preference is exactly as fundamental as discrimination and that if the organism exhibits a discrimination, it must also exhibit a preference and conversely” (Irwin, 1958). Following Saaty (1994) the approach to decision making based on pairwise comparisons, and the AHP in particular, is grounded in the relative measurement theory and it is in this framework that Saaty (1993) too claimed that pairwise comparisons should be ‘near consistent’ to ensure that they are a sufficiently good approximation of the decision makers’ real preferences. This seems to support the importance of having reliable tools capable of capturing the degree of inconsistency of pairwise comparisons. The importance of having reliable inconsistency indices becomes even more evident when one considers that their practical use has gone beyond the sole quantification of inconsistency. For instance, they have been employed by Lamata and Peláez (2002) and Shiraishi et al. (1999) to estimate missing comparisons, by Harker (1987) to derive ratings of alternatives from incomplete preferences and by Xu and Cuiping (1999) and Xu and Xia (2013) to improve the consistency of pairwise comparisons.

On this fertile ground, researchers have proposed various inconsistency indices—functions associating pairwise comparisons to real numbers representing the degrees of inconsistency of the pairwise judgments. In this paper we concern ourselves with the fact that inconsistency indices have been introduced heuristically and independently from each other, neither referring to a general definition, nor to a set of axiomatic properties. Hence, this paper introduces some axiomatic properties for inconsistency indices and shows that some indices proposed in literature fail to satisfy these axioms. This paper is outlined as follows. In Section 2 we introduce preliminary notions and the notation. In Section 3 we shortly define the inconsistency indices that are studied in this paper. Next, in Section 4 we introduce and interpret five axioms and in Section 5 we present results regarding the inconsistency indices and prove that some of them satisfy the required axioms while four others do not. For a simpler description, some proofs are given in the appendix. In Section 6, we conclude the discussion of the axioms and draw the conclusions.
Throughout this paper, we refer to ‘inconsistency indices’, since what they really measure is the amount of inconsistency in pairwise comparisons. Nevertheless, in literature such indices are often referred to as ‘consistency indices’, while both expressions refer to an index which estimates the deviation from consistency.

## 2 Preliminaries

Pairwise comparison matrices are convenient tools to model the decision makers’ pairwise intensities of preference over sets of alternatives. Formally, given a set of alternatives , Saaty (1977) defined a pairwise comparison matrix as a positive and reciprocal square matrix of order , i.e. , where is an estimation of the degree of preference of over . A pairwise comparison matrix is consistent if and only if the following transitivity condition holds:

 aik=aijajk ∀i,j,k. (1)

Property (1) means that preferences are fully coherent, and each direct comparison between and is confirmed by all indirect comparisons . If and only if is consistent, then there exists a priority (or weight) vector such that

 aij=wiwj ∀i,j. (2)

Crawford and Williams (1985) proved that if is consistent, then the components of vector can be obtained by using the geometric mean method,

 wi=(n∏j=1aij)1n ∀i. (3)

Another method for obtaining the priorities is the eigenvector method by Saaty (1977). Namely, the priority vector is the solution of the following equation

 Aw=λmaxw, (4)

where is the maximum eigenvalue of whose existence and properties refer to the Perron-Frobenius theorem. If is consistent, both methods yield the same priority vector, while they may give different vectors if is not consistent.
We define the set of all pairwise comparison matrices as

 A={A=(aij)n×n|aij>0,aijaji=1 ∀i,j, n>2}.

Similarly, the set of consistent pairwise comparison matrices is defined as

 A∗={A=(aij)n×n|A∈A,aik=aijajk ∀i,j,k}.

Seen from this perspective, a matrix can either be consistent or non-consistent (inconsistent). However, often, degrees of inconsistency are assigned to pairwise comparison matrices so that, if the inconsistency is not too high, the judgments in the pairwise comparison matrix are taken to be as sufficiently reliable. To sum up, the idea is that a good inconsistency index should indicate ‘how much’ the pairwise comparison matrix deviates from the full consistency. Thus, an inconsistency index is a real-valued function

 I:A→R. (5)

Although the codomain of the definition is the set of real numbers, each inconsistency index is univocally associated to a given image .

## 3 Inconsistency indices

In this section we shortly recall some inconsistency indices, giving a special emphasis to those which will be analyzed in the next section with respect to the five axioms. For a survey the reader can refer to Brunelli et al. (2013a). The first index is the Consistency Index, proposed by Saaty (1977).

###### Definition 1 (Consistency Index (Saaty, 1977)).

Given a pairwise comparison matrix , the Consistency Index is defined as

 CI(A)=λmax−nn−1, (6)

where is the principal right eigenvalue of .

Formula (6) refers to the property that the maximum eigenvalue of a pairwise comparison matrix is equal to if and only if the matrix is consistent, and greater than otherwise. Saaty proposed also a more suitable measure of inconsistency, called Consistency Ratio (CR),

 CR(A)=CI(A)RI (7)

where , Random Index, is a suitable normalization factor.

Golden and Wang (1989) proposed a method to compute the deviations between the entries of a pairwise comparison matrix and their theoretical values .

###### Definition 2 (Index Gw(Golden and Wang, 1989)).

Given a pairwise comparison matrix , the entries of every column are normalized by dividing them by the sum of the elements of their column . Let us denote by the new normalized matrix. Each priority vector associated (by either (3) or (4)) with is normalized by dividing each component by the sum of the components and denoted by , so that . The inconsistency index is defined as

 GW(A)=1nn∑i=1n∑j=1|¯aij−¯wi|. (8)

Crawford and Williams (1985), and later Aguaròn and Moreno-Jimènez (2003), proposed and refined an index that also computes distances between the decision maker’s judgments and their theoretical values obtained as ratios .

###### Definition 3 (Geometric Consistency Index (Aguaròn and Moreno-Jimènez, 2003)).

Given a pairwise comparison matrix of order , the Geometric Consistency Index is defined as follows

 GCI(A)=2(n−1)(n−2)n−1∑i=1n∑j=i+1ln2(aijwjwi), (9)

where the weights are obtained by means of the geometric mean method (3).

Barzilai (1998) formulated a normalized index based on squared errors. By using open unbounded scales, he stated several relevant algebraic and geometric properties.

###### Definition 4 (Relative Error (Barzilai, 1998)).

Given a pairwise comparison matrix , the relative error, , is defined as

 RE(A)=1−n∑i=1n∑j=1(1nn∑k=1logaik−1nn∑k=1logajk)2n∑i=1n∑j=1(logaij)2. (10)

for all matrices , and zero if .

Peláez and Lamata (2003) defined an inconsistency index for a pairwise comparison matrix as the average of all the determinants of its submatrices, each containing a different transitivity of the original matrix.

###### Definition 5 (Index CI∗(Peláez and Lamata, 2003)).

Given a pairwise comparison matrix of order , the index is

 CI∗(A)=n−2∑i=1n−1∑j=i+1n∑k=j+1(aikaijajk+aijajkaik−2)/(n)3. (11)

Shiraishi et al. (1998) proposed the coefficient of the characteristic polynomial of as an index of inconsistency. Brunelli et al. (2013b) proved that index is proportional to .

Stein and Mizzi (2007) considered the general result that the columns of a consistent pairwise comparison matrix are proportional, i.e. , if and only if is consistent. Thus, they formulated an index which takes into account how far the columns are from being proportional to each other.

###### Definition 6 (Harmonic Consistency Index (Stein and Mizzi, 2007)).

Let be a pairwise comparison matrix and for . Then, the harmonic consistency index is

 HCI(A)=(HM(A)−n)(n+1)n(n−1), (12)

where is the harmonic mean of :

 HM(A)=n∑nj=11sj. (13)

Koczkodaj (1993) and Duszak and Koczkodaj (1994) introduced a max-min based inconsistency index which was later compared with by Bozóki S. and Rapcsák (2008). Cavallo and D’Apuzzo (2009) characterized pairwise comparison matrices by means of Abelian linearly ordered groups and stated their inconsistency index in this general framework.

Another index, , was introduced by Ramík and Korviny (2010) to estimate the inconsistency of pairwise comparison matrices with elements expressed as triangular fuzzy numbers. Expressing judgments in such a way is popular to account for uncertainties in the decision making process. Nevertheless, pairwise comparison matrices can be seen as special cases of matrices with fuzzy entries and therefore this index can be introduced in the context of pairwise comparison matrices with real entries.

###### Definition 7 (Index NIσn(Ramík and Korviny, 2010)).

Given a real number and a pairwise comparison matrix of order with entries in the interval , the index is defined as

 NIσn(A)=γσnmaxi,j{∣∣∣wiwj−aij∣∣∣},

where the weights are obtained by means of the geometric mean method (3) and

 γσn=⎧⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪⎨⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪⎩1max{σ−σ2−2nn,σ2((2n)2n−2−(2n)nn−2)},if σ<(n2)nn−21max{σ−σ2−2nn,σ2n−2n−σ},if σ≥(n2)nn−2

is a positive normalization factor.

Other notable indices are the parametric method by Osei-Bryson (2006) and the ambiguity index by Salo (1993).

## 4 Axioms

In spite of the large number of indices, the question on how well they estimate inconsistency of pairwise comparisons has been left unanswered. To answer this question, in this section we introduce and justify five properties to narrow the general definition of inconsistency index given in (5) and to shed light on those indices which do not satisfy minimal reasonable requirements. Throughout this and the next sections we are going to propose some examples in order to provide numerical and visual evidence of the necessity of the following axiomatic system.

### Axiom 1: Existence of a unique element representing consistency

With axiom 1 (A1) we require that all the consistent matrices are identified by a unique real value of an inconsistency index. This allows to distinguish between matrices that either belong or do not to . Formally, A1 is as follows.

###### Axiom 1.

An inconsistency index satisfies A1, if and only if

 ∃!ν∈R such that I(A)=ν⇔A∈A∗ (14)
###### Example 1.

The following inconsistency index satisfies A1 with .

 I(A)=n∑i=1n∑j=1n∑k=1|aik−aijajk|.

For sake of simplicity, we assume, without loss of generality, that, for every inconsistency index , the value associated with each consistent matrix is the minimum value of the index: . The assumption is that the more inconsistent is , the greater is . Some already introduced indices assume the opposite. By considering, for example, the index introduced by Shiraishi et al. (1998), it is , while the consistency value is . Nevertheless, in such cases it is sufficient to change the sign of the index to fulfill our assumption.

### Axiom 2: Invariance under permutation of alternatives

It is desirable that an inconsistency index does not depend on the order in which the alternatives are associated with rows and columns of . Therefore, an inconsistency index should be invariant under row-column permutations. To formalize this second axiom (A2), we recall that a permutation matrix is a square binary matrix that has exactly one entry equal to 1 on each row and each column and 0’s elsewhere (see Horn and Johnson 1985). We also recall that is the matrix obtained from through the row-column permutations associated with .

###### Axiom 2.

An inconsistency index satisfies A2, if and only if

 I(PAPT)=I(A)∀A∈A (15)

and for any permutation matrix .

###### Example 2.

Given a pairwise comparison matrix and a permutation matrix

 A=⎛⎜⎝1251/2121/51/21⎞⎟⎠    P=⎛⎜⎝010100001⎞⎟⎠

one obtains

 PAPT=⎛⎜⎝010100001⎞⎟⎠⎛⎜⎝1251/2121/51/21⎞⎟⎠⎛⎜⎝010100001⎞⎟⎠=⎛⎜⎝11/222151/21/51⎞⎟⎠

for which (15) is required to hold.

### Axiom 3: Monotonicity under reciprocity-preserving mapping

Unlike the previous axioms, which were simple regularity conditions imposed to , axiom 3 (A3) is more constraining. The idea is that, if preferences are intensified, then an inconsistency index cannot return a lower value. However, before we formalize it, we describe its meaning. If all the expressed preferences indicate indifference between alternatives, it is , and is consistent. Going farther from this uniformity means having stronger judgments and this should not make their possible inconsistency less evident. In other words, intensifying the preferences (pushing them away from indifference) should not de-emphasize the characteristics of these preferences and their possible contradictions. Clearly, the crucial point is to find a transformation which can intensify preferences and preserve their structure at the same time. In the following, we are going to prove that such a transformation exists and is unique. Given , we denote such transformation with . The newly constructed matrix obtained from by means of must be positive and reciprocal so that it still belongs to . Hence

 ^aji=1/^aij,

which is

 f(aji) =1/f(aij) f(1/aij) =1/f(aij) f(aij)f(1/aij) =1

or, more compactly, with ,

 f(x)f(1/x)=1. (16)

Equation (16) is a special case, for , of the well-known Cauchy functional equation

 f(x)f(y)=f(xy). (17)

In fact, by substituting into (16), it is . Since must be positive, it follows . Then, (16) can also be written in the form . Taking into account that , it is therefore sufficient to assume the continuity of in order to obtain a unique non-trivial solution of (16) (see Aczel 1966)

 f(x)=xb,b∈R. (18)

Therefore, the only continuous transformation preserving reciprocity is (18), i.e. . In the following, we will denote matrix as . Clearly, for each entry is moved farther from indifference value 1, which represents an intensification of preferences:

 b>1,aij>1 ⇒abij>aij>1 b>1,0

The opposite occurs for , thus representing a weakening of the preferences. For full indifference is obtained, , while corresponds to preference reversal. Moreover, transformation (18) is consistency-preserving, i.e. if is consistent, then also is consistent. The proof is straightforward, since from immediately follows . Furthermore, (18) is also the unique consistency-preserving transformation, the proof being similar to the one described above for reciprocity.

To summarize, the only continuous transformation that intensifies preferences and preserves reciprocity (and consistency) is with and then A3 can be formalized.

###### Axiom 3.

Define . Then, an inconsistency index satisfies A3 if and only if

 I(A(b))≥I(A)  ∀b>1,∀A∈A. (19)
###### Example 3.

Consider the following matrix

 A=⎛⎜⎝121/21/21221/21⎞⎟⎠.

Then, modifying entries of by means of function with exponent one obtains the following matrix

 A(3)=⎛⎜ ⎜⎝13231/231/231323231/2313⎞⎟ ⎟⎠=⎛⎜⎝181/81/81881/81⎞⎟⎠.

If an inconsistency index satisfies A3, then it must be . In words, if A3 holds, then cannot be judged less inconsistent than .

Note that transformation has been used for other scopes. Saaty (1977) himself proposed it in his seminal paper to show that his results on consistency were general enough to cover scales other than . Such a function was also employed by Herrera-Viedma et al. (2004) to find a suitable mapping to rescale the entries of a pairwise comparison matrix into the interval and by Fedrizzi and Brunelli (2009) to define consistency-equivalence classes.

### Axiom 4: Monotonicity on single comparisons

Let us consider a consistent matrix with at least one non-diagonal entry . If we increase or decrease the value of , and modify its reciprocal accordingly, then the resulting matrix is not anymore consistent. In fact, in agreement with A1, the resulting matrix will have a degree of inconsistency which exceeds that of the consistent matrix. Axiom 4 (A4) establishes a condition of monotonicity for the inconsistency index with respect to single comparisons by requiring that the larger the change of from its consistent value, the more inconsistent the resulting matrix will be. More formally, given a consistent matrix , let be the inconsistent matrix obtained from A by replacing the entry with , where . Necessarily, must be replaced by to preserve reciprocity. Let be the inconsistent matrix obtained from A by replacing entries and with and respectively. A4 can then be formulated as

 δ′>δ>1⇒I(Apq(δ′))≥I(Apq(δ))δ′<δ<1⇒I(Apq(δ′))≥I(Apq(δ)). (20)

Axiom 4 can be equivalently formalized as follows.

###### Axiom 4.

An inconsistency index satisfies A4, if and only if is a non-decreasing function of for and a non-increasing function of for , for all the and .

###### Example 4.

Consider the consistent matrix

 A=⎛⎜⎝1241/2121/41/21⎞⎟⎠∈A∗.

Then, choosing, for instance, entry and changing its value and the value of its reciprocal accordingly, we obtain

 A′=⎛⎜⎝1251/2121/51/21⎞⎟⎠   A′′=⎛⎜⎝1291/2121/91/21⎞⎟⎠ (21)

If an inconsistency index satisfies A4, then , where the inequality between and becomes strict if A1 holds. Note that, in this example, with and with .

Moreover, we note that A4 formalizes a property proved by Aupetit and Genest (1993) for Saaty’s Consistency Index and considered by the authors as a necessary property. Furthermore, the case of a potentially consistent matrix with one deviating comparison was considered by Bryson (1995) in a property that he called ‘single outlier neutralization’ and by Choo and Wedley (2004) in their comparative study of methods to elicit the weight vector. A4 is also in the spirit of other known axiomatic systems. As examples, Cook and Kress (1988) considered two matrices differing by only one comparison and Kemeny and Snell (1962) proposed a similar axiomatic assumption for the distance between rankings.

### Axiom 5: Continuity

As defined in (5), an inconsistency index is a function of . With this fifth axiom (A5), the continuity of the function is required in the set . More precisely, an index is considered as a function of the variables and continuity of is meant as the continuity of a function of real variables. Axiom 5 can be formalized as follows

###### Axiom 5.

An inconsistency index satisfies A5 if and only if it is a continuous function of the entries of , with .

The importance of continuity in mathematical modelling has origins in the fact that it guarantees that infinitesimal variations in the input only generates an infinitesimal variation of the output, thus excluding functions with ‘jumps’.

### 4.1 Significance of the axioms

Let us briefly discuss the necessity of the five axioms by showing that their violation could result in an unreasonable inconsistency measurement:

• If A1 is violated, two perfectly consistent matrices can have two different numerical consistency evaluations.

• If A2 is violated, different consistency evaluations could be associated to the same set of preferences, simply by renaming of alternatives.

• The effect of violation of A3 is apparent from Example 3. If A3 is not respected, matrix , where inconsistent preferences are reinforced, could be evaluated less inconsistent than .

• Let us consider Example 4 to show the necessity of A4. In the Example, matrix clearly differs from the consistent matrix more than does. As a consequence, cannot be evaluated less inconsistent than .

• As stated by Barzilai (1998), continuity ‘is a reasonable requirement of any measure of amount of inconsistency’. In the proof of Proposition 5, referring to Barzilai’s own index, we will show that a discontinuous index may assign the largest inconsistency evaluation to a matrix which is arbitrarily close to a consistent one.

In spite of the reasonability of A1–A5, they could be suspected of being too weak in order to characterize an inconsistency index. On the contrary, they turn out to be strictly demanding, since Propositions 5, 6, 7, and 8 will surprisingly show that they are not satisfied by four indices based on seemingly reasonable definitions.

### 4.2 Logical consistency and independence

A natural question is whether the axiomatic properties A1–A5 form an axiomatic system or not. In fact, in an axiomatic system, the axioms must be consistent (in a logical sense) and independent. The existence of, at least, one index satisfying A1–A5 proves that the axiomatic system is not logically contradictory and therefore the system is logically consistent. Another important result regards the independence of the axioms. Proving the independence would show that the axioms are not redundant, and therefore all of them shall be considered necessary.

###### Theorem 1.

Axiomatic properties A1–A5 are logically consistent and independent.

###### Proof.

Propositions 1, 2 and 3 in Section 5 state that indices , and , respectively, satisfy all the five properties. Then, the axiomatic properties A1–A5 are logically consistent. Independence of a given axiom can be shown by providing an example of index satisfying all axioms except the one at stake. To prove the independence of A1, one can consider the following ad hoc constructed index,

 I1(A)=max{CI∗(A)−1,0}.

As a consequence of Proposition 2, index satisfies A2–A5. Nevertheless, assigns value also to some inconsistent pairwise comparison matrices, so that it does not satisfy A1. To prove independence of A2, one could instead consider

 I2(A)=n−2∑i=1n−1∑j=i+1n∑k=j+1(aikaijajk+aijajkaik−2)wijk

with and for some and . Index does not satisfy A2, due to the presence of . Conversely, it satisfies all the other axioms, the proof being similar to that of Proposition 2. Independence of A3 directly follows from Proposition 7. To prove independence of A4, we propose the following index,

 I4(A)=(maxi≠jmax{aij,aji}−mini≠jmax{aij,aji}+ϵ)×(CI∗(A))0.1.

It can be proved that fails to satisfy A4 for a convenient choice of and . Finally, it is easy to check that

 I5(A)={0if A∈A∗,1if A∉A∗

satisfies axioms A1–A4, but is not continuous and therefore does not fulfill A5, thus showing its independence. ∎

Next, we shall investigate if existing inconsistency indices—especially those defined in the previous section—satisfy the axioms A1–A5.

## 5 On the satisfaction of the axioms

We first consider three inconsistency indices and prove that they satisfy all the axioms. Only later, we shall prove that some others do not satisfy some axioms. As anticipated in the introduction, most of the proofs are given in the appendix in order to simplify the description.

Saaty’s and indices and satisfy the five axioms A1–A5. We can formalize it in the following propositions.

###### Proposition 1.

Saaty’s Consistency Index (6) satisfies the five axioms A1–A5.

###### Proposition 2.

Index satisfies the five axioms A1–A5.

###### Proposition 3.

The Geometric Consistency Index satisfies the five axioms A1–A5.

Let us now consider Barzilai’s inconsistency index and formulate the following interesting result,

###### Proposition 4.

Let and . Then and therefore is invariant w.r.t. .

Proposition 4 could be seen as a restriction of A3, where an inconsistency index is required to be invariant under function . Clearly, this implies that index satisfies A3. The general result on index is stated by the following proposition

###### Proposition 5.

Index satisfies A1–A3, but it does not satisfy A4 and A5.

Note that Proposition 5 disproves the continuity of claimed in the original paper by Barzilai (1998).

The following proposition concerns index , introduced by Ramík and Korviny (2010), see definition 7. It remains unproved whether satisfies A3 or not.

###### Proposition 6.

Index satisfies axioms A1, A2 and A5 but it does not satisfy A4.

###### Proof.

The proof that satisfies axioms A1 and A2 is straightforward. To prove that does not satisfy A4, let us consider the following consistent pairwise comparison matrix,

 A=⎛⎜ ⎜ ⎜ ⎜⎝11/31/31/93111/33111/39331⎞⎟ ⎟ ⎟ ⎟⎠∈A∗. (22)

If entry is changed, and its reciprocal varies accordingly, then, the violation of A4 can be appreciated in Figure 1, where is plotted as a function of , being and .

In fact, A4 implies that such a function should be monotonically increasing for , but this is not the case in this example, since, e.g., but the value of corresponding to is smaller than the value of corresponding to . Continuity of follows from continuity of , so that A5 is satisfied. ∎

We can only conjecture that the behavior described in Figure 1 is related with the fact, noted by Brunelli (2011), that index fails to identify the most inconsistent matrix.

We next consider another index which fails to satisfy one of the axioms: the Harmonic Consistency Index (12).

###### Proposition 7.

Index satisfies A1, A2, A4 and A5 but it does not satisfy A3.

The following example derives from the proof of proposition 7 and is aimed to clarify the behavior of and to show the importance of A3.

###### Example 5.

Consider the following matrix and its derived matrix

 A=⎛⎜ ⎜ ⎜ ⎜⎝141/221/411/4224121/21/21/21⎞⎟ ⎟ ⎟ ⎟⎠     A(b)=⎛⎜ ⎜ ⎜ ⎜⎝1b4b(1/2)b2b(1/4)b1b(1/4)b2b2b4b1b2b(1/2)b(1/2)b(1/2)b1⎞⎟ ⎟ ⎟ ⎟⎠.

It is possible to illustrate the behavior of by means of Figure 2. The index initially increases, but then it decreases and converges to full -consistency as grows.

The last index considered in this section is the index of Golden and Wang (1989) and the following proposition states the corresponding results. It remains unproved whether satisfies A4 or not.

###### Proposition 8.

Index satisfies A1, A2 and A5. If the priority vector is computed by means of the geometric mean method, then index does not satisfy A3.

The following example derives from the proof of Proposition 8 and, similarly to example 5, shows the convergence to zero of the index .

###### Example 6.

Consider the pairwise comparison matrix obtained from as

 A=⎛⎜ ⎜ ⎜ ⎜⎝131/421/311/7247161/21/21/61⎞⎟ ⎟ ⎟ ⎟⎠    A(b)=⎛⎜ ⎜ ⎜ ⎜ ⎜⎝1b3b(1/4)b2b(1/3)b1b(1/7)b2b4b7b1b6b(1/2)b(1/2)b(1/6)b1b⎞⎟ ⎟ ⎟ ⎟ ⎟⎠ (23)

and note that the third row of contains all the greatest elements of each column. It is possible to plot the behavior of and obtain the graph in Figure 3, which represents the -inconsistency of matrix in (23).

Finally, Table 1 summarizes the findings obtained in this section.

## 6 Discussion and Future Research

Propositions 6, 7 and 8, together with the corresponding examples, suggest that A3 and A4 are the most demanding axioms. Let us make some other remarks to clarify A3 and A4. First, we propose a geometrical interpretation that could be useful to emphasize the role of A4 in requiring the non-decreasing property of an inconsistency index when moving away from consistency. Let us represent a consistent matrix as a point in the Cartesian space , where the dimension is the number of upper-diagonal elements which are necessary and sufficient to identify a pairwise comparison matrix of order . By increasing (decreasing) entry , the point departs from set and moves in the direction of the corresponding axis. Thus, A4 requires that an inconsistency index does not decrease whenever moves away from the initial consistent position, in any of the possible directions. On the other hand, by referring to the same geometrical representation in the Cartesian space, the type of translation of point induced by A3 is different from the one induced by A4, so that the joint effect is more general than the single ones.

In decision problems based on pairwise comparisons there are two phases: preference elicitation and priority vector computation. In the previous sections we defined and studied five axioms characterizing the inconsistency evaluation of the preferences elicited by a decision maker independently from the method used in deriving the priority vector. Therefore, we focused on pairwise comparison matrix and the property of transitivity . Nevertheless, by considering that consistency of a pairwise comparison matrix can equivalently be characterized by property , it is possible and relevant to study also the relationship between the inconsistency indices and the methods used for priority vector computation. Other investigations will add new insight on the relationship between the inconsistency indices and the methods used for computing priority vectors.

### 6.1 Conclusions

The purpose of this paper was to introduce some formal order in the topic of consistency evaluation for pairwise comparison matrices. We proposed few and simply justifiable axiomatic properties to characterize inconsistency indices, discovering that some indices proposed in the literature fail to satisfy some properties. We hope that our proposal will open a debate and stimulate further studies.

## References

• (1)
• Aczel (1966) Aczel J (1966). Lectures on Functional Equations and their Applications. Academic Press: New York.
• Aguaròn and Moreno-Jimènez (2003) Aguaròn, J and Moreno-Jimènez, J M (2003). The geometric consistency index: Approximated threshold. European Journal of Operational Research 147(1): 137–145.
• Aupetit and Genest (1993) Aupetit B and Genest C (1993). On some useful properties of the Perron eigenvalue of a positive reciprocal matrix in the context of the analytic hierarchy process. European Journal of Operational Research 70(2): 263–268
• Barzilai (1998) Barzilai J (1998). Consistency measures for pairwise comparison matrices. Journal of Multi-Criteria Decision Analysis 7(3): 123–132.
• Bozóki S. and Rapcsák (2008) Bozóki S and Rapcsák T (2008). On Saaty’s and Koczkodaj’s inconsistencies of pairwise comparison matrices. Journal of Global Optimization 42(2): 157–175.
• Bozóki et al. (2010) Bozóki S, Fülöp J and Rónyai L (2010). On optimal completion of incomplete pairwise comparison matrices. Mathematical and Computer Modelling 52(1–2): 318–333.
• Brunelli (2011) Brunelli M (2011). A note on the article “Inconsistency of pair-wise comparison matrix with fuzzy elements based on geometric mean” [Fuzzy Sets and Systems 161 (2010) 1604–1613]. Fuzzy Sets and Systems 176(1): 76–78.
• Brunelli et al. (2013a) Brunelli M, Canal L and Fedrizzi M (2013a). Inconsistency indices for pairwise comparison matrices: a numerical study. Annals of Operations Research doi: 10.1007/s10479-013-1329-0.
• Brunelli et al. (2013b) Brunelli M, Critch A and Fedrizzi M (2013b). A note on the proportionality between some consistency indices in the AHP. Applied Mathematics and Computation 219(14): 7901–7906.
• Bryson (1995) Bryson N (1995). A goal programming method for generating priority vectors. Journal of the Operational Research Society 46(5): 641–648.
• Cavallo and D’Apuzzo (2009) Cavallo B and D’Apuzzo L (2009). A general unified framework for pairwise comparison matrices in multicriterial methods. International Journal of Intelligent Systems 24(4): 377–398.
• Choo and Wedley (2004) Choo E U and Wedley W C (2004). A common framework for deriving preference values from pairwise comparison matrices. Computers & Operations Research 31(6): 893–908.
• Cook and Kress (1988) Cook W D and Kress M (1988). Deriving weights from pairwise comparison ratio matrices: An axiomatic approach. European Journal of Operational Research 37(3): 355–362.
• Crawford and Williams (1985) Crawford G and Williams C (1985). A note on the analysis of subjective judgement matrices. Journal of Mathematical Psychology 29(4): 25–40.
• Duszak and Koczkodaj (1994) Duszak Z and Koczkodaj W W (1994). Generalization of a new definition of consistency for pairwise comparisons. Information Processing Letters 52(5): 273–276.
• Fedrizzi and Brunelli (2009) Fedrizzi M and Brunelli M (2009). Fair consistency evaluation in reciprocal relations and group decision making. New Mathematics and Natural Computation 5(2): 407–420.
• Golden and Wang (1989) Golden B L and Wang Q (1989). An alternate measure of consistency. In: Golden B L., Wasil E A and Harker P T (eds), The Analytic Hierarchy Process, Applications and studies, (pp. 68–81), Springer-Verlag: Berlin–Heidelberg.
• Harker (1987) Harker P T (1987). Incomplete pairwise comparisons in the analytic hierarchy process. Mathematical Modelling 9(11): 837–848.
• Harker and Vargas (1987) Harker P T, Vargas L G (1987). The theory of ratio scale estimation: Saaty’s Analytic Hierarchy Process. Management Science 33(11): 1383–1403.
• Herrera-Viedma et al. (2004) Herrera-Viedma E, Herrera F, Chiclana F and Luque M (2004). Some issues on consistency of fuzzy preference relations. European Journal of Operational Research 154(1): 98–109.
• Horn and Johnson (1985) Horn R A and Johnson C R (1985). Matrix Analysis, Cambridge University Press: New York.
• Irwin (1958) Irwin F W (1958). An analysis of the concepts of discrimination and preference. The American Journal of Psychology 71(1): 152–163.
• Ishizaka et al. (2011) Ishizaka A (2011). Does AHP help us make a choice? An experimental evaluation. Journal of the Operational Research Society 62(10): 1801–1812.
• Ishizaka and Labib (2011) Ishizaka A and Labib A (2011). Review of the main developments in the analytic hierarchy process. Expert Systems with Applications 38(11): 14336–14345.
• Kemeny and Snell (1962) Kemeny, J G and Snell J L (1962). Mathematical Models in the Social Sciences. Blaisdell: New York.
• Kingman (1961) Kingman, J F C (1961). A convexity property of positive matrices. The Quarterly Journal of Mathematics. Oxford. Second Series 12(1), 283–284.
• Koczkodaj (1993) Koczkodaj W W (1993). A new definition of consistency of pairwise comparisons. Mathematical and Computer Modelling 18(7): 79–84.
• Lamata and Peláez (2002) Lamata M T and Peláez J I (2002). A method for improving the consistency of judgments. International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems 10(6): 677–686.
• Osei-Bryson (2006) Osei-Bryson N (2006). An action learning approach for assessing the consistency of pairwise comparison data. European Journal of Operational Research 174(1): 234–244.
• Peláez and Lamata (2003) Peláez J I and Lamata M T (2003). A new measure of inconsistency for positive reciprocal matrices. Computer and Mathematics with Applications 46(12): 1839–1845.
• Ramík and Korviny (2010) Ramík J and Korviny P (2010). Inconsistency of pair-wise comparison matrix with fuzzy elements based on geometric mean. Fuzzy Sets and Systems 161(11): 1604–1613.
• Saaty (1977) Saaty T L (1977). A scaling method for priorities in hierarchical structures. Journal of Mathematical Psychology 15(3): 234–281.
• Saaty (1993) Saaty T L (1993). What is relative measurement? The ratio scale phantom. Mathematical and Computer Modelling 17(4–5): 1–12.
• Saaty (1994) Saaty T L (1994). Highlights and critical points in the theory and application of the Analytic Hierarchy Process. European Journal of Operational Research 74(3): 426–447.
• Salo (1993) Salo A A (1993). Inconsistency analysis by approximately specified priorities. Mathematical and Computer Modelling 17(4–5): 123–133.
• Shiraishi et al. (1998) Shiraishi S, Obata T and Daigo M (1998). Properties of a positive reciprocal matrix and their application to AHP. Journal of the Operations Research Society of Japan 41(3): 404–414.
• Shiraishi et al. (1999) Shiraishi S, Obata T, Daigo M and Nakajima N (1999). Assessment for an incomplete matrix and improvement of the inconsistent comparison: computational experiments. Proceedings of ISAHP 1999, Kobe, Japan
• Stein and Mizzi (2007) Stein W E and Mizzi P J (2007). The harmonic consistency index for the analytic hierarchy process. European Journal of Operational Research 177(1): 488–497.
• Xu and Cuiping (1999) Xu Z and Cuiping W (1999). A consistency improving method in the analytic hierarchy process. European Journal of Operational Research 116(2): 443–449.
• Xu and Xia (2013) Xu Z and Xia M (1999). Iterative algorithms for improving consistency of intuitionistic preference relations. Journal of Operational Research Society doi: 10.1057/jors.2012.178.

## Appendix

###### Proof of Proposition 1.

In the following we prove each axiom separately.

A1

This was proved already by Saaty (1977).

A2

It is known that the characteristic polynomial of a matrix equals the characteristic polynomial of the matrix where is any non-singular matrix and its inverse (see Horn and Johnson (1985), p. 45). We also know that any permutation matrix is an orthogonal matrix, and therefore its inverse is its transpose. Thus, using the notation used in (15), we also know that the characteristic polynomial of is the same of . This implies that the index remains unchanged.

A3

The proof relies on a theorem from linear algebra by Kingman (1961) stating that if the elements of a matrix are logconvex functions of , then the maximum eigenvalue is a logconvex (and hence convex) function of . Given a pairwise comparison matrix , let us apply the preference intensifying function . The entries of the obtained matrix are logconvex functions of parameter (see Bozóki et al. 2010). From the theorem, is a convex function of . Moreover, reaches its minimum value for , since is the consistent matrix with all entries equal to one (indifference matrix). It follows that is a non-decreasing function for and hence, in particular, for . The same applies to , being an increasing affine transform of . Therefore, satisfies (19).

A4

It was proved by Aupetit and Genest (1993), that must be either increasing, decreasing or U-shaped as a function of a single upper triangular entry, say , of a positive reciprocal matrix. Let A be a consistent matrix, and let be an arbitrarily fixed entry of A. Then it is . Using the notation of section 4, let be the inconsistent matrix obtained from A by replacing with with a different value , being consequently replaced with . Then becomes larger than , . That is, reaches its minimum value when . It follows that is U-shaped as a function of , reaching its minimum value for . The same applies to , being an increasing affine transform of , which proves that A4 is satisfied.

A5

Continuity holds due to the continuous dependence of the zeroes of a polynomial on its coefficients

###### Proof of Proposition 2.

We shall proceed by proving that it satisfies each axiom:

A1

This axiom is satisfied by as is can be proved that the non-negative quantity

 aikaijajk+aijajkaik−2

reaches its minimum, which is zero, if and only if . The proof is based on the study of the function where . Considering , then reaches its minimum for . This implies .

A2

Observe that: (i) expression (11) considers all the triplets only once; (ii) given a triplet , the terms in (11) corresponding to these indices are independent from the order of the indices, that is

 aikaijajk+aijajkaik=aπ(i)π(k)aπ(i)π(j)aπ(j)π(k)+aπ(i)π(j)aπ(j)π(k)aπ(i)π(k) (24)

holds for any permutation map .

A3

It is possible to prove that each term of the sum (11) corresponding to an inconsistent triplet increases if we apply . In fact, applying one obtains:

 abikabijabjk+abijabjkabik−2=(aikaijajk)b+(aijajkaik)b−2. (25)

It is then necessary to prove that (25) is increasing with respect to . Let us recall that and . Let

 g(x,b)=xb+1xb−2. (26)

We then need to prove that is increasing with respect to , that is

 b2>b1⇒g(x,b2)≥g(x,b1) (27)

for all . It is

 ∂g(x,b)∂b=∂xb∂b+∂x−b∂b=xblnx−x−blnx=(lnx)(xb−1xb). (28)

To study the sign of , one can more simply analyze in , where

 h(z)=z−1z=z2−1z. (29)

Then , and . Consequently, for ,

 00
 x>1 ⇒ lnx>0 ⇒ z=xb>1⇒h(z)=(xb−1xb)>0 ⇒ ∂g(x,b)∂b=(lnx)(xb−1xb)>0.

Finally, if , then

 ∂g(1,b)∂b=0. (30)

Therefore, is increasing respect to for . For , representing consistent triples, we get .

To conclude, each term of the sum (11) corresponding to an inconsistent triplet increases by applying function with , hence

 CI∗(A)=0 ⇒ CI∗(A(b))=CI∗(A)=0 CI∗(A)>0 ⇒ CI∗(A(b))>CI∗(A).
A4

Without loss of generality, let us fix the entry of with , and replace with and with . Considering index (11) for the obtained matrix , some terms of the sum contain , while others do not. Those not containing remain unchanged. Let us first consider the following terms of the sum:

 aδpqapjajq+apjajqaδpq−2. (31)

Since is consistent, we can set