Judgment Aggregation in Multi-Agent Argumentation

Judgment Aggregation in Multi-Agent Argumentation

Edmond Awad Masdar Institute of Science & Technology, UAE Richard Booth Mahasarakham University, Thailand Fernando Tohmé Universidad Nacional del Sur, Argentina Iyad Rahwan
Abstract

Given a set of conflicting arguments, there can exist multiple plausible opinions about which arguments should be accepted, rejected, or deemed undecided. We study the problem of how multiple such judgments can be aggregated. We define the problem by adapting various classical social-choice-theoretic properties for the argumentation domain. We show that while argument-wise plurality voting satisfies many properties, it fails to guarantee the collective rationality of the outcome. We then present more general results, proving multiple impossibility results on the existence of any good aggregation operator. After characterising the sufficient and necessary conditions for satisfying collective rationality, we study whether restricting the domain of argument-wise plurality voting to classical semantics allows us to escape the impossibility result. We close by mentioning a couple of graph-theoretical restrictions under which the argument-wise plurality rule does produce collectively rational outcomes. In addition to identifying fundamental barriers to collective argument evaluation, our results contribute to research at the intersection of the argumentation and computational social choice fields.

1 Introduction

Argumentation has recently become one of the key approaches to automated reasoning and rational interaction in Artificial Intelligence [5, 28]. A key milestone in the development of argumentation in AI has been Dung’s landmark framework [15], known as abstract argumentation framework (AAF). Arguments are viewed as abstract entities (a set ), with a binary defeat relation (denoted ) over them. The defeat relation captures the fact that one argument somehow attacks or undermines another. This view of argumentation enables high-level analysis while abstracting away from the internal structure of individual arguments. In Dung’s approach, given a set of arguments and a defeat relation, a rule specifies which arguments should be accepted.

Often, there are multiple reasonable ways in which an agent may evaluate a given argument structure (e.g. accepting only conflict-free, self-defending sets of arguments). Each possible evaluation corresponds to a so-called extension [15] or labelling [8, 9]. Different argumentation semantics yield different restrictions on the possible extensions. Most previous research has focused on evaluating and comparing different semantics based on the (objective) logical properties of their extensions [3].

One of the essential properties, which is common, is the condition of admissibility: that accepted arguments must not attack one another, and must defend themselves against counter-arguments, by attacking them back. A stronger notion is called completeness, and is captured, in terms of labelling, in the following two conditions:

  1. An argument is labelled accepted (or ) if and only if all its defeaters are rejected (or ).

  2. An argument is labelled rejected (or ) if and only if at least one of its defeaters is accepted (or ).

Otherwise, an argument may be labelled . Thus, evaluating a set of arguments amounts to labelling each argument using a labelling function to capture these three possible labels. Any labelling that satisfies the above conditions is also called a legal labelling. We will often use legal labelling and complete labelling interchangeably.

The above conditions attempt to evaluate arguments from a single point of view. Indeed, most research on formal models of argumentation discounts the fact that argumentation takes place among self-interested agents, who may have conflicting opinions and preferences over which arguments end up being accepted, rejected, or undecided. Consider the following simple example.

Example 1 (A Murder Case).

A murder case is under investigation. To start with, there is an argument that the suspect should be presumed innocent (). However, there is evidence that he may have been at the crime scene at the time (), which would counter the initial presumption of innocence. There is also, however, evidence that the suspect was attending a party that day (). Clearly, and are mutually defeating arguments since the suspect can only be in one place at any given time. Hence, we have a set of arguments and a defeat relation . There are three possible labellings that satisfy the above conditions:

  • , , .

  • , , .

  • , , .

The graph and possible labellings are depicted in Figure 1.

Figure 1: Argument graph with three possible labellings

Example 1 highlights a situation in which multiple points of view can be taken, depending on whether one decides to accept the argument that the suspect was at the party or the crime scene. The question we explore in this paper can be highlighted through the following example, extending Example 1.

Example 2 (Three Detectives).

A team of three detectives, named , , and , have been assigned to the murder case described in Example 1. Each detective’s judgment can only correspond to a legal labelling (otherwise, her judgment can be discarded). Suppose that each detective’s judgment is such that , and . That is, detectives and agree but differ with detective . These labellings are depicted in the labelled graph of Figure 2. The detectives must decide which (aggregated) argument labelling best reflects their collective judgment.

Figure 2: Three detectives with different judgments

Example 2 highlights an aggregation problem, similar to the problem of preference aggregation [2, 16, 33] and the problem of judgment aggregation on propositional formulae [23, 20, 22, 19]. It is perhaps obvious in this particular example that must be rejected (and thus the defendant be considered guilty), since most detectives seem to think so. For the same reason, must be rejected and must be accepted. Thus, labelling (see Example 1) wins by majority. As we shall see in our analysis below, things are not that simple, and counter-intuitive situations may arise. We summarise the main question asked in the paper as follows.

Given a set of agents, each with a specific subjective evaluation (i.e. labelling) of a given set of conflicting arguments, how can agents reach a collective decision on how to evaluate those arguments?

While Arrow’s Impossibility Theorem can be expected to ensue for this problem [1],111Arrow’s Theorem claims that four quite natural constraints, that capture abstractly the properties of a democratic aggregation process, cannot be simultaneously satisfied. there exist many differences between labellings and preference relations (for which Arrow’s result apply), stemming from their corresponding order-theoretic characterisations. In other words, aggregating preferences assumes that agents submit a full order of preferences over candidates, while in labelling aggregation, agents submit their top labelling for a set of logically connected arguments.

The problem of labelling aggregation is more comparable to the judgment aggregation problem [23, 20, 22, 19], by considering arguments as propositions which are logically connected by the conditions of legal labelling. However, one important difference is that in judgment aggregation, each proposition can have two values: True or False. In labelling aggregation, on the other hand, each argument can have three values: , , or . This makes labelling aggregation be more comparable to non-binary evaluations [13, 14]. Considering the general framework in [14], our settings can be considered as focusing on special classes of feasible evaluations, which are the conditions imposed by the legal labelling (or other semantics). Additionally, the possible evaluations of each issue (argument, in our case) are to accept (labels as ), reject (labels as ), or be undecided (labels as ).

In this paper, we conduct an extensive social-choice-theoretic analysis of argument evaluation semantics by means of labellings. We assume that individuals are presented with a shared argumentation framework (AF) and need to make a decision about how to evaluate this AF. Individuals are assumed to have different, but reasonable, evaluations. There can be many scenarios in which such settings are present. For example, consider a jury members that are all provided with the same information, each of them has a different opinion about these information and yet they all need to come up with a collective decision. Another example is a company board committee who need to make an informed decision. They can be all presented with the same information about the current economic status and the possible strategies, each one of them has his/her own opinion about what should be done, yet they all need to reach a collective decision.

The paper makes three distinct contributions to the state-of-the-art in the computational modelling of argumentation. Firstly, the paper introduces the study of aggregating different individual judgments on how a given set of arguments is to be evaluated.222In fact, this idea was first introduced in [29] for which this paper is a substantially extended and revised version. Section 6 which introduces the impossibility of good aggregation operator is significantly enhanced by adding three impossibility results. Sections 8 and 9 are completely new. Section 10 contains more elaborate discussion of related and future work. Finally, further explanation, motivation, discussion and background is added to the other sections to improve clarity and presentation of the paper. This requires adapting classical social-choice properties to the argumentation domain, and sometimes demands special treatment (e.g. different versions of some properties).

The second contribution of this paper is proving the impossibility of the existence of any aggregation operator that satisfies some minimal properties. In doing so, we show impossibility results that concern dealing with ties and producing a collectively rational evaluation of arguments. These results establish the limits of aggregation in the context of argumentation, and come in accordance with the impossibility results in the topics of aggregation such as preference aggregation [1, 31, 25, 17, 30] and judgment aggregation [21]. Hence, as is the case with other aggregation domains, the aggregation paradox in argument evaluation is an example of a more fundamental barrier. These results are important because they give conclusive answers and focus research in more constructive directions (e.g. weakening the desired properties in order to avoid the paradox). Aiming to investigate possible relaxations in order to circumvent the impossibility in the context of argumentation, we broke down the Collective Rationality postulate into sub-postulates. This helps in taking a deeper look at the distinct parts of the postulate. As a consequence, satisfying any of these parts can be used to weaken the collective rationality.

The third contribution of this paper is an extensive analysis of an aggregation rule, namely argument-wise plurality rule. We analyse the properties of the argument-wise plurality rule in general, and investigate whether the restriction of the domain of votes to a particular classical semantics would ensure the fulfillment of these conditions. This highlights a novel use of classical semantics, which are originally used to resolve issues in single-agent nonmonotonic reasoning. Finally, we provide graph-theoretical restrictions on argumentation frameworks under which the argument-wise plurality rule would be guaranteed to produce collectively rational outcomes.

The paper is organised as follows. In section 2, we start by giving a brief background on abstract argumentation systems. Sections 3, 4, 6 and 7 focus on the problem of aggregating sets of judgments over argument evaluation. Sections 5, 8, and 9 focus on introducing and analysing the argument-wise plurality rule. We conclude the paper and discuss some related work in Section 10.

2 Background

In this section, we briefly outline key elements of abstract argumentation frameworks. We begin with Dung’s abstract characterisation of an argumentation system [15]. We restrict ourselves to finite sets of arguments.

Definition 1 (Argumentation framework).

An argumentation framework is a pair where is a finite set of arguments and is a defeat relation. We say that an argument defeats an argument if (sometimes written ).

For an argument , we use to denote the set of arguments that defeat i.e. .

Figure 3: A simple argument graph

An argumentation framework can be represented as a directed graph in which vertices are arguments and directed arcs characterise defeat among arguments. An example argument graph is shown in Figure 3. Argument has two defeaters (i.e. counter-arguments) and , which are themselves defeated by arguments and respectively.

There are two approaches to define semantics that assess the acceptability of arguments. One of them is extension-based semantics by Dung [15], which produces a set of arguments that are accepted together. Another equivalent labelling-based semantics is proposed by Caminada [8, 9], which gives a labelling for each argument. With argument labellings, we can accept arguments (by labelling them as ), reject arguments (by labelling them as ), and abstain from deciding whether to accept or reject (by labelling them as ). Caminada [8, 9] established a correspondence between properties of labellings and the different extensions. In this paper, we employ the labelling approach.

Definition 2 (Argument Labelling).

Let be an argumentation framework. An argument labelling is a total function .

We write (resp. , ) for the set of arguments that are labelled (resp. , ) by . A labelling can be represented as ,,.

However, labellings should follow some given conditions. A minimal reasonable condition is the conflict-freeness.

Definition 3 (Conflict-freeness).

A labelling satisfies conflict-freeness iff , .

One of the essential semantics, which satisfies conflict-freeness is the complete semantics. We already informally defined complete labellings via two conditions in the introduction. We find it convenient to equivalently formulate it as three conditions as follows.

Definition 4 (Complete labelling).

Let be an argumentation framework. A complete labelling is a total function such that:

  • ;

  • ; and

We will use to denote the set of all complete labellings for .

As an example, consider the following.

Example 3.

Consider the graph in Figure 4. Here, we have three complete labellings: , , and .

Figure 4: Graph with three complete labellings.

In addition to the complete labelling, there are other semantics which assume further conditions.

Definition 5 (Other Labelling Semantics).

Let be an argumentation framework. Let be a complete labelling.

  • is a grounded labelling if and only if is minimal, or equivalently is minimal, or equivalently is maximal (w.r.t set inclusion) among all complete labellings.

  • is a preferred labelling if and only if is maximal, or equivalently is maximal (w.r.t set inclusion) among all complete labellings.

  • is a semi-stable labelling if and only if is minimal (w.r.t set inclusion) among all complete labellings.

  • is a stable labelling if and only if .

Note that the grounded labelling is always unique, and stable labellings might not exist. Consider the following example.

Example 4.

Consider the graph in Figure 4. Here, we have the grounded labelling is . We have only two preferred labellings: , and . These are also the only stable and semi-stable labellings for this framework.

Clearly, for any , , and , where , , , and refer to the set of stable, semi-stable, preferred, and grounded labellings for . We refer to the previous semantics as classical semantics. There exist other semantics which we do not consider in this work.

3 Aggregation of Argument Labellings

To date, most analyses inspired by Dung’s framework have focused on analysing and comparing the properties of various types of extensions/labellings (i.e. semantics) [3]. The question is, therefore, whether a particular type of labelling is appropriate for a particular type of reasoning task in the presence of conflicting arguments.

In contrast with most existing work on Dung frameworks, our concern here is with multi-agent systems. Since each labelling captures a particular rational point of view, we ask the following question: Given an argumentation framework and a set of agents, each with a legitimate subjective evaluation of the given arguments, how can the agents reach a collective compromise on how to evaluate those arguments?

Thus, the problem we face is that of judgment aggregation [21] in the context of argumentation frameworks. This problem can be formulated as a set of individuals that collectively decide how an argumentation framework must be labelled.

Definition 6 (Labelling aggregation problem).

Let be a finite non-empty set of agents, and be an argumentation framework. A labelling aggregation problem is a pair .

Each individual has a labelling which expresses the evaluation of by this individual. A labelling profile is an -tuple of labellings.

Definition 7 (Labelling profile).

Let be a labelling aggregation problem. We use to denote a labelling profile, where is the class of labellings of . Additionally, we use to denote the labelling profile (i.e. an -tuple) of an argument i.e. .

The aggregation of individuals’ labellings can be defined as a partial function.333We state that the function is partial to allow for cases in which collective judgment may be undefined (e.g. when there is a tie in voting).

Definition 8 (Aggregation function).

Let be a labelling aggregation problem. An aggregation function for is a function .

For each , denotes the collective label assigned to , if is defined for .

4 Desirable Properties of Aggregation Operators

Aggregation involves comparing and assessing different points of view. There are, of course, many ways of doing this, as extensively discussed in the literature of Social Choice Theory [16]. In this literature, a consensus on some normative ideals has been reached, identifying what a ‘fair’ way of adding up votes should be. So for instance, if everybody agrees, the outcome must reflect that agreement; no single agent can impose her view on the aggregate; the aggregation should be performed in the same way in each possible case, etc. These informal requirements can be formally stated as properties that should satisfy [21, 12]. In all of the following postulates, it is assumed that a fixed labelling aggregation problem is given. The postulates can be grouped as follows:444This style of presentation of postulates was inspired by [18] which is on binary aggregation.

Group 1: Domain and co-domain postulates

In judgment aggregation, two postulates that are commonly assumed are those of Universal Domain and Collective Rationality. The former requires that any profile of labellings chosen from a pre-specified set of feasible labellings can be used as input to and will return an answer. The question is: what do we take to be the set of feasible labellings in our setting? This depends on which semantics we assume is being used. Theoretically we can have a different version of Universal Domain for each semantics. However since complete semantics represent reasonable and self-defending points of views, it represents the best counterpart for the logical consistency in judgment aggregation:

Universal Domain can take as input all profiles such that

However, in Subsection 8.2 we will use other semantics as a domain for .

Similarly we could have a different version of Collective Rationality - one for each semantics - stating that the output of the aggregation should also be feasible. Again, since we focus on complete semantics, we focus on the following version:

Collective Rationality For all profiles such that is defined, .

Later, in Section 7, we will break this postulate down into further constituents.

Group 2: Fundamental postulates

Next we come to the standard property that forms the cornerstone of the usual impossibility results in judgment aggregation. It says the collective label of an argument depends only on the votes on that argument, independent of the other arguments.

Independence For any two profiles , such that and are defined, and for all , if for all , then .

The effect of Independence is that aggregation is done “argument-by-argument”. To be slightly more precise, each argument essentially has its own aggregation operator associated to it, that takes an -tuple of labels as input (representing the “vote” of each agent on the label of ) and returns another label as output (the “collective label”) of . Then . Note that the necessity of Independence is questionable in our settings because of the dependencies between arguments that come already encoded in the form of the attack relation. Nevertheless, it is usually investigated in the judgment aggregation and preference aggregation literature because of its role in analysing strategy-proofness. Though the relation between Independence and strategy-proofness is not established yet in our settings, our task in this paper is to stick close to the methodology in judgment aggregation, and there it is often assumed.

Next, we have Anonymity, which says the identity of which agent submits which labelling is irrelevant.

Anonymity For any profile , if for some permutation on , and and are both defined, then .

If we add Anonymity to Independence, then it means the outputs of the functions described above depend only on the number of votes that each label gets in . Essentially it means outputs a collective label just taking as input the triple of numbers denoting, respectively, the number of votes for , and in .

Proposition 1.

Let be an aggregation operator. Then satisfies both Independence and Anonymity iff for each there exists a function such that, for all we have .

Outline.

The “if” case is straightforward, since permuting the rows does not change the vote distribution and so Anonymity will hold. Independence is also clear.

For the “only if” case, Independence gives us the existence of the function such that and then Anonymity implies that two vectors that have the same vote distribution will give the same results, so we can set where is any vote which has as its distribution. ∎

A weakening of Anonymity is Non-Dictatorship:555Since a violation of the latter would imply a violation of the former.

Non-Dictatorship There is no such that, for every profile for which is defined, we have .

Group 3: Unanimity postulates

Next we move to Unanimity, and some other postulates related to it.

Unanimity If is such that is defined and there exists some s.t.  for all , then .

This postulate is also familiar from judgment aggregation, but the move to 3-valued labellings rather than the 2 usually seen in judgment aggregation opens up the possibility to define other variants of Unanimity, one of which is used by Dokow and Holzman [14], called Supportiveness:

Supportiveness For any profile such that is defined, and for all , there exists such that .

Supportiveness says that, for each argument and label , the collective judgment cannot be set to without at least one agent voting for that . Clearly Supportiveness implies Unanimity.

It might seem natural to have the collective label of an argument as even when nobody votes for it, if we interpret as a halfway label between and . Then if half the agents say and the other half says then might be a reasonable compromise. Given this, a weaker version of Supportiveness that only applies to and can be defined. We call it -Supportiveness.

-Supportiveness For any profile such that is defined, and for all , if then there exists some agent such that .

Group 4: Systematicity postulates

Now we come to the Systematicity postulates which deal with neutrality issues across arguments and labels. We can list two variants, both of which imply Independence. We start with the stronger version:

Strong Systematicity For any two profiles and such that and are defined, and for all , and for every permutation on the set of labels , if , then .

To illustrate Strong Systematicity, consider the example in Figure 5. We have the following three labellings: , , .

Figure 5: An example illustrating Strong Systematicity.

Consider the profiles and . Then, and . Let be the permutation on labels such that , , and . Then, we can see that in this example . Strong Systematicity requires that .

The postulate forces us to give an even-handed treatment to the labels , and (in addition to treating each argument independently and similarly). This makes sense if we consider , and as three independent labels. However, one might be tempted to consider as a middle label between and . Hence, the equal treatment might not be desirable in this case. One might suggest a version of Systematicity that treats and equally. Following, we define this version (which we call -Systematicity).

-Systematicity For any two profiles and such that and are defined, and for all , and for every -preserving permutation on the set of labels (), if , then .

-Systematicity lies in the middle between Strong Systematicity and the following version of Systematicity which can be obtained by restricting the class of permutations, until we only consider the identity.

Weak Systematicity For any two profiles and such that and are defined, and for all , if , then .

Clearly Independence follows from Weak Systematicity by just setting . If we strengthen Independence to Weak Systematicity then the functions , mentioned earlier, are identical for all arguments.

Group 5: Monotonicity postulates

Our final group relates to Monotonicity.

Monotonicity Let be such that given two profiles and (differing only in the labellings of agents ) such that and are defined, where and , if while for all , then implies that .

Monotonicity states that if a set of agents switch their label of argument to the collective label of then the collective label of remains the same. Similar to Supportiveness and Systematicity, a weaker version of Monotonicity that only apply to and can be defined. We call it -Monotonicity.

-Monotonicity Let be such that given two profiles and (differing only in the labellings of agents ) such that and are defined, where and , if while for all , then implies that .

5 The Argument-Wise Plurality Rule

An obvious candidate aggregation operator to check out is the plurality voting operator . In this section, we analyse a number of key properties of this operator. Intuitively, for each argument, it selects the label that appears most frequently in the individual labellings.

Definition 9 (Argument-Wise Plurality Rule (AWPR)).

Let be an argumentation framework. Given any argument and any profile , it holds that iff

Note that is defined for all profiles that cause no ties, i.e. is defined iff there does not exist any argument for which we have at least two labels and with and

One can directly notice that AWPR violates Universal Domain, because it is not defined for all profiles in .

Example 5 (Three Detectives (cont.)).

Continuing on Example 2, applying the argument-wise plurality rule, we have , , and .

5.1 Properties of Argument-Wise Plurality Rule

We now analyse whether AWPR satisfies the properties listed above.

Proposition 2.

The argument-wise plurality rule operator satisfies Supportiveness, Anonymity, Strong Systematicity, and Monotonicity.

Proof.

In this proof, the considered profiles are restricted to those for which is defined.

  • Supportiveness: consider any profile . Suppose, towards a contradiction, that for some argument , there exists no agent such that where . Then . But, (the last inequality holds since is non-empty). Contradiction.

  • Anonymity: consider any profile . if and only if if and only if , which is equivalent to
    .

  • Strong Systematicity: consider, for any two profiles and , and for any , the permutation . Suppose, towards a contradiction, that for any , , and but . But then, while for any , . So, if then, we have as well. Contradiction.

  • Monotonicity: Consider the following two profiles and (differing only in the labellings of agents ) where and . Suppose, towards a contradiction, that for and a label we have that while for all , and we have that while . But then, in the profile while in the profile , we have and for every other labelling . Then . Contradiction.

Corollary 1.

The argument-wise plurality rule operator satisfies Unanimity, Weak Systematicity, Independence, and Non-Dictatorship.

Proof.

Weak Systematicity and Independence follow from Strong Systematicity, Unanimity follows from Supportiveness, and Non-Dictatorship follows from Anonymity. ∎

Despite all these promising results, it turns out that plurality operator violates Universal Domain and Collective Rationality postulates. The violation of Universal Domain is because AWPR is not defined for profiles that cause ties, which means that it cannot take as input every possible profile . However, a weaker version of Universal Domain can be defined.

No-Tie Universal Domain An aggregation operator can take as input all profiles such that does not cause a tie and .

Since there are no restrictions (other than having no ties) on how labellings are defined, AWPR satisfies No-Tie Universal Domain. Note that one might be tempted to make AWPR satisfy Universal Domain by adding a deterministic666The use of a non-deterministic tie-breaking rule has its own issues too, such as producing different outcomes given the same profile. tie-breaking rule to deal with ties. However, as we show in the next section, the use of any tie-breaking rule would result in violating Anonymity, and/or Strong Systematicity. While the violation of Universal Domain represents a minor inconvenience that can be justified, the violation of Collective Rationality poses a serious issue as the collective decision is usually expected to be reasonable. The following example shows how AWPR violates Collective Rationality.

Example 6.

Suppose argument has two defeaters, and , and argument (resp. ) defeats and is defeated by argument (resp. ). Suppose we have agents, with votes as shown in Figure 6. We have , but it is not the case that or .

Figure 6: Counterexample to Collective Rationality

Interestingly, the above counterexample demonstrates a variant of the discursive dilemma [21] in the context of argument evaluation, which itself is a variant of the well-known Condorcet paradox.

6 The Impossibility of Good Aggregation Operators

In the previous section, we analysed a particular judgment aggregation operator (namely, argument-wise plurality rule). We showed that while it satisfies most key properties, it fails to satisfy Universal Domain and Collective Rationality. In this section, we show a couple of impossibility results that involve these two postulates. The following result shows that introducing a tie-breaking rule to satisfy Universal Domain would result in violating Anonymity and/or Strong Systematicity.

The previous result can be read in two ways: First, the AWPR cannot be made to satisfy Universal Domain without violating Strong Systematicity or Anonymity. Second, there exists no aggregation operator at all that satisfies Universal Domain, Strong Systematicity and Anonymity.

Note that the previous theorem was stated for a set of agents divisible by three. Essentially, three-way ties would only happen if the cardinality of the agents is divisible by three (since there are only three possible labels for each argument, and each individual has to submit one label for each argument). Hence, one might wonder whether we could rule out the possibility of three-way ties, by assuming cannot be a multiple of three.777It was shown in [24] that Anonymity, Neutrality (a weaker version of Strong Systematicity) and Resolution can be satisfied together if and only if the number of alternatives cannot be written as the sum of non-trivial dividers of the number of voters. Resolute rules always produce a single outcome, so it resembles No-Tie Universal Domain. Also, in our settings, the number of candidates is three. So this result says that we can have these postulates together if the number of voters is not a multiple of three. However, with even number of agents, we can show that there is still a large class of s which do not have an operator satisfying those three postulates without violating Collective Rationality.

Theorem 2.

There exists an argumentation framework AF such that, for any set of agents of even cardinality, there exists no labelling aggregation operator satisfying Universal Domain, Anonymity, Strong Systematicity and Collective Rationality.

Proof.

It is enough to assume an AF that contains at least one argument that can feasibly take on just two out of the three possible labels. For concreteness suppose can only take on labels and (An example of such a framework and an argument can be seen in the proof of Theorem 3 below, in which can only be either or ). Let and be two complete labellings such that and . Divide the agents into two groups , of equal size. By Universal Domain, all profiles consisting of legal labellings are valid input, so assume a profile in which everyone in provides labelling and everyone in provides . Denote the resulting profile by and assume for contradiction that is an aggregation operator for this AF that satisfies Universal Domain, Anonymity, Strong Systematicity and Collective Rationality. Let be the permutation that swaps and , i.e., and , and let .888Note here that all labellings in the profile are still complete labellings. This is because does not uniformly exchange all labels in a given labelling, it is just a permutation on the set of labels. By Anonymity we know . Then it cannot be that , for if so then Strong Systematicity would imply , and similarly it cannot be that . Thus we must have . But by Collective Rationality . Contradiction. ∎

The careful reader can realise that Collective Rationality can be substituted with Supportiveness in the previous theorem. As for the proof, the last sentence becomes: “Thus we must have . But by Supportiveness . Contradiction”.

However, one might argue that Strong Systematicity is quite a strong condition. Treating , , and differently can be tolerated. Then, it is interesting to ask: “Does there exist an operator that satisfies Universal Domain, Weak Systematicity, and Anonymity?”. The answer for this question is positive. Consider a modified version of the AWPR that deals with ties by labelling every argument that has a tie with . One can show that this operator satisfies these three properties together. However, this operator still violates Collective Rationality (Example 6 holds as a counterexample). In fact, we show that any operator that satisfies Universal Domain, Weak Systematicity, and Anonymity, would violate either Collective Rationality or Unanimity.

One might note that all of the above theorems exploit the use of profiles that include ties. Then, one would ask: What if we relax Universal Domain to No-Tie Universal Domain? Do we still have impossibility results then? Following, we show that an aggregation operator which satisfies No-Tie Universal Domain (but not necessarily Universal Domain) cannot also satisfy Weak Systematicity, Anonymity, Collective Rationality, and Supportiveness together.

One can draw a connection between this result and the previous one. Relaxing Universal Domain to No-Tie Universal Domain, introduces another impossibility result, in which Unanimity is replaced with the stronger postulate Supportiveness. Additionally, one can compare this result to the analogue of Arrow’s theorem in judgment aggregation [22], which involves Unanimity, Independence, and Non-dictatorship, the weaker versions of Supportiveness, Weak Systematicity, and Anonymity respectively in our theorem. However, their result also involves completeness, i.e. no proposition can be collectively undecided, which we do not have as a condition in our result.

The above impossibility results highlight a major barrier to reaching good collective judgment about argument evaluation in general. These establish the limits of aggregation in the context of argumentation, and come in accordance with the similar topics of aggregation such as preference aggregation [1] and judgment aggregation [21]. Unfortunately, there is no escape from violating the involved conditions or accepting irrational aggregate argument labellings without somewhat lowering our standards in terms of desirable criteria.

7 Collective Rationality Postulates

In this section, we characterise Collective Rationality in terms of conditions that need to be satisfied by profiles. To do this, we need to go back to the definition of legal (i.e. complete) labelling (Definition 4), and break it down into further constituents defined over the outcome of an aggregation operator.

The following condition, which we call IN-Collective Rationality (IN-CR), requires that if an argument is collectively accepted by the agents, then the agents must collectively reject all counter-arguments against .

IN-Collective Rationality (IN-CR) For any profile and , if then:

and

Note that IN-CR1, the first part of IN-CR, represents the the condition of conflict-freeness applied on the output. The condition of conflict-freeness is usually agreed on as a minimal reasonable condition in argument evaluation.

We present now the OUT-Collective Rationality (OUT-CR) condition. Intuitively, this condition means that if an argument is collectively rejected by the agents, then the agents must also collectively agree on accepting at least one of the counter-arguments against .

OUT-Collective Rationality (OUT-CR) For any profile and , if then , such that and .

We present now the UNDEC-Collective Rationality (UNDEC-CR) condition. An argument must be labelled if and only if: (i) it is not the case that all of its defeaters are , that is, at least one of its defeaters is ; and (ii) none of its defeaters is .

UNDEC-Collective Rationality (UNDEC-CR) For any profile and , if then:

and

The following result follows immediately from the definitions.

Proposition 3.

An argument aggregation operator satisfies Collective Rationality if and only if for each profile in its domain, it satisfies the IN-CR, OUT-CR, and UNDEC-CR conditions.

8 Plurality Rule with Classical Semantics

In this section, we analyse the performance of AWPR with respect to Collective Rationality when agents labellings are restricted to some classical semantics (i.e. complete, grounded, stable, semi-stable, and preferred). This investigation gives a novel meaning to classical semantics in social choice settings. Rather than simply being compared by their logical rigour from the perspective of a single agent, semantics are compared based on the extent to which they facilitate collectively rational agreement among agents.

Our strategy will be based on the following approach. Since, by Proposition 3, Collective Rationality arises iff IN-CR, OUT-CR, and UNDEC-CR are satisfied, it is enough to check whether AWPR satisfies those properties.

8.1 Complete Semantics

Since the complete semantics generalises other classical semantics, we provide analysis for it first. Every property that is satisfied by AWPR when individuals’ labellings are complete labellings would be also satisfied by AWPR when individuals’ labellings are restricted to the other classical semantics that we consider.

It is very interesting to see that, as the proposition below shows, when agents collectively accept an argument, the structure of the AWPR will ensure that they will not collectively accept any of its defeaters:

It is important to recognise that Proposition 4 is a non-trivial result. It shows that, with AWPR, the postulate IN-CR1 is satisfied. This means, as we mentioned earlier, that AWPR satisfies the “collective” version of conflict-freeness, a condition that is usually agreed on as a minimal reasonable condition in argument evaluation. This comes “for free” as a result of the intrinsic structure of the individual labellings, leading to coordinated votes. Note, however, that the IN-CR postulate is not fully satisfied. Although Proposition 4 guarantees that a collectively accepted argument will never have a collectively accepted defeater, it does not guarantee IN-CR2, that none of its defeaters will be collectively undecided. This is demonstrated in the following remark.

Remark 1.

AWPR violates IN-CR2. If an argument is collectively accepted, some of its defeaters might be collectively undecided.

Proof.

Suppose argument has two defeaters, and . Suppose we have agents, with votes as shown in Figure 7. Clearly, while is collectively accepted because , one of its defeaters is not collectively rejected because .

Figure 7: Seven votes collectively accepting , without collectively rejecting

As we saw earlier in Example 6, OUT-CR is violated by AWPR.

Remark 2.

AWPR violates OUT-CR. If an argument is collectively rejected, it is not guaranteed that one of its defeaters will be collectively accepted.

Proof.

See Example 6 for a counterexample. ∎

The following remark shows that there are no intrinsic guarantees for satisfying UNDEC-CR1.

Remark 3.

AWPR violates UNDEC-CR1. If an argument is collectively undecided, it is possible that one of its defeaters will be collectively accepted.

Proof.

Suppose argument has two defeaters, and . Suppose we have agents. Suppose the votes are as shown in Figure 8. We have with votes, but we have with votes, thus violating the postulate.

Figure 8: Seven agents collectively undecided on , but collective accepting

Similarly, the remark below shows that UNDEC-CR2 is not intrinsically guaranteed.

Remark 4.

AWPR violates UNDEC-CR2. If an argument is collectively undecided, it is possible that none of its defeaters will be collectively undecided.

Proof.

Suppose argument has two defeaters, and . Suppose we have agents, with votes as shown in Figure 9. Clearly, we have , but we have and , which would have required to be .

Figure 9: Three votes collectively undecided about , but not collectively undecided about any of its defeaters or .

8.2 Other Classical Semantics

As we noted before, each possible complete labelling represents a valid self-defending viewpoint, therefore restricting votes to complete labellings is akin to requiring that each vote in judgment aggregation is consistent, or that each preference in preference aggregation is transitive and complete. Other classical semantics are essentially restrictions (i.e. sub-cases) of complete semantics. For example, restricting votes to preferred semantics requires each individual to be more committed, maximizing (w.r.t. set-inclusion) the set of accepted (or the set of rejected) arguments, while restricting votes to semi-stable semantics requires each individual to be less conservative, minimizing (w.r.t. set-inclusion) the set of arguments about which they are undecided. It is not clear, a priori, what such requirements, applied on the individual, would have on the collective rationality of the outcome of voting.

In this subsection, we provide an analysis for the grounded, stable, semi-stable, and preferred semantics as more restricted forms of labellings to choose from. Note that the definition of Universal Domain, introduced earlier using complete semantics, is now redefined with respect to these semantics, while the definition of Collective Rationality is unchanged.

The following proposition looks trivial but, as we will see, it is the most positive result in this subsection.

Proposition 5.

If for every argument, agents can only vote for the grounded labelling, then satisfies IN-CR1, IN-CR2, OUT-CR, UNDEC-CR1 and UNDEC-CR2. Equivalently, satisfies Collective Rationality.

Proof.

Trivial since there always exists one grounded labeling [15, 8], and satisfies Unanimity.

As a corollary of Proposition 4, when agents votes are restricted to stable (respectively semi-stable or preferred) labellings, AWPR satisfies IN-CR1.

Corollary 2.

When agents can only vote for stable (respectively semi-stable or preferred) labellings, AWPR satisfies IN-CR1

Proof.

From Proposition 4, if agents can only vote for complete labellings, then AWPR satisfies IN-CR1. Since every stable (respectively semi-stable or preferred) labelling is a complete labelling, then when agents votes are restricted to these semantics, AWPR satisfies IN-CR1. ∎

Lemma 1.

When agents can only vote for a stable labelling, AWPR satisfies IN-CR2. If an argument is collectively accepted, none of its defeaters is collectively undecided.

Proof.

Suppose, towards a contradiction, that there exists an argument that is collectively accepted and one of its defeaters is collectively undecided. Then, by Supportiveness, there exists one submitted labelling (by some agent) in which this argument is undecided. However, agents are only allowed to submit a stable labelling, and stable labellings have no argument labelled undecided. Contradiction. ∎

Remark 5.

When agents can only vote for stable (respectively semi-stable or preferred) labellings, AWPR violates OUT-CR. If an argument is collectively rejected, it is possible that none of its defeaters is collectively accepted.

Proof.

See Example 6 for a counterexample. ∎

Lemma 2.

When agents can only vote for a stable labelling, AWPR satisfies UNDEC-CR (i.e. it satisfies both UNDEC-CR1 and UNDEC-CR2). If an argument is collectively undecided, none of its defeaters is collectively accepted, and at least one of its defeaters is collectively undecided.

Proof.

Since in stable labelling no argument is labelled undecided, by Supportiveness, there is no argument that is collectively undecided. Then, this lemma holds. ∎

We continue with the semi-stable and preferred semantics.

Remark 6.

When agents can only vote for a semi-stable (respectively preferred) labelling, AWPR violates IN-CR2. If an argument is collectively accepted, it is possible that one of its defeaters is collectively undecided.

Proof.

Suppose argument has two defeaters, and . Suppose we have agents, with votes as shown in Figure 10. Clearly, while is collectively accepted because , one of its defeaters, namely , is collectively undecided because . ∎

Figure 10: A counterexample shows how, given semi-stable (respectively preferred) semantics, AWPR violates IN-CR2 and UNDEC-CR1.
Remark 7.

When agents can only vote for a semi-stable (respectively preferred) labelling, AWPR violates UNDEC-CR1. If an argument is collectively undecided, it is possible that one of its defeaters is collectively accepted.

Proof.

Suppose argument has two defeaters, and . Suppose we have agents, with votes as shown in Figure 10. Clearly, while is collectively undecided because , one of its defeaters, namely , is collectively accepted because . ∎

Remark 8.

When agents can only vote for a semi-stable (respectively preferred) labelling, AWPR violates UNDEC-CR2. If an argument is collectively undecided, it is possible that none of its defeaters is collectively undecided.

Proof.

Suppose argument has two defeaters, and . Suppose we have agents, with votes as shown in Figure 11. Clearly, while is collectively undecided because , none of its defeaters is collectively undecided. ∎

Figure 11: A counterexample shows how, given semi-stable (respectively preferred) semantics, AWPR violates UNDEC-CR2.

To sum up, the only restriction that would satisfy the Collective Rationality is the grounded semantics (Proposition 5). This is trivially true because only one grounded labelling exists. However, stable semantics violates Collective Rationality only because it violates OUT-CR. As for the semi-stable and preferred semantics, they only satisfy IN-CR1, a property they inherit from the complete semantics. Refer to Table 1 for a summary of the results we have found.

Semantics IN-CR OUT-CR UNDEC-CR
IN-CR1 IN-CR2 UND-CR1 UND-CR2
Grounded Yes Yes Yes Yes Yes
(Prop. 5) (Prop. 5) (Prop. 5) (Prop. 5) (Prop. 5)
Stable Yes Yes No Yes Yes
(Cor. 2) (Lem. 1) (Rem. 5) (Lem. 2) (Lem. 2)
Semi-stable Yes No No No No
(Cor. 2) (Rem. 6) (Rem. 5) (Rem. 7) (Rem. 8)
Preferred Yes No No No No
(Cor. 2) (Rem. 6) (Rem. 5) (Rem. 7) (Rem. 8)
Complete Yes No No No No
(Prop. 4) (Rem. 1) (Rem. 2) (Rem. 3) (Rem. 4)
Table 1: The Collective Rationality properties that are satisfied/violated by AWPR given different semantics.

9 Restricting the Domain of Argumentation Graphs to Satisfy Collective Rationality

In an earlier section, we showed that, AWPR violates Universal Domain and Collective Rationality. In this section, we investigate whether AWPR can satisfy Collective Rationality by restricting the argumentation framework to graphs with certain graph-theoretical properties. We show that graphs consisting of disconnected issues (a notion we define below) and graphs in which arguments have limited defeaters (in some sense) guarantee collectively rational outcomes when the AWPR is used.

9.1 Disconnected Issues

The notion of “issue” was defined in [7] in order to quantify disagreement between graph labellings. In this section, we use this notion to provide a possibility result.

Crucial to the definition of the “issue” is the concept of “in-sync”. Two arguments and are said to be in-sync if the (complete) label of one cannot be changed without causing a change of equal magnitude to the label of the other.

Definition 10 (in-Sync [7]).

Let be the set of all complete labellings for argumentation framework . We say that two arguments are in-sync ():

(4)

where:

  • iff .

  • iff

This relation forms an equivalence relation over the arguments, and the equivalence classes are called “issues”.

Definition 11 (Issue [7]).

Given the argumentation framework , a set of arguments