Judgment Aggregation in MultiAgent Argumentation
Abstract
Given a set of conflicting arguments, there can exist multiple plausible opinions about which arguments should be accepted, rejected, or deemed undecided. We study the problem of how multiple such judgments can be aggregated. We define the problem by adapting various classical socialchoicetheoretic properties for the argumentation domain. We show that while argumentwise plurality voting satisfies many properties, it fails to guarantee the collective rationality of the outcome. We then present more general results, proving multiple impossibility results on the existence of any good aggregation operator. After characterising the sufficient and necessary conditions for satisfying collective rationality, we study whether restricting the domain of argumentwise plurality voting to classical semantics allows us to escape the impossibility result. We close by mentioning a couple of graphtheoretical restrictions under which the argumentwise plurality rule does produce collectively rational outcomes. In addition to identifying fundamental barriers to collective argument evaluation, our results contribute to research at the intersection of the argumentation and computational social choice fields.
1 Introduction
Argumentation has recently become one of the key approaches to automated reasoning and rational interaction in Artificial Intelligence [5, 28]. A key milestone in the development of argumentation in AI has been Dung’s landmark framework [15], known as abstract argumentation framework (AAF). Arguments are viewed as abstract entities (a set ), with a binary defeat relation (denoted ) over them. The defeat relation captures the fact that one argument somehow attacks or undermines another. This view of argumentation enables highlevel analysis while abstracting away from the internal structure of individual arguments. In Dung’s approach, given a set of arguments and a defeat relation, a rule specifies which arguments should be accepted.
Often, there are multiple reasonable ways in which an agent may evaluate a given argument structure (e.g. accepting only conflictfree, selfdefending sets of arguments). Each possible evaluation corresponds to a socalled extension [15] or labelling [8, 9]. Different argumentation semantics yield different restrictions on the possible extensions. Most previous research has focused on evaluating and comparing different semantics based on the (objective) logical properties of their extensions [3].
One of the essential properties, which is common, is the condition of admissibility: that accepted arguments must not attack one another, and must defend themselves against counterarguments, by attacking them back. A stronger notion is called completeness, and is captured, in terms of labelling, in the following two conditions:

An argument is labelled accepted (or ) if and only if all its defeaters are rejected (or ).

An argument is labelled rejected (or ) if and only if at least one of its defeaters is accepted (or ).
Otherwise, an argument may be labelled . Thus, evaluating a set of arguments amounts to labelling each argument using a labelling function to capture these three possible labels. Any labelling that satisfies the above conditions is also called a legal labelling. We will often use legal labelling and complete labelling interchangeably.
The above conditions attempt to evaluate arguments from a single point of view. Indeed, most research on formal models of argumentation discounts the fact that argumentation takes place among selfinterested agents, who may have conflicting opinions and preferences over which arguments end up being accepted, rejected, or undecided. Consider the following simple example.
Example 1 (A Murder Case).
A murder case is under investigation. To start with, there is an argument that the suspect should be presumed innocent (). However, there is evidence that he may have been at the crime scene at the time (), which would counter the initial presumption of innocence. There is also, however, evidence that the suspect was attending a party that day (). Clearly, and are mutually defeating arguments since the suspect can only be in one place at any given time. Hence, we have a set of arguments and a defeat relation . There are three possible labellings that satisfy the above conditions:

, , .

, , .

, , .
The graph and possible labellings are depicted in Figure 1.
Example 1 highlights a situation in which multiple points of view can be taken, depending on whether one decides to accept the argument that the suspect was at the party or the crime scene. The question we explore in this paper can be highlighted through the following example, extending Example 1.
Example 2 (Three Detectives).
A team of three detectives, named , , and , have been assigned to the murder case described in Example 1. Each detective’s judgment can only correspond to a legal labelling (otherwise, her judgment can be discarded). Suppose that each detective’s judgment is such that , and . That is, detectives and agree but differ with detective . These labellings are depicted in the labelled graph of Figure 2. The detectives must decide which (aggregated) argument labelling best reflects their collective judgment.
Example 2 highlights an aggregation problem, similar to the problem of preference aggregation [2, 16, 33] and the problem of judgment aggregation on propositional formulae [23, 20, 22, 19]. It is perhaps obvious in this particular example that must be rejected (and thus the defendant be considered guilty), since most detectives seem to think so. For the same reason, must be rejected and must be accepted. Thus, labelling (see Example 1) wins by majority. As we shall see in our analysis below, things are not that simple, and counterintuitive situations may arise. We summarise the main question asked in the paper as follows.
Given a set of agents, each with a specific subjective evaluation (i.e. labelling) of a given set of conflicting arguments, how can agents reach a collective decision on how to evaluate those arguments?
While Arrow’s Impossibility Theorem can be expected to ensue for this problem [1],^{1}^{1}1Arrow’s Theorem claims that four quite natural constraints, that capture abstractly the properties of a democratic aggregation process, cannot be simultaneously satisfied. there exist many differences between labellings and preference relations (for which Arrow’s result apply), stemming from their corresponding ordertheoretic characterisations. In other words, aggregating preferences assumes that agents submit a full order of preferences over candidates, while in labelling aggregation, agents submit their top labelling for a set of logically connected arguments.
The problem of labelling aggregation is more comparable to the judgment aggregation problem [23, 20, 22, 19], by considering arguments as propositions which are logically connected by the conditions of legal labelling. However, one important difference is that in judgment aggregation, each proposition can have two values: True or False. In labelling aggregation, on the other hand, each argument can have three values: , , or . This makes labelling aggregation be more comparable to nonbinary evaluations [13, 14]. Considering the general framework in [14], our settings can be considered as focusing on special classes of feasible evaluations, which are the conditions imposed by the legal labelling (or other semantics). Additionally, the possible evaluations of each issue (argument, in our case) are to accept (labels as ), reject (labels as ), or be undecided (labels as ).
In this paper, we conduct an extensive socialchoicetheoretic analysis of argument evaluation semantics by means of labellings. We assume that individuals are presented with a shared argumentation framework (AF) and need to make a decision about how to evaluate this AF. Individuals are assumed to have different, but reasonable, evaluations. There can be many scenarios in which such settings are present. For example, consider a jury members that are all provided with the same information, each of them has a different opinion about these information and yet they all need to come up with a collective decision. Another example is a company board committee who need to make an informed decision. They can be all presented with the same information about the current economic status and the possible strategies, each one of them has his/her own opinion about what should be done, yet they all need to reach a collective decision.
The paper makes three distinct contributions to the stateoftheart in the computational modelling of argumentation. Firstly, the paper introduces the study of aggregating different individual judgments on how a given set of arguments is to be evaluated.^{2}^{2}2In fact, this idea was first introduced in [29] for which this paper is a substantially extended and revised version. Section 6 which introduces the impossibility of good aggregation operator is significantly enhanced by adding three impossibility results. Sections 8 and 9 are completely new. Section 10 contains more elaborate discussion of related and future work. Finally, further explanation, motivation, discussion and background is added to the other sections to improve clarity and presentation of the paper. This requires adapting classical socialchoice properties to the argumentation domain, and sometimes demands special treatment (e.g. different versions of some properties).
The second contribution of this paper is proving the impossibility of the existence of any aggregation operator that satisfies some minimal properties. In doing so, we show impossibility results that concern dealing with ties and producing a collectively rational evaluation of arguments. These results establish the limits of aggregation in the context of argumentation, and come in accordance with the impossibility results in the topics of aggregation such as preference aggregation [1, 31, 25, 17, 30] and judgment aggregation [21]. Hence, as is the case with other aggregation domains, the aggregation paradox in argument evaluation is an example of a more fundamental barrier. These results are important because they give conclusive answers and focus research in more constructive directions (e.g. weakening the desired properties in order to avoid the paradox). Aiming to investigate possible relaxations in order to circumvent the impossibility in the context of argumentation, we broke down the Collective Rationality postulate into subpostulates. This helps in taking a deeper look at the distinct parts of the postulate. As a consequence, satisfying any of these parts can be used to weaken the collective rationality.
The third contribution of this paper is an extensive analysis of an aggregation rule, namely argumentwise plurality rule. We analyse the properties of the argumentwise plurality rule in general, and investigate whether the restriction of the domain of votes to a particular classical semantics would ensure the fulfillment of these conditions. This highlights a novel use of classical semantics, which are originally used to resolve issues in singleagent nonmonotonic reasoning. Finally, we provide graphtheoretical restrictions on argumentation frameworks under which the argumentwise plurality rule would be guaranteed to produce collectively rational outcomes.
The paper is organised as follows. In section 2, we start by giving a brief background on abstract argumentation systems. Sections 3, 4, 6 and 7 focus on the problem of aggregating sets of judgments over argument evaluation. Sections 5, 8, and 9 focus on introducing and analysing the argumentwise plurality rule. We conclude the paper and discuss some related work in Section 10.
2 Background
In this section, we briefly outline key elements of abstract argumentation frameworks. We begin with Dung’s abstract characterisation of an argumentation system [15]. We restrict ourselves to finite sets of arguments.
Definition 1 (Argumentation framework).
An argumentation framework is a pair where is a finite set of arguments and is a defeat relation. We say that an argument defeats an argument if (sometimes written ).
For an argument , we use to denote the set of arguments that defeat i.e. .
An argumentation framework can be represented as a directed graph in which vertices are arguments and directed arcs characterise defeat among arguments. An example argument graph is shown in Figure 3. Argument has two defeaters (i.e. counterarguments) and , which are themselves defeated by arguments and respectively.
There are two approaches to define semantics that assess the acceptability of arguments. One of them is extensionbased semantics by Dung [15], which produces a set of arguments that are accepted together. Another equivalent labellingbased semantics is proposed by Caminada [8, 9], which gives a labelling for each argument. With argument labellings, we can accept arguments (by labelling them as ), reject arguments (by labelling them as ), and abstain from deciding whether to accept or reject (by labelling them as ). Caminada [8, 9] established a correspondence between properties of labellings and the different extensions. In this paper, we employ the labelling approach.
Definition 2 (Argument Labelling).
Let be an argumentation framework. An argument labelling is a total function .
We write (resp. , ) for the set of arguments that are labelled (resp. , ) by . A labelling can be represented as ,,.
However, labellings should follow some given conditions. A minimal reasonable condition is the conflictfreeness.
Definition 3 (Conflictfreeness).
A labelling satisfies conflictfreeness iff , .
One of the essential semantics, which satisfies conflictfreeness is the complete semantics. We already informally defined complete labellings via two conditions in the introduction. We find it convenient to equivalently formulate it as three conditions as follows.
Definition 4 (Complete labelling).
Let be an argumentation framework. A complete labelling is a total function such that:

;

; and


We will use to denote the set of all complete labellings for .
As an example, consider the following.
Example 3.
Consider the graph in Figure 4. Here, we have three complete labellings: , , and .
In addition to the complete labelling, there are other semantics which assume further conditions.
Definition 5 (Other Labelling Semantics).
Let be an argumentation framework. Let be a complete labelling.

is a grounded labelling if and only if is minimal, or equivalently is minimal, or equivalently is maximal (w.r.t set inclusion) among all complete labellings.

is a preferred labelling if and only if is maximal, or equivalently is maximal (w.r.t set inclusion) among all complete labellings.

is a semistable labelling if and only if is minimal (w.r.t set inclusion) among all complete labellings.

is a stable labelling if and only if .
Note that the grounded labelling is always unique, and stable labellings might not exist. Consider the following example.
Example 4.
Consider the graph in Figure 4. Here, we have the grounded labelling is . We have only two preferred labellings: , and . These are also the only stable and semistable labellings for this framework.
Clearly, for any , , and , where , , , and refer to the set of stable, semistable, preferred, and grounded labellings for . We refer to the previous semantics as classical semantics. There exist other semantics which we do not consider in this work.
3 Aggregation of Argument Labellings
To date, most analyses inspired by Dung’s framework have focused on analysing and comparing the properties of various types of extensions/labellings (i.e. semantics) [3]. The question is, therefore, whether a particular type of labelling is appropriate for a particular type of reasoning task in the presence of conflicting arguments.
In contrast with most existing work on Dung frameworks, our concern here is with multiagent systems. Since each labelling captures a particular rational point of view, we ask the following question: Given an argumentation framework and a set of agents, each with a legitimate subjective evaluation of the given arguments, how can the agents reach a collective compromise on how to evaluate those arguments?
Thus, the problem we face is that of judgment aggregation [21] in the context of argumentation frameworks. This problem can be formulated as a set of individuals that collectively decide how an argumentation framework must be labelled.
Definition 6 (Labelling aggregation problem).
Let be a finite nonempty set of agents, and be an argumentation framework. A labelling aggregation problem is a pair .
Each individual has a labelling which expresses the evaluation of by this individual. A labelling profile is an tuple of labellings.
Definition 7 (Labelling profile).
Let be a labelling aggregation problem. We use to denote a labelling profile, where is the class of labellings of . Additionally, we use to denote the labelling profile (i.e. an tuple) of an argument i.e. .
The aggregation of individuals’ labellings can be defined as a partial function.^{3}^{3}3We state that the function is partial to allow for cases in which collective judgment may be undefined (e.g. when there is a tie in voting).
Definition 8 (Aggregation function).
Let be a labelling aggregation problem. An aggregation function for is a function .
For each , denotes the collective label assigned to , if is defined for .
4 Desirable Properties of Aggregation Operators
Aggregation involves comparing and assessing different points of view. There are, of course, many ways of doing this, as extensively
discussed in the literature of Social Choice Theory [16]. In this literature, a consensus on some normative ideals has been reached, identifying what a ‘fair’ way of adding up votes should be. So for instance, if everybody agrees, the outcome must reflect that
agreement; no single agent can impose her view on the aggregate; the aggregation should be performed in the same way in each possible
case, etc. These informal requirements can be formally stated as properties that should satisfy [21, 12]. In all of the following postulates, it is assumed that a fixed labelling aggregation problem is given. The postulates can be grouped as follows:^{4}^{4}4This style of presentation of postulates was inspired by [18] which is on binary aggregation.
Group 1: Domain and codomain postulates
In judgment aggregation, two postulates that are commonly assumed are those of Universal Domain and Collective Rationality. The former requires that any profile of labellings chosen from a prespecified set of feasible labellings can be used as input to and will return an answer. The question is: what do we take to be the set of feasible labellings in our setting? This depends on which semantics we assume is being used. Theoretically we can have a different version of Universal Domain for each semantics. However since complete semantics represent reasonable and selfdefending points of views, it represents the best counterpart for the logical consistency in judgment aggregation:
Universal Domain can take as input all profiles such that
However, in Subsection 8.2 we will use other semantics as a domain for .
Similarly we could have a different version of Collective Rationality  one for each semantics  stating that the output of the aggregation should also be feasible. Again, since we focus on complete semantics, we focus on the following version:
Collective Rationality For all profiles such that is defined, .
Later, in Section 7, we will break this postulate down into further constituents.
Group 2: Fundamental postulates
Next we come to the standard property that forms the cornerstone of the usual impossibility results in judgment aggregation. It says the collective label of an argument depends only on the votes on that argument, independent of the other arguments.
Independence For any two profiles , such that and are defined, and for all , if for all , then .
The effect of Independence is that aggregation is done “argumentbyargument”. To be slightly more precise, each argument essentially has its own aggregation operator associated to it, that takes an tuple of labels as input (representing the “vote” of each agent on the label of ) and returns another label as output (the “collective label”) of . Then . Note that the necessity of Independence is questionable in our settings because of the dependencies between arguments that come already encoded in the form of the attack relation. Nevertheless, it is usually investigated in the judgment aggregation and preference aggregation literature because of its role in analysing strategyproofness. Though the relation between Independence and strategyproofness is not established yet in our settings, our task in this paper is to stick close to the methodology in judgment aggregation, and there it is often assumed.
Next, we have Anonymity, which says the identity of which agent submits which labelling is irrelevant.
Anonymity For any profile , if for some permutation on , and and are both defined, then .
If we add Anonymity to Independence, then it means the outputs of the functions described above depend only on the number of votes that each label gets in . Essentially it means outputs a collective label just taking as input the triple of numbers denoting, respectively, the number of votes for , and in .
Proposition 1.
Let be an aggregation operator. Then satisfies both Independence and Anonymity iff for each there exists a function such that, for all we have .
Outline.
The “if” case is straightforward, since permuting the rows does not change the vote distribution and so Anonymity will hold. Independence is also clear.
For the “only if” case, Independence gives us the existence of the function such that and then Anonymity implies that two vectors that have the same vote distribution will give the same results, so we can set where is any vote which has as its distribution. ∎
A weakening of Anonymity is NonDictatorship:^{5}^{5}5Since a violation of the latter would imply a violation of the former.
NonDictatorship There is no such that, for every profile for which is defined, we have .
Group 3: Unanimity postulates
Next we move to Unanimity, and some other postulates related to it.
Unanimity If is such that is defined and there exists some s.t. for all , then .
This postulate is also familiar from judgment aggregation, but the move to 3valued labellings rather than the 2 usually seen in judgment aggregation opens up the possibility to define other variants of Unanimity, one of which is used by Dokow and Holzman [14], called Supportiveness:
Supportiveness For any profile such that is defined, and for all , there exists such that .
Supportiveness says that, for each argument and label , the collective judgment cannot be set to without at least one agent voting for that . Clearly Supportiveness implies Unanimity.
It might seem natural to have the collective label of an argument as even when nobody votes for it, if we interpret as a halfway label between and . Then if half the agents say and the other half says then might be a reasonable compromise. Given this, a weaker version of Supportiveness that only applies to and can be defined. We call it Supportiveness.
Supportiveness For any profile such that is defined, and for all , if then there exists some agent such that .
Group 4: Systematicity postulates
Now we come to the Systematicity postulates which deal with neutrality issues across arguments and labels. We can list two variants, both of which imply Independence. We start with the stronger version:
Strong Systematicity For any two profiles and such that and are defined, and for all , and for every permutation on the set of labels , if , then .
To illustrate Strong Systematicity, consider the example in Figure 5. We have the following three labellings: , , .
Consider the profiles and . Then, and . Let be the permutation on labels such that , , and . Then, we can see that in this example . Strong Systematicity requires that .
The postulate forces us to give an evenhanded treatment to the labels , and (in addition to treating each argument independently and similarly). This makes sense if we consider , and as three independent labels. However, one might be tempted to consider as a middle label between and . Hence, the equal treatment might not be desirable in this case. One might suggest a version of Systematicity that treats and equally. Following, we define this version (which we call Systematicity).
Systematicity For any two profiles and such that and are defined, and for all , and for every preserving permutation on the set of labels (), if , then .
Systematicity lies in the middle between Strong Systematicity and the following version of Systematicity which can be obtained by restricting the class of permutations, until we only consider the identity.
Weak Systematicity For any two profiles and such that and are defined, and for all , if , then .
Clearly Independence follows from Weak Systematicity by just setting . If we strengthen Independence to Weak Systematicity then the functions , mentioned earlier, are identical for all arguments.
Group 5: Monotonicity postulates
Our final group relates to Monotonicity.
Monotonicity Let be such that given two profiles and (differing only in the labellings of agents ) such that and are defined, where and , if while for all , then implies that .
Monotonicity states that if a set of agents switch their label of argument to the collective label of then the collective label of remains the same. Similar to Supportiveness and Systematicity, a weaker version of Monotonicity that only apply to and can be defined. We call it Monotonicity.
Monotonicity Let be such that given two profiles and (differing only in the labellings of agents ) such that and are defined, where and , if while for all , then implies that .
5 The ArgumentWise Plurality Rule
An obvious candidate aggregation operator to check out is the plurality voting operator . In this section, we analyse a number of key properties of this operator. Intuitively, for each argument, it selects the label that appears most frequently in the individual labellings.
Definition 9 (ArgumentWise Plurality Rule (AWPR)).
Let be an argumentation framework. Given any argument and any profile , it holds that iff
Note that is defined for all profiles that cause no ties, i.e. is defined iff there does not exist any argument for which we have at least two labels and with and
One can directly notice that AWPR violates Universal Domain, because it is not defined for all profiles in .
Example 5 (Three Detectives (cont.)).
Continuing on Example 2, applying the argumentwise plurality rule, we have , , and .
5.1 Properties of ArgumentWise Plurality Rule
We now analyse whether AWPR satisfies the properties listed above.
Proposition 2.
The argumentwise plurality rule operator satisfies Supportiveness, Anonymity, Strong Systematicity, and Monotonicity.
Proof.
In this proof, the considered profiles are restricted to those for which is defined.

Supportiveness: consider any profile . Suppose, towards a contradiction, that for some argument , there exists no agent such that where . Then . But, (the last inequality holds since is nonempty). Contradiction.

Anonymity: consider any profile . if and only if if and only if , which is equivalent to
. 
Strong Systematicity: consider, for any two profiles and , and for any , the permutation . Suppose, towards a contradiction, that for any , , and but . But then, while for any , . So, if then, we have as well. Contradiction.

Monotonicity: Consider the following two profiles and (differing only in the labellings of agents ) where and . Suppose, towards a contradiction, that for and a label we have that while for all , and we have that while . But then, in the profile while in the profile , we have and for every other labelling . Then . Contradiction.
∎
Corollary 1.
The argumentwise plurality rule operator satisfies Unanimity, Weak Systematicity, Independence, and NonDictatorship.
Proof.
Weak Systematicity and Independence follow from Strong Systematicity, Unanimity follows from Supportiveness, and NonDictatorship follows from Anonymity. ∎
Despite all these promising results, it turns out that plurality operator violates Universal Domain and Collective Rationality postulates. The violation of Universal Domain is because AWPR is not defined for profiles that cause ties, which means that it cannot take as input every possible profile . However, a weaker version of Universal Domain can be defined.
NoTie Universal Domain An aggregation operator can take as input all profiles such that does not cause a tie and .
Since there are no restrictions (other than having no ties) on how labellings are defined, AWPR satisfies NoTie Universal Domain. Note that one might be tempted to make AWPR satisfy Universal Domain by adding a deterministic^{6}^{6}6The use of a nondeterministic tiebreaking rule has its own issues too, such as producing different outcomes given the same profile. tiebreaking rule to deal with ties. However, as we show in the next section, the use of any tiebreaking rule would result in violating Anonymity, and/or Strong Systematicity. While the violation of Universal Domain represents a minor inconvenience that can be justified, the violation of Collective Rationality poses a serious issue as the collective decision is usually expected to be reasonable. The following example shows how AWPR violates Collective Rationality.
Example 6.
Suppose argument has two defeaters, and , and argument (resp. ) defeats and is defeated by argument (resp. ). Suppose we have agents, with votes as shown in Figure 6. We have , but it is not the case that or .
Interestingly, the above counterexample demonstrates a variant of the discursive dilemma [21] in the context of argument evaluation, which itself is a variant of the wellknown Condorcet paradox.
6 The Impossibility of Good Aggregation Operators
In the previous section, we analysed a particular judgment aggregation operator (namely, argumentwise plurality rule). We showed that while it satisfies most key properties, it fails to satisfy Universal Domain and Collective Rationality. In this section, we show a couple of impossibility results that involve these two postulates. The following result shows that introducing a tiebreaking rule to satisfy Universal Domain would result in violating Anonymity and/or Strong Systematicity.
The previous result can be read in two ways: First, the AWPR cannot be made to satisfy Universal Domain without violating Strong Systematicity or Anonymity. Second, there exists no aggregation operator at all that satisfies Universal Domain, Strong Systematicity and Anonymity.
Note that the previous theorem was stated for a set of agents divisible by three. Essentially, threeway ties would only happen if the cardinality of the agents is divisible by three (since there are only three possible labels for each argument, and each individual has to submit one label for each argument). Hence, one might wonder whether we could rule out the possibility of threeway ties, by assuming cannot be a multiple of three.^{7}^{7}7It was shown in [24] that Anonymity, Neutrality (a weaker version of Strong Systematicity) and Resolution can be satisfied together if and only if the number of alternatives cannot be written as the sum of nontrivial dividers of the number of voters. Resolute rules always produce a single outcome, so it resembles NoTie Universal Domain. Also, in our settings, the number of candidates is three. So this result says that we can have these postulates together if the number of voters is not a multiple of three. However, with even number of agents, we can show that there is still a large class of s which do not have an operator satisfying those three postulates without violating Collective Rationality.
Theorem 2.
There exists an argumentation framework AF such that, for any set of agents of even cardinality, there exists no labelling aggregation operator satisfying Universal Domain, Anonymity, Strong Systematicity and Collective Rationality.
Proof.
It is enough to assume an AF that contains at least one argument that can feasibly take on just two out of the three possible labels. For concreteness suppose can only take on labels and (An example of such a framework and an argument can be seen in the proof of Theorem 3 below, in which can only be either or ). Let and be two complete labellings such that and . Divide the agents into two groups , of equal size. By Universal Domain, all profiles consisting of legal labellings are valid input, so assume a profile in which everyone in provides labelling and everyone in provides . Denote the resulting profile by and assume for contradiction that is an aggregation operator for this AF that satisfies Universal Domain, Anonymity, Strong Systematicity and Collective Rationality. Let be the permutation that swaps and , i.e., and , and let .^{8}^{8}8Note here that all labellings in the profile are still complete labellings. This is because does not uniformly exchange all labels in a given labelling, it is just a permutation on the set of labels. By Anonymity we know . Then it cannot be that , for if so then Strong Systematicity would imply , and similarly it cannot be that . Thus we must have . But by Collective Rationality . Contradiction. ∎
The careful reader can realise that Collective Rationality can be substituted with Supportiveness in the previous theorem. As for the proof, the last sentence becomes: “Thus we must have . But by Supportiveness . Contradiction”.
However, one might argue that Strong Systematicity is quite a strong condition. Treating , , and differently can be tolerated. Then, it is interesting to ask: “Does there exist an operator that satisfies Universal Domain, Weak Systematicity, and Anonymity?”. The answer for this question is positive. Consider a modified version of the AWPR that deals with ties by labelling every argument that has a tie with . One can show that this operator satisfies these three properties together. However, this operator still violates Collective Rationality (Example 6 holds as a counterexample). In fact, we show that any operator that satisfies Universal Domain, Weak Systematicity, and Anonymity, would violate either Collective Rationality or Unanimity.
One might note that all of the above theorems exploit the use of profiles that include ties. Then, one would ask: What if we relax Universal Domain to NoTie Universal Domain? Do we still have impossibility results then? Following, we show that an aggregation operator which satisfies NoTie Universal Domain (but not necessarily Universal Domain) cannot also satisfy Weak Systematicity, Anonymity, Collective Rationality, and Supportiveness together.
One can draw a connection between this result and the previous one. Relaxing Universal Domain to NoTie Universal Domain, introduces another impossibility result, in which Unanimity is replaced with the stronger postulate Supportiveness. Additionally, one can compare this result to the analogue of Arrow’s theorem in judgment aggregation [22], which involves Unanimity, Independence, and Nondictatorship, the weaker versions of Supportiveness, Weak Systematicity, and Anonymity respectively in our theorem. However, their result also involves completeness, i.e. no proposition can be collectively undecided, which we do not have as a condition in our result.
The above impossibility results highlight a major barrier to reaching good collective judgment about argument evaluation in general. These establish the limits of aggregation in the context of argumentation, and come in accordance with the similar topics of aggregation such as preference aggregation [1] and judgment aggregation [21]. Unfortunately, there is no escape from violating the involved conditions or accepting irrational aggregate argument labellings without somewhat lowering our standards in terms of desirable criteria.
7 Collective Rationality Postulates
In this section, we characterise Collective Rationality in terms of conditions that need to be satisfied by profiles. To do this, we need to go back to the definition of legal (i.e. complete) labelling (Definition 4), and break it down into further constituents defined over the outcome of an aggregation operator.
The following condition, which we call INCollective Rationality (INCR), requires that if an argument is collectively accepted by the agents, then the agents must collectively reject all counterarguments against .
INCollective Rationality (INCR) For any profile and , if then:
and
Note that INCR1, the first part of INCR, represents the the condition of conflictfreeness applied on the output. The condition of conflictfreeness is usually agreed on as a minimal reasonable condition in argument evaluation.
We present now the OUTCollective Rationality (OUTCR) condition. Intuitively, this condition means that if an argument is collectively rejected by the agents, then the agents must also collectively agree on accepting at least one of the counterarguments against .
OUTCollective Rationality (OUTCR) For any profile and , if then , such that and .
We present now the UNDECCollective Rationality (UNDECCR) condition. An argument must be labelled if and only if: (i) it is not the case that all of its defeaters are , that is, at least one of its defeaters is ; and (ii) none of its defeaters is .
UNDECCollective Rationality (UNDECCR) For any profile and , if then:
and
The following result follows immediately from the definitions.
Proposition 3.
An argument aggregation operator satisfies Collective Rationality if and only if for each profile in its domain, it satisfies the INCR, OUTCR, and UNDECCR conditions.
8 Plurality Rule with Classical Semantics
In this section, we analyse the performance of AWPR with respect to Collective Rationality when agents labellings are restricted to some classical semantics (i.e. complete, grounded, stable, semistable, and preferred). This investigation gives a novel meaning to classical semantics in social choice settings. Rather than simply being compared by their logical rigour from the perspective of a single agent, semantics are compared based on the extent to which they facilitate collectively rational agreement among agents.
Our strategy will be based on the following approach. Since, by Proposition 3, Collective Rationality arises iff INCR, OUTCR, and UNDECCR are satisfied, it is enough to check whether AWPR satisfies those properties.
8.1 Complete Semantics
Since the complete semantics generalises other classical semantics, we provide analysis for it first. Every property that is satisfied by AWPR when individuals’ labellings are complete labellings would be also satisfied by AWPR when individuals’ labellings are restricted to the other classical semantics that we consider.
It is very interesting to see that, as the proposition below shows, when agents collectively accept an argument, the structure of the AWPR will ensure that they will not collectively accept any of its defeaters:
It is important to recognise that Proposition 4 is a nontrivial result. It shows that, with AWPR, the postulate INCR1 is satisfied. This means, as we mentioned earlier, that AWPR satisfies the “collective” version of conflictfreeness, a condition that is usually agreed on as a minimal reasonable condition in argument evaluation. This comes “for free” as a result of the intrinsic structure of the individual labellings, leading to coordinated votes. Note, however, that the INCR postulate is not fully satisfied. Although Proposition 4 guarantees that a collectively accepted argument will never have a collectively accepted defeater, it does not guarantee INCR2, that none of its defeaters will be collectively undecided. This is demonstrated in the following remark.
Remark 1.
AWPR violates INCR2. If an argument is collectively accepted, some of its defeaters might be collectively undecided.
Proof.
Suppose argument has two defeaters, and . Suppose we have agents, with votes as shown in Figure 7. Clearly, while is collectively accepted because , one of its defeaters is not collectively rejected because .
∎
As we saw earlier in Example 6, OUTCR is violated by AWPR.
Remark 2.
AWPR violates OUTCR. If an argument is collectively rejected, it is not guaranteed that one of its defeaters will be collectively accepted.
Proof.
See Example 6 for a counterexample. ∎
The following remark shows that there are no intrinsic guarantees for satisfying UNDECCR1.
Remark 3.
AWPR violates UNDECCR1. If an argument is collectively undecided, it is possible that one of its defeaters will be collectively accepted.
Proof.
Suppose argument has two defeaters, and . Suppose we have agents. Suppose the votes are as shown in Figure 8. We have with votes, but we have with votes, thus violating the postulate.
∎
Similarly, the remark below shows that UNDECCR2 is not intrinsically guaranteed.
Remark 4.
AWPR violates UNDECCR2. If an argument is collectively undecided, it is possible that none of its defeaters will be collectively undecided.
Proof.
Suppose argument has two defeaters, and . Suppose we have agents, with votes as shown in Figure 9. Clearly, we have , but we have and , which would have required to be .
∎
8.2 Other Classical Semantics
As we noted before, each possible complete labelling represents a valid selfdefending viewpoint, therefore restricting votes to complete labellings is akin to requiring that each vote in judgment aggregation is consistent, or that each preference in preference aggregation is transitive and complete. Other classical semantics are essentially restrictions (i.e. subcases) of complete semantics. For example, restricting votes to preferred semantics requires each individual to be more committed, maximizing (w.r.t. setinclusion) the set of accepted (or the set of rejected) arguments, while restricting votes to semistable semantics requires each individual to be less conservative, minimizing (w.r.t. setinclusion) the set of arguments about which they are undecided. It is not clear, a priori, what such requirements, applied on the individual, would have on the collective rationality of the outcome of voting.
In this subsection, we provide an analysis for the grounded, stable, semistable, and preferred semantics as more restricted forms of labellings to choose from. Note that the definition of Universal Domain, introduced earlier using complete semantics, is now redefined with respect to these semantics, while the definition of Collective Rationality is unchanged.
The following proposition looks trivial but, as we will see, it is the most positive result in this subsection.
Proposition 5.
If for every argument, agents can only vote for the grounded labelling, then satisfies INCR1, INCR2, OUTCR, UNDECCR1 and UNDECCR2. Equivalently, satisfies Collective Rationality.
As a corollary of Proposition 4, when agents votes are restricted to stable (respectively semistable or preferred) labellings, AWPR satisfies INCR1.
Corollary 2.
When agents can only vote for stable (respectively semistable or preferred) labellings, AWPR satisfies INCR1
Proof.
From Proposition 4, if agents can only vote for complete labellings, then AWPR satisfies INCR1. Since every stable (respectively semistable or preferred) labelling is a complete labelling, then when agents votes are restricted to these semantics, AWPR satisfies INCR1. ∎
Lemma 1.
When agents can only vote for a stable labelling, AWPR satisfies INCR2. If an argument is collectively accepted, none of its defeaters is collectively undecided.
Proof.
Suppose, towards a contradiction, that there exists an argument that is collectively accepted and one of its defeaters is collectively undecided. Then, by Supportiveness, there exists one submitted labelling (by some agent) in which this argument is undecided. However, agents are only allowed to submit a stable labelling, and stable labellings have no argument labelled undecided. Contradiction. ∎
Remark 5.
When agents can only vote for stable (respectively semistable or preferred) labellings, AWPR violates OUTCR. If an argument is collectively rejected, it is possible that none of its defeaters is collectively accepted.
Proof.
See Example 6 for a counterexample. ∎
Lemma 2.
When agents can only vote for a stable labelling, AWPR satisfies UNDECCR (i.e. it satisfies both UNDECCR1 and UNDECCR2). If an argument is collectively undecided, none of its defeaters is collectively accepted, and at least one of its defeaters is collectively undecided.
Proof.
Since in stable labelling no argument is labelled undecided, by Supportiveness, there is no argument that is collectively undecided. Then, this lemma holds. ∎
We continue with the semistable and preferred semantics.
Remark 6.
When agents can only vote for a semistable (respectively preferred) labelling, AWPR violates INCR2. If an argument is collectively accepted, it is possible that one of its defeaters is collectively undecided.
Proof.
Suppose argument has two defeaters, and . Suppose we have agents, with votes as shown in Figure 10. Clearly, while is collectively accepted because , one of its defeaters, namely , is collectively undecided because . ∎
Remark 7.
When agents can only vote for a semistable (respectively preferred) labelling, AWPR violates UNDECCR1. If an argument is collectively undecided, it is possible that one of its defeaters is collectively accepted.
Proof.
Suppose argument has two defeaters, and . Suppose we have agents, with votes as shown in Figure 10. Clearly, while is collectively undecided because , one of its defeaters, namely , is collectively accepted because . ∎
Remark 8.
When agents can only vote for a semistable (respectively preferred) labelling, AWPR violates UNDECCR2. If an argument is collectively undecided, it is possible that none of its defeaters is collectively undecided.
Proof.
Suppose argument has two defeaters, and . Suppose we have agents, with votes as shown in Figure 11. Clearly, while is collectively undecided because , none of its defeaters is collectively undecided. ∎
To sum up, the only restriction that would satisfy the Collective Rationality is the grounded semantics (Proposition 5). This is trivially true because only one grounded labelling exists. However, stable semantics violates Collective Rationality only because it violates OUTCR. As for the semistable and preferred semantics, they only satisfy INCR1, a property they inherit from the complete semantics. Refer to Table 1 for a summary of the results we have found.
Semantics  INCR  OUTCR  UNDECCR  

INCR1  INCR2  UNDCR1  UNDCR2  
Grounded  Yes  Yes  Yes  Yes  Yes 
(Prop. 5)  (Prop. 5)  (Prop. 5)  (Prop. 5)  (Prop. 5)  
Stable  Yes  Yes  No  Yes  Yes 
(Cor. 2)  (Lem. 1)  (Rem. 5)  (Lem. 2)  (Lem. 2)  
Semistable  Yes  No  No  No  No 
(Cor. 2)  (Rem. 6)  (Rem. 5)  (Rem. 7)  (Rem. 8)  
Preferred  Yes  No  No  No  No 
(Cor. 2)  (Rem. 6)  (Rem. 5)  (Rem. 7)  (Rem. 8)  
Complete  Yes  No  No  No  No 
(Prop. 4)  (Rem. 1)  (Rem. 2)  (Rem. 3)  (Rem. 4) 
9 Restricting the Domain of Argumentation Graphs to Satisfy Collective Rationality
In an earlier section, we showed that, AWPR violates Universal Domain and Collective Rationality. In this section, we investigate whether AWPR can satisfy Collective Rationality by restricting the argumentation framework to graphs with certain graphtheoretical properties. We show that graphs consisting of disconnected issues (a notion we define below) and graphs in which arguments have limited defeaters (in some sense) guarantee collectively rational outcomes when the AWPR is used.
9.1 Disconnected Issues
The notion of “issue” was defined in [7] in order to quantify disagreement between graph labellings. In this section, we use this notion to provide a possibility result.
Crucial to the definition of the “issue” is the concept of “insync”. Two arguments and are said to be insync if the (complete) label of one cannot be changed without causing a change of equal magnitude to the label of the other.
Definition 10 (inSync [7]).
Let be the set of all complete labellings for argumentation framework . We say that two arguments are insync ():
(4) 
where:

iff .

iff
This relation forms an equivalence relation over the arguments, and the equivalence classes are called “issues”.
Definition 11 (Issue [7]).
Given the argumentation framework , a set of arguments