Over-caution of large committees of experts ∗
Rune Midjord, Tomas Rodriguez Barraquer, and Justin Valasek
†February 19, 2013
Work in progress. We appreciate comments.
Abstract
In this paper, we provide an explanation for why committees may behave signif- icantly more risk averse than individuals. Specifically, we study a model in which a committee of experts must decide whether to approve or reject a proposal. Whether the proposal is beneficial to society or not depends on an unobservable underlying state of the world and each expert’s signal is only indirectly related to that state of the world by way of the experts’ common discipline of expertise. In addition to a payoff linked to the adequateness of the committee’s decision, each committee member can receive a disesteem payoff linked to his vote. An example is FDA drug advisory committees, where committee members can be exposed to a disesteem (negative) payoff if they vote to pass a drug that proves to be very harmful to some users. We show that no matter how small the disesteem payoffs are, information aggregation fails completely in large committees: under any fixed majority rule, the committee will reject the proposal with probability approaching one. This inefficiency is robust to pre-vote communication unless the decision is taken by unanimity. That is, information aggregation in large committees with disesteem payoffs is only efficient if there is pre-vote communication and each committee member is fully responsible for the committee’s decision.
Keywords: Committees, Information aggregation, Disesteem payoffs JEL Classification Codes: D71, D72
∗We thank Andrea Galeotti, Macartan Humphreys, Nagore Iriberri, Annick Laruelle, Henry Mak, Andrea Matozzi, Peter Norman Sorensen, Jes´us V´azquez, Fernando Vega-Redondo, Georg Weizs¨acker and seminar participants at EUI -Villa San Paolo- and -Villa La Fonte-, Public University of Navarra, and the University of the Basque Country. All errors are our own.
†University of the Basque Country, Hebrew University of Jerusalem, and WZB Berlin respectively. Con- tact e-mail: rune.midjord@ehu.es.
1 Introduction
The theoretical understanding of decision-making by groups of individuals dates back to Condorcet (1785), who showed that committees are an effective way to aggregate private information. This theoretical finding, however, contrasts with the commonly-held opinion that decisions made by committees are often not efficient. A specific observation that we seek to explain is why committees often seem more cautious than individuals, even in cases were all individuals have preferences that are aligned. This observation, while anecdotal, is pervasive and widely held (see discussion in Hao Li (2001)) .
In this paper, we provide an explanation for why committees behave more risk averse than individuals by formally modeling the case where committee members receive esteem payoffs linked to the accuracy of their individual vote, in addition to payoffs linked to the committee outcome.1 In the process, we uncover an important mechanism: committee decisions are very sensitive to disesteem payoffs, even when they are outweighed by payoffs related to the committee outcome. Specifically, we examine the following case: consider a committee of experts, such as a board of FDA reviewers, that makes a decision or recommendation over whether to accept a proposed innovation for public use, such as a pharmaceutical drug. Each committee member, just like each individual in society, prefers to accept the drug only if it is safe for use. The preferences of the committee members differ from society at large, however, in that if the committee passes a drug that proves to be fatal for some individuals, committee members will receive an additional negative (disesteem) payoff if they personally voted to approve the drug (we discuss the foundation for these payoffs in more detail below).
On one hand, it is unsurprising that committees behave more risk averse in this setting, since committee members with such disesteem payoffs are more risk averse than society at large.
We show, however, that this risk aversion is magnified by the collective action problem;
therefore, aggregate committee behavior is more risk averse even relative to the preferences its members. Moreover, no matter how small the disesteem payoffs are, a large committee will always behave infinitely risk averse. That is, information aggregation fails completely in large committees, and the committee will reject the innovation regardless of the information held by its members.
This result stands in stark contrast to the theoretical understanding of committees. The existing literature largely supports Condorcet’s original theorem that large committees will
1Brennan and Pettit (2004) introduce utility payoffs that are linked to esteem, where esteem captures an agent’s general regard by other members of society (also see their discussion of the relevant psychological literature). We argue that committee members are exposed to esteem payoffs to the extent that their decision is evaluated by the general public. Specifically, committee members can be held accountable for committee decisions that have ‘disastrous’ outcomes, a mechanism we explain in detail below.
make better decisions than individuals, as long as committee members have similar prefer- ences over outcomes and have some information about which outcome is preferable.
The FDA advisory committees for the approval of new drugs, are a good example of the kind of setting to which the model that we study applies. Among other responsibilities, the United States’ Food and Drug Administration (FDA) must approve or reject new drugs by means of an assessment process called a “new drug application” (NDA). As a result of going through this process a new drug is rejected as unsafe or is declared “safe and effective when used as directed”.2 One key input for the FDA administration stemming from this process is the independent expert advice from advisory committees. The FDA relies on such committees for gathering recommendations and information on various sorts of scientific issues, and in particular on the “approvability” of new products. The communication between the FDA and the advisory committees takes place in two ways: (1) The FDA learns from the discussions among committee members and individual recommendations (2) Committees vote on questions posed by the FDA. Although the FDA makes careful observation of the communication among committee members and takes into account the qualitative details of individual views, voting plays an important role in the advisory procedures because it can provide concrete feedback on specific questions.3 The main points of the guidelines for voting procedures provided by the FDA to its advisory committees are: (1) Open discussion should precede any voting. (2) The question for the vote should be simple (3) Voting should be simultaneous and the names of voters and their votes should be made part of the public record. There are currently 17 different FDA advisory committees addressing issues related to drugs, and the current number of voting members in each of these ranges from to 11 to 26.4
Another example is jury trials. Specifically, the jury members might receive a negative payoff if they vote to set a suspect free who then goes on to commit a murder. If they
2Additionally the NDA process also aims to provide sufficient information for establishing (1) whether the manufacturing process is sound and capable of producing the proposed drug in the proposed scale and over time and (2) what information should be included in the drug’s label.
3 As noted in the FDA’s guidelines for voting procedures: ‘Since all members vote on the same question, the results help FDA gauge a committee’s collective view on complex, multi-faceted issues.”, Guidance for FDA Advisory Committee Members and FDA Staff: Voting Procedures for Advisory Committee Meetings, August 2008
4In alphabetical order, with the number of members in parenthesis: Anesthetic and analgesic drug prod- ucts (13) , Anti-Infective drugs (13), Antiviral drugs (13), Arthritis drugs (11), Cardiovascular and Renal drugs (11), Dermatologic and ophtalmic drugs (15), Drugs safety and risk management (13), Endocrinologic and metabolic drugs (11), Gastrointestinal drugs (11), Medical imaging drugs (12), Non prescription drugs (14), Oncologic drugs (13), Peripheral and central nervous system drugs (11), Pharmaceutical science and clinical pharmacology (26), Psycopharmacologic drugs (11), Pulmonary allergy drugs (13), Reproductive health drugs (13).
vote to convict and the suspect is jailed for life, however, then the counterfactual is never observed: the suspect will never have a chance to commit murder. Therefore, jury members will have an incentive to hedge against a verdict of innocence.5
A common feature of our motivating examples share is that the committees consist of experts or individuals who all have access to a common body of evidence. This evidence, however, might not perfectly indicate the true state of the world. Therefore, we propose the theoretical concept of the state of the art, which is the decision the best available interpretation of the body of evidence would arrive at. The state of the art t can be thought of as the decision to which an ideal computer, programmed with the best available decision procedures and criteria for classifying all the evidence, would arrive at.
We then model the knowledge of each member of the committee as an disesteem departure from the state of the art; that is, each expert is a coarse embodiment of the state of the art.
The coarseness reflects idiosyncrasies at the individual decision making level such as possible errors of interpretation, conceptual misunderstandings, lapses of attention (all these often classified as “human error”), but also inspired hunches and extraordinary insights.
This view of each expert’s knowledge captures what we think is an important feature of the advisory committees in the kind of applications discussed previously. An expert’s sincere opinion may be wrong for purely disesteem reasons -in the sense that a hypothetical large majority in a randomly chosen set of experts in the discipline would hold the opposite and correct opinion- or he might be wrong because the state of the art of the discipline which he represents is in fact wrong. This approach differs from the standard model of committee behavior, where signals are generated by the state of the world, and hence a large committee that aggregates the signals efficiently will never make an error. In the state of the art model, since signals are generated by the state of the art, aggregating information effectively implies the ideal, but not necessarily correct, decision.
Another modeling feature we would like to highlight is the structure of the disesteem payoffs.
The motivation for this payoff is as follows: If the committee votes to approve the innovation, or to release a suspect, there is some probability of a ‘disastrous’ outcome. For example, a drug could have severe side-effects, or the suspect could commit a murder upon release. While the risk of a disastrous event is ‘priced’ into the common payoffs, these events also cause the realization of disesteem payoffs. Specifically, when a disastrous event occurs the committee’s decision, and hence to committee member’s vote, becomes salient, either through media
5This contrasts with a guilty verdict which is later proved to be incorrect, perhaps due to the introduction of DNA evidence. Unlike our motivating example, however, the decision is shown to be wrong given new evidence that was not available to the committee at the time of the trial. It is unclear disesteem payoffs are an appropriate assumption in this case.
attention, social and professional networks, or by causing internal deliberation. In these cases, conditional on the committee incorrectly approving a drug, the committee member faces the risk of a negative disesteem payoff if they personally voted to approve the drug.6 Since only committee members who vote for the innovation are exposed to the disesteem payoff, this gives them an incentive to hedge against approving the innovation. The main result of our paper is that this hedging behavior is magnified by the collective action problem in committees.
On one hand, the intuition behind our result comes from the familiar pivotal voter problem, cast in a new light: Since the probability of an individual’s vote being decisive decreases as the size of the committee grows, the incentive to vote to hedge against the disesteem payoff increases relative to the incentive to vote for the outcome which is more likely to be correct.
On the other hand, this intuition is incomplete, since as the size of the committee grows the probability that the committee incorrectly decides to approve the innovation also decreases.
The result therefore also depends on the state of the art. That is, while the probability of being pivotal approaches zero, the probability that a decision to accept to innovation is incorrect is always bounded away from zero: at best the committee can replicate the state of the art, which aways has a positive probability of an error.
Therefore, in large committees, the incentive to hedge against the disesteem payoff will dominate, and all committee members will vote to reject the innovation, regardless of their individual signal. This effect results in a tradeoff between increasing the number of commit- tee members, and hence the number of signals about which decision is correct, and decreasing the informativeness of each committee member’s vote.
1.1 Communication
In committees of experts, committee members most often deliberate prior to voting. In a setting without disesteem payoffs, deliberation will lead to single agent efficiency: committee members will share their signal, and will vote for the option that is socially optimal condi- tional on all information. In a setting with disesteem payoffs, however, deliberation is not effective for most decision rules.
6We argue that disesteem payoffs will realize even when the committee member’s vote is not decisive, and show that our results are robust to ‘dilution’. Additionally, we consider a negative disesteem payoff to be the relevant payoff in these applications, since salient positive shocks are unlikely to occur. This corresponds to the anecdotal evidence that committee decisions are only noticed when they go wrong. In a related project,
“Irresponsibility in Committees,” we analyze more general realizations of disesteem payoffs in the context of elected politicians, whose reelection probabilities might increase or decrease based on their individual vote.
First, even if committee members accurately share their individual signals, resulting in a common information game at the voting stage, efficiency is not achieved with disesteem payoffs. This follows from the fact that the reasoning behind the main result is robust to
‘better’ information: as the committee becomes large, the incentive to hedge against the disesteem payoff still dominates since the probability of the committee committing an error does not approach zero. Second, truthful reporting is not always an equilibrium in the presence of disesteem preferences: because the proportion of committee member’s voting to accept is ‘too low’ in equilibrium, there is also an incentive for committee members to misreport in an attempt to increase the proportion of committee member’s voting to accept.
Due to the strategic incentives implied by disesteem preferences, deliberation prior to voting does not generally result in efficiency. However, there is one decision rule which overcomes this problem: unanimity. If each committee member is fully responsible for the final decision (i.e. is pivotal), then deliberation results in full information sharing, and each committee member votes responsibly. This might explain why certain committees use a unanimity rule, such as juries, despite other disadvantages of this decision rule (see Feddersen and Pesendorfer (1998)).
The paper is organized as follows. Section 2introduces the payoff structure and the process that generates each expert’s opinion (signal). Specifically a committee composed by n experts must decide to approve or reject a proposed innovation using a q-rule, whereby if more than a fraction q of the committee members vote for approval, the innovation is approved and it is otherwise rejected. Whether the innovation is beneficial to society or not depends on an unobservable binary state of the world ω. If the innovation is rejected, committee members get a payoff of 0. If on the other hand, the innovation is approved, the state of the world is revealed and each expert gets a payoff of W > 0 if the innovation is beneficial. If the innovation is not beneficial all committee members get −C < 0, and those committee members having supported the approval get a further penalty of −K < 0. Each expert represents an independent coarsening of the state of the art of the area of knowledge that is being called upon to pass judgement on the innovation. Concretely, we represent the state of the art by an unobservable binary random variable t which matches the unobservable true state of the world with an exogenously given precision.7 The binary signal of each expert si, correctly reflects the state of the art t with probability 1 − ε. Conditional on the state of the art, the experts’ opinions are independent and identically distributed draws.
Section 3 studies in detail two related problems in order to contextualize the paper’s main result. First, it considers the hypothetical case of a single agent who can directly observe
7We model the precision by way of exogenously given probabilities of a type I error and type II errors, α and β.
n expert signals, and who must decide whether to approve or reject the innovation and whose payoffs have the same structure as those introduced in Section 2. This single agent solves the problem by using a threshold, such that if the fraction of signals recommending approval exceeds this threshold he approves it, and otherwise he rejects it. The specific threshold depends on the agent’s risk preferences and the number of signals that he can observe. However, the influence of the specific risk preferences on this optimal threshold fades away as the number of signals increases. In the limit, the threshold is the same for all risk preferences.8 With the single agent benchmark in place, Section 3.2 shows that in the absence of disesteem payoffs (K = 0), the committee of n experts behaves exactly as the single agent with the exact same payoffs would, when the q-rule used by the committee coincides with the threshold that would be used by the agent. This coincidence is not new, in fact, it is just a rendering of the results of Austen-Smith and Banks (1996) in our model.
When K = 0, the difference between the two models is that in Austen-Smith and Banks (1996), the expert signals are independent conditioning on the state of the world, while in our model they are independent conditioning on the state of the art -and therefore in fact, not independent conditioning on the state of the world-. These two benchmark cases, together, provide a robust rationale for the delegation by a principal of a decision to a committee of experts in the absence of disesteem payoffs in the payoff functions of committee members.
And furthermore it lends support to the folk idea that the larger the committee of experts, the better. In fact, even if the risk preferences of the principal and the committee members differ, the behavior of the committee under simple majority converges to the optimal behavior of the principal (if he had direct access to the expert signals) as the number of experts n goes to infinity.9
Section 4 studies decision making by the committee when K > 0, and presents the paper’s main result, which is that as the committee size grows, the probability that it rejects the innovation converges to 1. In Appendix B we show that the result also holds when the disesteem payoffs converge to 0, provided that they do so slowly enough. Finally in section 5 we discuss the fact that pre-play communication does not alter the conclusions, and end with a redemptive proposition of the notion that large groups of experts can provide better advice than smaller groups of experts, in a world with disesteem payoffs. Specifically we show that if the committee relies on a unanimity decision rule10, then it is an equilibrium for all agents to reveal their signals during pre-voting cheap talk, and the committee can replicate the single agent’s behavior. The above discussion of the usefulness of an advisory
8In the reasonable range of risk preferences within which if the agent could observe the state of the art directly, he would approve the innovation if the state of art suggested him to do so, and would reject the innovation otherwise.
9See footnote 10.
10The unanimity decision rule cannot be represented as a fixed q-rule for all n.
committee to a principal when there are no disesteem payoffs, even if he does not share the risk preferences of the committee members, therefore also applies to this case.
1.2 Related Literature
This paper contributes to the game theoretic literature on committee decision making, see Gerling et al (2005) and Li and Suen (2009) for surveys. Consider a committee confronted with the choice between the status quo and an alternative. The decision is made under uncertainty on the binary state of the world and each committee member (expert) receives an independent and imperfect signal of the true state. The experts have identical preferences over the final outcome, which is determined by the committee’s decision and the realized state. The objective of the experts, and the committee as a whole, is to match the decision with the state. According to Condorcet’s (1785) jury theorem, the majority of experts who vote independently on whether to accept the proposal or not are more likely to make the optimal decision than a single expert and as the number of experts goes to infinity the probability that the committee makes the right decision converges to one. It follows that if the experts’ signals are noisy versions of the state of the art instead of the actual state of the world and the state of the art is a good enough predictor of the state of the world, then the committee’s decision would converge to the recommendation of the state of the art, as the number of experts increases.
An underlying assumption behind the Condorcet jury theorem is that the committee mem- bers vote according to their private signal (informative voting). In their seminal paper, Austen-Smith and Banks (1996) shows that informative voting is generally not rational.
What matters for expert i is his vote in the event that he is pivotal and conditional on being pivotal he deduces, given the other experts’ equilibrium strategies, information about their signals and in this light his own signal is unlikely to influence his vote.
This is not to say that the Condorcet Jury theorem fails to hold when the experts can vote strategically. McLennan (1998) proves that whenever there is a profile of votes and voting rule for which the jury theorem holds, there exists a profile of votes that achieves the same outcome and is a Nash equilibrium of the voting game. Feddersen and Pesendorfer (1998) shows that in the standard setting there exists a unique symmetric equilibrium that almost surely selects the right outcome as the number of voters becomes large. Peleg and Zamir (2012) provides a generalization of the theorem for exchangeable sequences of signals11 and
11A sequence of signals s1, s2, s3, ... is said to be exchangeable if for all k and all permutations π of 1, 2, 3, ..., k, the distribution of s1, s2, ..., sk is the same as that of sπ(1), sπ(2), ..., sπ(k). Sequences of i.i.d signals are a special case, and so are the sequences of signals (i.i.d draws, conditional on a binary underlying state of the art) that we consider in this paper.
fully taking into account all strategic considerations.
Another test of the Condorcet jury theorem is when the jury members have different “stan- dards of proof” (heterogeneous preferences) with respect to convicting an innocent defendant and acquitting a guilty defendant. Gerardi (2000) develops a model of collective decision making where jury members with conflicting interests have to aggregate private signals in order to make an informed decision. For any non-unanimous voting rule the probability to convict the innocent as well as the probability to acquit the guilty converge to zero as the committee size goes to infinity. In a similar vein, Feddersen and Pesendorfer (1998) demon- strates how unanimity voting rules are inferior to other rules under strategic voting and no pre-voting deliberation. Feddersen and Pesendorfer (1997) analyzes two-candidate elections when voters have different preference types. Each voter knows his own preference type but not the other voters’ types. Every voter receives a private signal that is correlated with the true state of the world. As the size of the electorate goes to infinity, the fraction of voters who condition their votes on their private information goes to zero. Nevertheless, voting fully aggregates information in the sense that with almost certainty the alternative is elected that would have been chosen if all private information was common knowledge. Yildirim (2012) considers ex post optimal voting rules when committee members have interdepen- dent valuations and shows that the commitment problem can be alleviated by increasing the committee size.
Our model of strategic voting differs from the above mentioned literature in one important respect: An expert’s utility can be affected by his vote conditional on not being pivotal.
More concretely, the committee members are exposed to a disesteem payoff in case they vote to accept an alternative which turns out to be a failure (e.g. an FDA approved drug that proves to be fatal for some users). In this way, our idiosyncratic payoff depends on the state, the committee’s decision, and individual votes. Several papers consider voting when voters have common interest with respect to making the right decision and additional “selfish”
concerns.
Callander (2007) considers voting under majority rule when voters care about selecting the best candidate and voting for the winning candidate. The additional conformity payoff de- pends on individual votes and the majority winning candidate and, contrary to our disesteem fear, the extra payoff is independent of the true state (i.e. who is the more competent can- didate). Considering optimal equilibria as the population becomes large, Callander (2007) shows that in elections without a dominant front-running candidate the better candidate is almost surely elected, whereas in races with a front-runner information cannot be fully aggregated in equilibrium.
There is a significant number of papers on committees in which committee members receive
an additional reputation payoff if they “hit” the state with their vote. Different from our model, this reputation payoff is independent of the committee’s decision. In Ottaviani and Sorensen (2001) the experts only care about making the right recommendation (i.e. voting together with the true state). Each expert is of unknown ability type (good or bad), where a good expert has a more precise signal. When the experts speak sequentially reputation concerns give rise to herding (experts suppressing their private signals, see Scharfstein and Stein (1990)) and the expected decision quality may decrease when an expert’s signal quality increases. When the voting is simultaneous the first best can be achieved if the probability distribution over the binary state variable is not too skewed. Levy (2007) studies the effects of transparency when committee members care about their reputation for matching their vote to the state.
Visser and Swank (2007) consider a committee deliberating and voting on whether to approve a project or maintain the status quo. The committee members care about the value of the project and their reputation for being well informed (high versus low signal precision). The
“market”, the people whose judgement committee members care about, forms a belief about the competence levels of committee members. The market does not observe the value of the project, only the decision taken. Visser and Swank (2007) show that reputational concerns make the a priori unconventional decision more attractive and lead committees to show a united front. As the number of committee members grows, the committee decision becomes a weaker indicator of signal concurrence in the committee which lowers the reputation concerns and leads on average to better decisions.
Morgan and Vardy (2013) study a model in which voters are driven by instrumental and expressive values. The additional expressive motive provides voters with some consumption utility if they vote in a particular way (e.g. in accordance with one’s norms) that is irrespec- tive of the true state and the implemented decision. Some voters will be in conflict because their signal goes against their expressive motive. If the degree of conflict is low and thus the expressive preferences are mostly shaped by the facts (signals) then the Condorcet jury theorem holds and large voting bodies make correct decisions. However, when expressive preferences are relatively impervious to facts large voting bodies do no better than a coin flip. For large voting bodies, the chance of being decisive is negligible and the expressive motives dominates.
Another branch of the literature considers committee efficiency when individual committee members pay a cost of effort for becoming informed. Information is endogenous and each committee member initially decides how much effort to spend on gathering information. The costly information acquisition creates a free rider problem and resulting underinvestment in effort. The free rider problem may exacerbate for larger committees as the probability that each member’s information is pivotal decreases. Several papers come to the conclusion
that optimal committee size is bounded, see e.g. Li (2001), Persico (2003), Mukhopadhaya (2005), and Koriyama and Szentes (2009). However, the conclusion on the boundedness of the optimal committee size seems to depend on whether information acquisition is a discrete or continuous choice. Martinelli (2006) shows that if the size of the committee converges to infinity, then there is a sequence of symmetric equilibria in which each member invests only little in effort, and the probability of a correct decision converges to one.
2 The Model
A proposal is submitted for approval by a committee of experts that operates according to a q-rule: If strictly more than a fraction q of the committee members i ∈ {1, 2, ..., n} vote in favor of approval then the proposal passes and otherwise it is rejected. We denote the votes of each committee members i ∈ {1, 2, ..., n} by vi ∈ {a, r}, and the decision of the committee by X ∈ {a, r}. The payoff to each agent i depends on the decision of the committee, an underlying state of the world ω ∈ {A, R} and the agent’s vote vi
U (X, vi, ω) =
0 if X = r
W if X = a, ω = A
−C if X = a, ω = R, vi = r
−(C + K) if X = a, ω = R, vi = a where W, C, K > 0.
One interpretation of the structure of the payoffs is as follows. If the proposal is rejected then payoffs to all agents in the committee are 0 reflecting that the status quo is preserved and no further information about its adequacy with respect to the true state of the world is generated. If the proposal is approved then the true state of the world is revealed and the committee members receive a common payoff and an individual disesteem payoff. The common payoff is W or −C depending on whether the committee has made the right decision with respect to the state of the world. The individual disesteem payoff is only awarded in the case that the committee has made the wrong decision, and is −K only for the agents that supported that wrong decision and is 0 otherwise 12.
12K can be thought of as the probability that the decision is disastrously wrong, e.g. side effects exist and are fatal, times the negative payoff that accrues to committee members who supported the committee’s decision. The structure of the payoffs is motivated by the idea that it is often the case that when committees make the right decisions, they go largely unnoticed, and in particular the role of individual committee members is ignored. But when for any reason a committee makes the wrong decision, then its operation and in particular the responsibilities of individual members may be scrutinized.
Note that if K is small these payoffs represent a seemingly small departure towards hetero- geneity from a pure common values situation, in which the payoffs to all committee members are identical in all possible events. As our main result Proposition2 shows, for a sufficiently large committee this small departure can imply a large difference in equilibrium behavior.
2.1 The state of the art and expert’s opinions (signals)
We denote by pA = p(ω = A) society’s prior (uninformed) beliefs on the state of the world.
We think of the committee members as experts in a single given relevant discipline for the decision at hand. We model the knowledge of each member of the committee as an id- iosyncratic departure from the state of the art of that discipline. This view of each expert’s knowledge captures what we think is an important feature of the advisory committees in the kind of applications discussed in the introduction. An expert’s sincere opinion may be wrong for purely idiosyncratic reasons -in the sense that a hypothetical large majority in a randomly chosen set of experts in the discipline would hold the opposite and correct opinion- or he might be wrong because the state of the art of the discipline which he represents is in fact wrong.
The state of the art:
We denote the state of the art by t ∈ {a, r} and let α = p(ω = A|t = r), 0 < α < 12 denote the probability that the state of the art is wrong when it indicates r (t = r) -a type I error- and β = p(ω = R|t = a), 0 < β < 12 denote the probability that the state of the art is wrong when it indicates a, (t = a) -a type II error-.
In terms of the example of the FDA recommendation committee, there is a collection of hard evidence -the whole battery of tests available to every committee member- and the state of the art t can be thought of as the decision to which an ideal computer would arrive at, programmed with all the decision procedures of medical science and state of the art criteria for classifying all the evidence into an “r set” and an “a set”.
The signals:
We think of an expert as a coarse embodiment of the state of the art. The coarseness reflects idiosyncrasies at the individual decision making level such as possible errors of interpreta- tion, conceptual misunderstandings, lapses of attention (all these often classified as “human error”), but also inspired hunches and extraordinary insights. We further assume that these individual differences with respect to the state of the art are purely idiosyncratic, in the sense that conditioning on t, the sincere opinions of different experts (which we henceforth refer to as signals) are independent. Concretely, we assume that with probability 1 − ε the
signal of expert i, si, coincides with the state of the art t (si = t) and with probability ε it differs with respect to the state of the art.13 That is, p(si = t|t) = 1 − ε and p(si 6= t|t) = ε, where ε < 12.
3 Benchmark models
The main model in our paper has two main differences with respect to the standard models in the voting in committees literature when agents are concerned about making the correct decision. The first difference is that the truth is revealed (and thus impacts payoffs) only if one of the two possible alternatives is chosen by the committee. The second is that each agent perceives an individual disesteem payoff, which depends not only on the decision taken by the committee but also on his vote. In order to obtain insights on the origin of the results in our main model it is useful to introduce these two variations in stages. In this section we analyze two benchmark decision processes. In the first one there are no tensions arising from collective action. For that purpose we consider a committee consisting of a single individual that receives n signals, and whose payoffs are identical to the ones that we consider in the case of the multiple agent committees. In the second one there is collective action (multiple members in the committee), but common payoffs in all events, and thus no conflict.
3.1 Decision making by just one agent
We consider a hypothetical situation in which a single agent is able to gather the genuine opinions of n experts, {s1, s2, ..., sn}. Denoting the agent’s decision by v, his payoffs are given by:
U (v, ω) =
0 if v = r
W if v = a and ω = A
−(C + K) if v = a and ω = R The agent finds it optimal to accept the proposal (v = a) if and only if:
p(ω = A|s1, ..., sn)
p(ω = R|s1, ..., sn) ≥ C + K
W (1)
13The state of the art can be thought of in an alternative, more constructive way. Rather than thinking of the opinions of the experts as idiosyncratic distortions of a pre-existing state of the art, we can think of the state of the art as the probability limit of the average of the signals 1
n lim
n→∞
n
X
=1
siand explicitly set forth conditions which would deem the signals conditionally independent given this limit.
≡ p(s1, ..., sn|ω = A)
p(s1, ..., sn|ω = R) ≥ ρ 1 − pA pA
(2)
where letting ρ = C+KW , the above inequality just follows from Bayes rule. Letting πA = p(t = a|ω = A) and πR= p(t = r|ω = R) denote the probabilities that the state of the art is right in each of the two states of the world, and given the independence of the signals from the state of the world conditional on the state of the art, (2) is equivalent to:
≡ p(s1, ..., sn|t = a)πA+ p(s1, ..., sn|t = r)(1 − πA)
p(s1, ..., sn|t = a)(1 − πR) + p(s1, ..., sn|t = r)πR ≥ ρ 1 − pA pA
≡ p(s1, ..., sn|t = a)
p(s1, ..., sn|t = r) ≥ M if ρ < 1 − β
β (3a)
p(s1, ..., sn|t = r) 1 − α α − 1
ρ
≤ 0 if ρ = 1 − β
β (3b)
p(s1, ..., sn|t = a)
p(s1, ..., sn|t = r) ≤ M if ρ > 1 − β
β (3c)
where M =
1−α α − 1ρ
1 ρ −1−ββ
! 1 − πA πA
noting that by Bayes’ theorem (1−ππApA
R)(1−pA) = p(ω=A|t=a)p(ω=R|t=a) = 1−ββ and (1−ππ A)pA
R(1−pA) = p(ω=A|t=r)p(ω=R|t=r) =
α
1−α. From (3a), (3b) and (3c) and relying on the independence of the signals of different experts conditional on the state of the art, it follows that the agent’s decision rule is given by:
• If ρ ≤ 1−αα ⇒ the agent finds it optimal (strictly) to accept proposal, regardless of the vector of signals.
• If 1−αα < ρ < 1−ββ ⇒ the agent finds it optimal to accept the proposal if and only if:
h n ≥ 1
2
1
nlog (M ) + log 1−εε log 1−εε
!
, where h = |{i : si = a}| (4)
• If 1−ββ ≤ ρ ⇒ the agent finds it optimal to reject the proposal , regardless of the vector of signals.
The last case represents a situation in which the relative cost of incorrectly accepting the proposal is so high with respect to the odds that the state of the art is correct when indicating an approval, that the agent would not take the risk of approving the proposal even if he had direct access to the state of the art. Similarly, the first case represents situations in which the agent can never be confident enough to reject the proposal. Throughout what follows, we
will be interested in intermediate situations in which a single agent14 would find it optimal to accept the proposal under some realizations of the vector of signals and reject it under other realizations.
Another feature of the model worth noting, is that regardless of the precision of the state of-the-art (α, β), as the number of expert signals increases, the agent’s decision threshold converges to 12. The reason is that due to the fact that the noise in the expert’s signal is symmetric (p(si = t|t = a) = p(si = t|t = b) for sufficiently large n, the majoritarian signal among the experts, coincides with very high probability with the state-of-the-art evidence.
Whether the acceptance threshold is smaller or greater than 12 along the sequence depends on whether M is greater or smaller than 1.
For future reference we denote the decision threshold in the intermediate case by q(M,1−εε , n).
That is, q(M,1−εε , n) = 12
1
nlog(M )+log(1−εε )
log(1−εε )
3.2 Decision making in a committee without disesteem payoffs
We now analyze the problem of decision making by committees in the absence of disesteem payoffs.
Payoffs: The utility of each committee member is given by:
U (X, ω) =
0 if X = r
W if X = a, ω = A
−(C + K) if X = a, ω = R
Let |Hi| = |{j 6= i : vj = a}|. After observing his signal si, expert i will find it optimal to vote for approval vi = a if and only if:
W
p |Hi| + 1
n > q, ω = A|si
− p |Hi|
n > q, ω = A|si
− (C + K)
p |Hi| + 1
n > q, ω = R|si
− p |Hi|
n > q, ω = R|si
≥ 0
Denoting the event that i is pivotal (|Hni| ≤ q < |Hin|+1) by pivi, the above is equivalent to:
p(pivi, ω = A|si) p(pivi, ω = R|si) ≥ ρ
≡ p(ω = A|pivi, si)
p(ω = R|pivi, si) ≥ ρ (10)
14With preferences given by ρ.
≡ p(pivi, si|ω = A)
p(pivi, si|ω = R) ≥ ρ 1 − pA pA
(20)
≡ πAp(pivi, si|t = a) + (1 − πA)p(pivi, si|t = r)
(1 − πR)p(pivi, si|t = a) + πRp(pivi, si|t = r) ≥ ρ 1 − pA pA
The last equivalence follows, as in the case of a single decision maker, from the independence of the experts’ opinions from the state of the world ω conditional on the state of the art t.
p(pivi, si|t = a)
p(pivi, si|t = r) ≥ M (30)
We therefore have that the expert will find it optimal to vote for approval of the proposal when his signal is a and for rejection when his signal is r if and only if :
ε 1 − ε
M ≤ p(pivi|t = a)
p(pivi|t = r) ≤ 1 − ε ε
M (40)
We use (4’) to fully characterize the set of symmetric equilibria of the game. In what follows we will we will use σi : {a, r} → [0, 1] to denote the possibly mixed strategy according to which agent i sets vi = a with probability σi(a) after receiving signal si = a, and sets vi = r with probability σi(r) after receiving signal si = a. We let:
E[U (σi, X, ω)|si, σ] = X
vi∈{a,r}
σi(si) X
X∈{a,r}
X
ω∈{A,R}
p(X, ω|(σ, σi), si)U (vi, X, ω)
where (σ, σi) denotes the strategy profile under which each player other than i behaves according to σ and i behaves according to σi. We will denote by E[U (vi, X, ω)|si, σ], the expected payoff to agent i when upon receiving signal si he takes action vi with probability 1.
3.2.1 Equilibrium
Throughout the analysis we rely on the concept of Bayesian Nash equilibrium and focus on symmetric strategies only -all agents use the same decision rule-.
Definition 1 (Symmetric Equilibrium) A strategy, σ, is a symmetric equilibrium if and only if for all i ∈ {1, 2, ..., n}, si ∈ {r, a} and v ∈ {a, r}:
E[U (σ, X, ω)|si, σ] ≥ E[U (v, X, ω)|si, σ]
Proposition 1 (Symmetric equilibria)
Let α, β, pA, ρ and n be given and let z = nq − bnqc.15 For all values of q there are two
15bxc denotes the largest integer smaller or equal to x.
babbling equilibria: one in which each agent sets vi = a regardless of si and one in which each agent sets vi = r regardless of si. In addition to these babbling equilibria we have the following symmetric equilibria (a unique equilibrium within each appropriate q-range) in which the behavior of the agents reveals some information about their signals:
1. If q(M,1−εε , n) −1−zn ≤ q ≤ q(M,1−εε , n) +nz, then the unique non-babbling equilibrium is the informative one σT. Where σT(a) = 1 and σT(r) = 0.
2. If
1 nlog(M )
log(1−εε )−1−zn < q < q(M,1−εε , n) −1−zn then the unique non-babbling equilibrium is given by σ(r) = 0 and σ(a) = (1−ε)−εG1−G , where G =
M 1−εε bnqc+1n−1−bnqc1 . 3. If q(M,1−εε , n)+zn < q <
1
nlog(M )+log(1−εε )
log(1−εε ) +zn then the unique non-babbling equilibrium is given by σ(a) = 1 and σ(r) = (1−ε)−εH(1−ε)H−ε, where H =
M 1−εε n−bnqcbnqc1 .
The existence of the babbling equilibria follows from the fact that in each of the two cases, when 0 ≤ q ≤ n−1n , the agent is never pivotal and therefore both sides of the expression leading to equation (1)0 are 0 and the inequality holds weakly. As a result, the agent is indifferent between setting vi = a and vi = r, and in particular finds it optimal to vote according to the corresponding babbling equilibrium.
An intuitive way of understanding the symmetric non-babbling equilibria is as follows. First note that since ε < 12, condition (40) implies that σ(a) ≥ σ(r) in all equilibria. More strongly, if an agent weakly prefers to set vi = a upon receiving signal r he will strictly prefer to set vi = a upon receiving signal a. Similarly if an agent weakly prefers to set vi = r upon receiving signal a he will strictly prefer to set vi = r upon receiving signal r. This implies that there are only two types of symmetric equilibria: 1) σ(r) = 0 and σ(a) ≥ 0 and 2) σ(r) ≥ 0 and σ(a) = 1.
When the committee’s decision threshold q is within the n1 window around q(M,1−εε , n) described in part 1, then when every other agent is playing according to σT and conditioning on being pivotal, agent i finds himself in exactly the same situation as he would be if he knew all the signals and the decision was completely up to him. The reason is that all that matters for his decision is the number of a signals and the number of r signals. Given that he is pivotal it must be the case that there are bnqc a signals among the other players. If q is below q(M,1−εε , n), the condition of being pivotal, assuming everyone else is truthfully revealing their signal, implies a smaller number of a signals than what he considers necessary for approving the proposal, so he would set vi = r . In order to restore his willingness to set vi = a, it must be the case that the fact of being pivotal leads him to infer a higher number
of a signals than bnqc. The only way in which this can be the case is if the other agents are under-reporting their a signals. Moreover, it must be the case that σ(r) = 0. A similar argument shows that as q rises above q(M,1−εε , n), then the unique non-babbling equilibrium involves σ(a) = 1 and σ(r) > 0. The algebraic proof, which can be found in Appendix A, is useful for seeing uniqueness besides allowing the derivation of concrete expressions for the mixed equilibria.
One of the crucial insights that we obtain from Proposition 1 is that when the q-rule cor- responds to the threshold that would be used by a single agent having all the signals, to deicide whether to approve the proposal or not, then the unique non-babbling equilibrium is the informative one. That is, in a purely common values setting there is no conflict and the committee can replicate the optimal decision of a single agent holding all the information and having the exact preferences as those of any one committee member. This result is anal- ogous to those discussed in detail in Austen Smith Banks (1996) in the context of experts that hold signals that are independent conditionally in the sate of the world.
We denote the game with n players, with an approval threshold of q by G0q,n. Corollary 1 (Equilibria as n grows)
In the limit as n → ∞ the sequence of unique non babbling equilibria of the sequence of games G0q,n converge to:
1. σT if q = 12
2. σ(r) = (1 − ε) − ε 1−εε 1−qq (1 − ε) 1−εε 1−qq
− ε
, σ(a) = 1 if 12 < q ≤ 1
3. σ(r) = 0, σ(a) = 1−(1−εε )1−qq
(1−ε)−ε(1−εε )1−qq if 0 ≤ q < 12
This corollary follows directly from Proposition 1 by taking limits as n → ∞. It indicates that while the members of a committee with a simple majority rule will in general not be reporting their signals truthfully as long as M > 0, when the committee size increases they become arbitrarily close to doing so. It also indicates that as n grows the committee’s decision rule (as a way to partition the signal space into an approval/rejection decision) approximates arbitrarily well the optimal decision of a representative single agent having all the information at his disposal. Formally, given the sequence of games G0q,n and a corresponding sequence of equilibria σn we say that the decision of the committee converges to the state of the art16,
16Precisely, to the recommendation of the unobservable state of the art t.
if limpσn(X = t) = 1. As long as p(t = a) > 0 and p(t = r) > 0 this is equivalent to limpσn(X = a|t = a) = 1 and limpσn(X = r|t = r) = 1. Since no sequence of equilibria σn which visits infinitely often one of the babbling equilibria can converge to the state of the art, we restrict attention to the sequence of non-babbling equilibria, and in what follows drop the subscript σn and use the notation p(X = a|t = a) and p(X = r|t = r). The following corollary, which follows from Corollary 1 and the law of large numbers, shows that for all values of q ∈ (0, 1) the decision of the committee converges to that of the state of the art.
It is a version in our setting of Theorem 3 of Feddersen and Pesendorfer (1997).
Corollary 2 (Convergence to the state of the art)
When committee members act according to the non-babbling equilibrium the decision of the committee converges to the state of the art for all q ∈ (0, 1).
Proof of Corollary 2: As discussed above, the committee’s decision converges to the state of the art if limp(X = a|t = a) = 1 and limp(X = r|t = r) = 1.
By the law of large numbers and the independence of the signals conditional on the state of the art t, this will be the case if:
n→∞lim p(vi = a|t = r) < q < lim
n→∞p(vi = a|t = a)
where p(vi = a|t = a) = (1 − ε)σ(a) + εσ(r) and p(vi = a|t = r) = εσ(a) + (1 − ε)σ(r). By Corollary1 it then follows that the decision of the committee converges to that of the state of the art for q ∈ (0, 1) if:
(1) When q = 12, ε < q < 1 − ε
(2) When q < 12, ε 1 − 1−εε 1−qq
(1 − ε) − ε 1−εε 1−qq < q < (1 − ε) 1 − 1−εε 1−qq (1 − ε) − ε 1−εε 1−qq (3) When q > 12, ε + (1 − ε)(1 − ε) − ε 1−εε 1−qq
(1 − ε) 1−εε 1−qq
− ε
< q < (1 − ε) + ε(1 − ε) − ε 1−εε 1−qq (1 − ε) 1−εε 1−qq
− ε The statement of the corollary follows by noting that these inequalities always hold when 0 < ε < 12 and q is in the corresponding subset.
4 Decision making in a committee with disesteem pay- offs
We now analyze the behavior of the committee with individual disesteem payoffs.
Payoffs: The utility of each agent is given by:
U (X, vi, ω) =
0 if X = r
W if X = a, ω = A
−C if X = a, ω = R, vi = r
−(C + K) if X = a, ω = R, vi = a
After observing his signal si, expert i will find it optimal to vote for approval (vi = a) if and only if:
W p |Hi| + 1
n > q, ω = A|si
− (C + K)p |Hi| + 1
n > q, ω = R|si
≥ W p |Hi|
n > q, ω = A|si
− Cp |Hi|
n > q, ω = R|si
≡ W p (pivi, ω = A|si) − Cp (pivi, ω = R|si) ≥ Kp |Hi| + 1
n > q, ω = R|si
(100) We denote the game with disesteem payoffs K, with q-rule q and n players GKn,q. The babbling strategy profile whereby committee members vote to reject the proposal regardless of their signals is an equilibrium for every value of q. The main result of the paper is captured by the following proposition. Strengthened by Corollary 3 it shows that as n grows, the behavior of the committee given any equilibrium of GKn,q converges to that of its behavior under this babbling strategy. That is, the committee converges to always rejecting the proposal.
Proposition 2 (The probability of acceptance converges to 0 as n → ∞)
Let K > 0 and consider the sequence of games GKn,q and any sequence of symmetric strategy profiles σn, such that for each n, σn is an equilibrium of GKn,q. We let pσn(X = a) denote the probability that the committee accepts the proposal in game GKn,q, playing according to σn. Then, pσn(X = a) → 0 as n → ∞. That is, for all δ > 0, there exists nδ such that for all n > nδ, pσn(X = a) < δ.
Proof of Proposition 2:
We prove the proposition by contradiction. Suppose that it is not true. That is, suppose that there exists a sequence of symmetric strategy profiles σn such that for each n, σn is an equilibrium of GKn,q and pσn(X = a) does not converge to 0. This implies that there exists δ > 0 such that for every m, there exists nm > m with pσnm(X = a) > δ.
Let i be any agent. Then, by expression (1”) i finds it optimal to set vi = a upon receiving signal si if and only if:
W pσn(pivi, ω = A|si) − Cpσn(pivi, ω = R|si) ≥ Kpσn |Hi| + 1
n > q, ω = R|si
(100)
The argument is divided into two parts. First, we show that if pσn(X = a) > δ then Kpσn|H
i|+1
n > q, ω = R|si
≥ Kδmin{βp(t = a|si), (1 − α)p(t = r|si)}. Second, we show that the LHS of (1”) has an upper bound that is independent of σn and which converges to 0. Then putting the two together we arrive at a contradiction of the assumption that pσn(X = a) does not converge to 0.
Part one: lower bound on the RHS Note that:
Kpσn |Hi| + 1
n > q, ω = R|si
= Kpσn |Hi| + 1
n > q, ω = R, t = a|si
+ Kpσn |Hi| + 1
n > q, ω = R, t = r|si
= Kpσn |Hi| + 1
n > q, ω = R|t = a
p(t = a|si) + Kpσn |Hi| + 1
n > q, ω = R|t = r
p(t = r|si)
where the second equality follows from the fact that conditional on t, si is independent of the state of the world ω and of the other committee members’ signals.
Now note that pσn(X = a) = pσn(X = a|t = a)p(t = a) + pσn(X = a|t = r)(1 − p(t = a)).
It must therefore be the case that at least one of (I) pσn(X = a|t = a) > δ or (II) pσn(X = a|t = r) > δ holds. First lets assume (I) holds, pσn(X = a|t = a) > δ.
Note that pσn(X = a|t = a) ≤ pσn|H
i|+1
n > q|t = a
where Hi = {j 6= i : vj = a}, and therefore (I) implies pσn|H
i|+1
n > q|t = a
> δ which in turn implies pσn(|Hin|+1 > q, ω = R|t = a) > δp(ω = R|t = a), since:
pσn |Hi| + 1
n > q, ω = R|t = a
= pσn |Hi| + 1
n > q|ω = R, t = a
p(ω = R|t = a)
= pσn |Hi| + 1
n > q|t = a
p(ω = R|t = a)
where the last equality follows from the fact that the voting behavior of the agents only depends on their signals and these are independent from ω conditional on t. We can therefore conclude that:
Kpσn |Hi| + 1
n > q, ω = R|si
≥ Kδp(ω = R|t = a)p(t = a|si) = Kδβp(t = a|si)
If (I) does not hold, then it must be the case that (II) holds, pσn(X = a|t = r) > δ, case in which we obtain pσn(|Hin|+1 > q, ω = R|t = r) > δp(ω = R|t = r) and we can conclude:
Kpσn |Hi| + 1
n > q, ω = R|si
≥ Kδp(ω = R|t = r)p(t = r|si) = Kδ(1 − α)p(t = a|si).
