Your Loss Is My Gain: A Recruitment Experiment with Framed Incentives

Your Loss Is My Gain:

A Recruitment Experiment with Framed Incentives ^∗

Jonathan de Quidt

First version: November 10, 2013 This version: November 4, 2014

JOB MARKET PAPER

Latest version: http://jondequidt.com/JMP

As predicted by loss aversion, numerous studies find that penalties elicit greater effort than bonuses, even when the underlying payoffs are identical except for how they are framed. Loss aversion also predicts that workers will demand higher wages to accept penalty contracts, a plausible explanation for why such contracts are rare. I recruited data entry workers under framed incentive contracts to test the second prediction. I reproduce the effort effect, but surprisingly penalty framing increased the job acceptance rate by 25 percent.

Follow-up experiments rule out explanations based on different beliefs induced by the frame and demand for penalties for self-commitment purposes. For example, the strong preference for penalties replicates in a guessing task with a 50 percent success probability, independent of effort. I argue that the most likely explanation is salience, whereby workers’ valuations are influenced by the mechanically higher “base pay” under penalty contracts. The results have implications for firms and theory, but the puzzle of why penalties are rarely used remains.

Keywords: loss aversion; reference points; framing; selection; salience;

Mechanical Turk

JEL Classification: D03, J41, D86

∗I thank STICERD for financial support, and many people for helpful discussions, particularly Philippe Aghion, Oriana Bandiera, Roland Bénabou, Tim Besley, Gharad Bryan, Tom Cunningham, Ernesto Dal Bó, Erik Eyster, Greg Fischer, Maitreesh Ghatak, Paul Heidhues, Dean Karlan, Matthew Levy, George Loewenstein, Rocco Mac- chiavello and Torsten Persson, seminar participants at the LSE, Berlin, IIES and NHH Bergen; IEA World Congress 2014, FUR 2014, EEA/ESEM 2014 and ESA North Amer- ica 2014. An earlier draft was circulated under the title “Recruiting Workers Under Framed Incentives: An Online Labor Market Experiment.”

†LSE and IIES, Stockholm University. email: jonathan.dequidt@iies.su.se.

Consider two job contracts, the first of which pays a base wage of $100, plus a bonus of $100 if a performance target is reached, while the second pays a base wage of $200, minus a penalty of $100 if the target is not reached. Rational agents will behave identically under either one. How- ever, a large body of research finds that behavior does respond to framing manipulations such as this. In particular, multiple lab and field studies find that workers exert higher effort under penalty contracts than equiv- alent bonus contracts.¹ The leading explanation for these findings is loss aversion around a reference point (Kahneman and Tversky, 1979), where the reference point is influenced by framing. Losses loom larger than gains, so people work harder to avoid a penalty than to achieve a bonus, just as they demand more to give up an endowment than they will pay to gain it.

But how willing are workers to accept penalty contracts? This is an im- portant question for three reasons. First, the existing literature implicitly or explicitly recommends the use of penalty incentives, but firms need to understand the participation constraint to be able to effectively use them.

Second, while we know a lot about how people make choices given a ref- erence point, such as between a safe or risky loss, we know little about preferences between reference points, such as between a low reference point bonus contract or high reference point penalty contract. Third, it speaks to an old empirical puzzle: why are penalty contracts rare?²

The standard reference-dependent framework used to explain effort re- sponses to penalty contracts also predicts that workers will dislike them. A high reference point is costly because it makes one more disappointed with any outcome, hence why one should not “count chickens before they are hatched.”³ Under a bonus contract the worker feels great when successful and fine when unsuccessful, under a penalty contract she feels fine when successful and terrible when unsuccessful. Firms must therefore pay more to recruit workers under penalty contracts, providing a simple, plausible

1Church et al. (2008), Armantier and Boly (2012) and Hochman et al. (2014) in the lab, Hossain and List (2012) and Fryer et al. (2012) in the field.

2Baker et al. (1988); Lazear (1991). Baker et al. write “Every economist understands that [the bonus and penalty contracts are equivalent]. What economists don’t under- stand is why compensation plans almost always are of the former type instead of the latter.”

3This property is central to, for example, Benartzi and Thaler’s (1995) explanation of the equity premium puzzle.

explanation for why firms seem unwilling to use them in practice: bonus contracts can elicit the same effort at lower cost.

I ran three online field experiments with 1,848 workers in total, designed to test this prediction. Simply posting differently framed job offers faces an inference problem: workers who receive a penalty contract might form different beliefs about the task or principal than workers who receive a bonus contract. I therefore use a two-stage design, where in stage 1 all workers perform a data entry task under the same contract, giving them experience of the task and of interaction with me. In stage 2, one week later, I re-recruit for the same task from the same pool of workers, now offering them either a randomly assigned bonus or penalty contract. I then study who accepts, and how they perform.

I replicate the finding that penalties elicit higher effort. Accuracy in the main experimental task, data entry, was 6 percent or 0.2 s.d. higher under the penalty contract. This estimate is highly robust to controlling for selection partly because, surprisingly, I see no evidence of selection effects:

bonus and penalty contract acceptors are essentially identical in terms of observables. Most surprisingly, I find a strong preference in favor of penalty contracts. Workers offered penalty framed incentives were 25 percent more likely to accept than those offered an equivalent bonus contract.

In other ways, my workers’ behavior is conventional. Switching from flat pay (stage 1) to performance pay (stage 2) improved performance by around 22 percent, and workers who performed better on stage 1 were more likely to accept the performance paid job offer in stage 2. These suggest that the surprising finding of a preference for penalty contracts is not anomalous, and the remainder of the paper explores possible mechanisms.

Combining evidence from all three experiments I rule out a number of mechanisms that can be divided into belief- and preference-based explana- tions. First, I alter the phrasing of the job offer to check whether workers understood the terms of the contract, and once again find a preference for penalties of similar magnitude. I then consider inference. Did penalty framing lead workers to believe that the probability of receiving the bonus was higher? I rule out a beliefs-based explanation, first by eliciting work- ers’ beliefs, and second by replicating the preference for penalties in a coin toss guessing task that eliminates (or at least strongly reduces) the scope

for wrong beliefs. To illustrate, a back-of-the-envelope calculation finds that to explain the result, beliefs of coin-toss subjects about their guessing performance would need to be wrong by five standard deviations.

Turning to preferences, I next consider whether penalties are valued as a commitment device, since the desire to avoid a loss increases effort and thus earnings. This is a plausible, appealing explanation, but acceptance rates do not track true realized earnings, and the coin toss experiment (where performance is independent of effort) firmly rules out this mechanism. I ar- gue instead that the workers formed a more positive subjective view of the penalty contract induced by its salient high base pay. Supporting this claim I present survey evidence suggesting that workers subjectively perceived the penalty contract as better paid despite understanding the objective terms of the contract. Related existing work finds that people underweight non- salient sales taxes (Chetty et al., 2009) and eBay shipping costs (Hossain and Morgan, 2006), and overweight salient features of a good’s price dis- tribution (Mazar et al., 2013) or transparently irrelevant anchors (Ariely et al., 2003). Paraphrasing Mazar et al. (2013), workers appear to “focus on something other than the total benefit that the [contract] confers to them.”

The results suggest that penalty contracts are a win-win proposition for firms, offering improved performance at lower cost. Why, then, are they rare? It could be that over time, workers’ reference points adjust, eroding the performance gains and leading some–those who were induced to accept by the framing–to quit, which is costly. The third experiment explored this, but found that the penalty contract was again more popular among workers invited back for an additional stage. Evidently more research is needed, and I conclude the paper with possible directions.

The rest of the paper is as follows. Section 1 outlines the theoreti- cal predictions of the effects of penalty framing in a loss aversion model.

Sections 2 and 3 present the experimental design and results. Section 4 analyzes mechanisms. Section 5 discusses the results, external validity and related literature, then section 6 concludes. Three Web Appendices contain additional theory, results and experiment materials.

1 A simple model

Consider an agent (A) deciding whether to accept a contract to perform a task, the success of which depends upon her effort. A chooses an effort level e ∈ [0, 1] which equals the probability that the task is successful. If unsuccessful, the contract pays an amount w, if successful it pays w + b.

In addition to these pecuniary incentives, A’s utility is reference-dependent and loss-averse, and her reference point is influenced by how the contract is framed, represented by F ∈ [0, 1]. F = 0 corresponds to a pure bonus frame where w is the “base pay” and b is a bonus for success. F = 1 is a pure penalty frame where w + b is the base pay and −b is the penalty for failure. F ∈ (0, 1) is a mixed frame with base pay w + F b, bonus (1 − F )b for success and penalty F b for failure.

Similar to Kőszegi and Rabin (2006, 2007) (henceforth, KR), I assume that A’s utility function is a sum of a standard component, equal to ex- pected income less a convex cost of effort c(e), and a gain-loss component that evaluates monetary payoffs against a reference point. Specifically, for monetary outcome x and reference point r, gain-loss utility is equal to µ(x − r) for x ≥ r (a gain) and −λµ(x − r) for x < r (a loss), where µ(0) = 0, µ⁰ > 0, µ⁰⁰ ≤ 0 and λ ≥ 0. λ is A’s coefficient of loss aversion.

If λ > 1 she is loss averse: the disutility of a loss exceeds the utility of an equal-sized gain. If µ⁰⁰ < 0 A exhibits diminishing sensitivity in the gain and loss domains. I follow KR in assuming no probability weighting.

For now, I assume that A’s reference point r is non-stochastic and equal to the “base pay” specified in the contract: r = w + F b.⁴ With probability e A earns w + b, a gain of (1 − F )b. With probability 1 − e she earns w and thus experiences a loss equal to F b.

A’s utility function is:

U (e, w, b, F ) = w + eb − c(e) + eµ((1 − F )b) − (1 − e)λµ(F b). (1) A’s optimal effort e^∗ solves the first order condition:⁵

b + µ((1 − F )b) + λµ(F b) − c⁰(e^∗(b, F )) = 0. (2)

4In Web Appendix A I allow the reference point to depend on A’s expected effort (via a simple extension of KR) and discuss reference dependence in effort and in the outside option.

5For simplicity, I focus on b, F such that the solution e^∗ is smaller than one.

A accepts a contract (w, b, F ) if her participation constraint is satisfied:

U^∗(w, b, F ) − ¯u ≥ 0 (3)

Where U^∗(w, b, F ) = U (e^∗(b, F ), w, b, F ) and ¯u is the utility of her outside option. This simple model yields three key testable predictions:

Prediction 1. If A is loss averse (λ > 1) her effort is strictly higher under a pure penalty contract than a pure bonus contract: e^∗(b, 1) > e^∗(b, 0).⁶ Prediction 2. Penalties have a larger effect on effort for more loss-averse agents: _{∂F ∂λ}^∂2e∗ > 0.⁷

Prediction 3. Penalty framing reduces A’s willingness to accept the con- tract: ^{∂U∗(w,b,F )}_∂F < 0.⁸

Predictions 1 and 2 match the findings in the existing literature on the effects of penalty framed incentives, and are also explored in the empirical analysis in this paper. The main focus of the paper is on Prediction 3, and the experiment is designed to test whether workers indeed prefer bonus contracts. Furthermore, agents may be heterogenous, for example differing in loss aversion or the cost of effort. The empirical part of the paper studies whether types differentially select into penalty contracts.

Interestingly, it turns out that the negative effect of penalty framing on agents’ willingness to accept the contract is sufficiently strong that it is more costly to elicit a given effort level using penalties than using bonuses, despite the positive incentive effects of penalties. Thus if Prediction 3 is correct, it may help to explain why firms are reluctant to use penalty contracts. This point is illustrated by the following Proposition (the proof is given in Web Appendix A):

Proposition 1. Consider a contract (w, b, F ), where F > 0, that elicits ef-

6If µ(.) is linear, effort is strictly increasing in F . With diminishing sensitivity it may not be (see Armantier and Boly (2012) for evidence). It is everywhere increasing if and only if λ > ^µ_µ⁰0⁽⁰⁾(b) ≥ 1. Proof: ^∂e∗_∂F^{(b,F )} = ^{λµ0(F b)−µ}_c00(e^∗(b,F ))^{0((1−F )b)}. The numerator is decreasing in F , so if it is positive for F = 1, it is for all F . Intuitively, diminishing sensitivity implies that outcomes far from the reference point are weighted less strongly than out- comes close to the reference point, so the incentives may be sharper with intermediate than extreme reference points.

7Strictly, penalties have a more positive effect, since ^∂e∗_∂F^{(b,F )} can be negative.

8This holds for all λ ≥ 0. It relies only on reference dependence, not loss aversion.

Also note that incorporating probability weighting would not change the result, as an increase in F decreases gain-loss utility however its components are weighted.

fort level e and gives A utility u. Then, there exists an alternative contract, (w⁰, b⁰, F⁰), where F⁰ < F , that elicits e, gives A at least u and where A’s expected compensation is strictly lower, i.e. w⁰ + eb⁰ < w + eb. Therefore, the lowest-cost contract that elicits e is a pure bonus contract with F = 0.

2 Experimental design

I ran three experiments with online workers on Amazon Mechanical Turk (MTurk, for short), each with the same basic design. Each consisted of a first stage where workers were recruited on MTurk for a real-effort task and survey, and paid a flat wage. Then, a week later, workers from the first stage were sent a surprise job offer to perform the task again (stage 2), this time under framed performance pay.

2.1 Platform: Amazon Mechanical Turk (MTurk)

MTurk is an online labor market for “micro outsourcing”. For example, a

“requester” that needs data entered, audio recordings transcribed, images categorized, proofreading, etc. can post a job on MTurk, and recruit “work- ers” to carry it out. Pay is set by the requester. MTurk enables testing for selection effects in a natural environment, where the worker’s outside option (should she reject the job offer) is the other tasks she can perform on MTurk. In the lab one must typically create an outside option either by giving the worker the option to choose another task (as in e.g. Dohmen and Falk (2011)) or a sum of money. Most work on MTurk is performed for low wages (my workers reported a mean reservation wage of $4.97 per hour, and mean typical hourly earnings of $4.70), enabling me to recruit a large sample to increase power. The average worker in my sample works for 17 hours per week on MTurk and has been a worker for 12 months.

MTurk is increasingly commonly used for research by economists. To cite a couple of examples, Bordalo et al. (2012) test their theory of salience using MTurk surveys. Barankay (2011) uses MTurk to study the effect on willingness to undertake more work of telling workers about their rank in an initial task. Horton et al. (2011) and Amir et al. (2012) replicate some classic experimental results with MTurk workers.

2.2 Real-effort tasks

The task in experiments 1 and 2 was transcribing 50 text strings, increas- ing in length from 10 to 55 characters. The strings were generated using random combinations of letters, numbers and punctuation and distorted to give the appearance of having been scanned or photocopied. Workers typically took 35-45 minutes to complete the task. The task was chosen to be implementable online and be sufficiently difficult to avoid ceiling effects, without putting the workers under time pressure (Amazon encourages re- questers to give workers time to complete tasks at their own pace).⁹ In each stage workers were randomly assigned to one of 10 possible sets of strings. An example screen is reproduced in Web Appendix Figure C2.

The task in experiment 3 was guessing 50 coin tosses, chosen to make it clear that performance did not depend on effort while mirroring the structure of experiments 1 and 2. It took around 10 minutes.

2.3 Experimental design

I use a two-stage design similar to Dohmen and Falk (2011). The design is summarized in a timeline in Web Appendix Figure C1. In the first stage, workers were recruited on MTurk for a “typing task and survey” or

“guessing task and survey,” for which they were paid a fixed amount, $3 for the typing task and $1 for the shorter guessing task. Flat pay was used to avoid exposing workers to more than one form of incentive pay during the experiment. The next day, they were sent an email informing them of their accuracy on the task (text is given in Appendix C.3).

Then, one week later workers were sent a surprise invitation to perform the same task again, only this time for randomized performance pay. Work- ers were allowed four days to complete the task, and were free to ignore the offer if not interested. Each contract had three components: a fixed pay component that did not depend on performance, an additional variable pay component that was paid if the accuracy check was passed, and a frame

9The task resembles CAPTCHA (Completely Automated Public Turing test to tell Computers and Humans Apart) puzzles, used in web forms to prevent bots and spam- mers from accessing sites. This has led to some spammers recruiting MTurk workers to solve CAPTCHAs for them. See e.g. New York Times blog, March 13, 2008: http://

bits.blogs.nytimes.com/2008/03/13/breaking-google-captchas-for-3-a-day/.

that was either “bonus” or “penalty”.

Table 1: Treatments

Experiment Group N Fixed pay Variable pay Frame

Experiment 1 0 192 $0.50 $1.50 Bonus

Data entry 1 188 $0.50 $1.50 Penalty

2 193 $0.50 $3 Bonus

3 191 $0.50 $3 Penalty

4 193 $2 $1.50 Bonus

5 189 $2 $1.50 Penalty

Experiment 2 6 153 $0.50 $3 Bonus

Data entry 7 151 $0.50 $3 Penalty

Experiment 3 8 202 $0.30 $1 Bonus

Coin toss 9 196 $0.30 $1 Penalty

Performance pay was calculated as follows. Workers were told that after completion of the task I would select, using a random number generator, one of the 50 strings or coin tosses that they had been assigned to type or guess. They would receive the bonus (avoid the penalty) conditional on that item being entered correctly. This structure means that workers’

probability of receiving the bonus was equal to their accuracy rate, the statistic reported to them in stage 1. Pay rates were chosen to be compa- rable to typical rates on MTurk (otherwise it would be difficult to generate selection effects) but relatively high powered to maximize statistical power, and are detailed in Table 1. Only the “pay” portion of the invitation dif- fered between treatments. The key phrasing is given in Table 2 and full email text in Web Appendix C.4. I deliberately avoided emotive words like

“bonus” and “penalty”.

Additionally, in experiment 2, workers were invited to a paid follow-up survey eight days after stage 2. In experiment 3, workers were invited to a stage 3 one week after stage 2, under the same terms as stage 2.

The two stage design serves three main purposes. First, it ensures that workers know the task and their ability. This is important because workers might make inference about the nature of the task from the incentive con- tract they are offered. For example, a contract that penalizes failure might be seen as easy (failure is unlikely) while a contract that rewards success

Table 2: Framing text

Experiment 1 Bonus Experiment 1 Penalty

. . . The basic pay for the task is $0.50.

We will then randomly select one of the 50 items for checking. If you entered it correctly, the pay will be increased by $3.00. . .

. . . The basic pay for the task is $3.50.

We will then randomly select one of the 50 items for checking. If you entered it incorrectly, the pay will be reduced by $3.00. . .

Experiments 2 & 3 Bonus Experiments 2 & 3 Penalty . . . The pay for this task depends

on your typing accuracy. We will randomly select one item for checking, and if it was entered correctly, the pay will be increased above the base pay. The base pay is $0.50 which will be increased by $3 if the checked item is correct. . .

. . . The pay for this task depends on your typing accuracy. We will randomly select one item for checking, and if it was entered incorrectly, the pay will be reduced below the base pay. The base pay is $3.50 which will be reduced by $3 if the checked item is incorrect. . .

Note: Experiment 3 referred to “guesses” and “coin tosses” instead of accuracy and items.

is seen as hard.¹⁰ Task experience should mitigate this effect. Second, it ensures that workers have interacted with the principal (me) before. This is important because penalty contract offers might be perceived as more or less trustworthy.¹¹ Third, it enables me to measure types prior to treat- ment, and ensure that the treatment randomization is balanced across type, both of which enable me to test for selection by comparing the distribution of types that accept each contract.¹²

2.4 Data

This section describes the data collected in the main survey and effort tasks.

Summary statistics are presented in Web Appendix Tables B1 and B2.

10Bénabou and Tirole (2003) analyze an asymmetric information context whereby if the principal offers higher pay for a task it signals that the task is undesirable. This result relies on the pay acting as a costly signal, while altering the frame is costless.

11Workers agreed to an informed consent form that states their work is part of a research project from the LSE (note that they were not told that it was an incentives study), gives my name and contact details. They were paid promptly after completing the first stage, and received a personalized performance report after stage 1.

12I did not use the approach of Karlan and Zinman (2009) because it would expose workers to both frames and therefore make transparent the equivalence of the two.

The measure of loss aversion I use is an unincentivized variant of that of Abeler et al. (2011). Workers indicated whether they would play each of 12 lotteries of the form “50% chance of winning $10, 50% chance of losing $X,”

where X varies from $0 to $11. I proxy for loss aversion with the number of rejected lotteries. 7 percent of workers made inconsistent choices, accepting a lottery that is dominated by one they rejected. A screenshot of the lottery questions is given in Web Appendix C.6. Two other key variables that I attempt to measure are workers’ reservation wages and their perceptions of what constitutes a “fair” wage. To elicit reservation wages I ask workers what is the minimum hourly wage at which they are willing to work on MTurk.¹³ I ask workers the minimum fair wage that requesters “should” pay on MTurk, and use this measure to proxy for fairness concerns. Reservation wages are smaller than or equal to fair wages for 92 percent of workers.

The main performance measure is “Accuracy Task X”, the fraction of strings entered correctly or tosses guessed correctly in stage X. In the typing task I also compute “Scaled Distance Task X”, which can be thought of as the error rate per character typed.¹⁴ Third, I try to measure how much time workers spent on their responses. There are large outliers since I cannot observe how long workers were actually working on a given page of responses, only how long the page was open for, so I take the time the worker spent on the median page, multiplied by 10 to estimate the total time. Finally, at the beginning of stage 2 workers were asked to estimate the mean accuracy rate from stage 1, a variable I label “Predicted Accuracy”.

In total 1,465 workers were recruited for experiments 1 and 2, of which 693 returned for stage 2. 15 are dropped from all of the analysis, six because I have strong reasons to suspect that the same person used two MTurk accounts to participate twice¹⁵ and nine because they scored zero

13Fehr and Gächter (2002) find in a buyer-seller experiment that penalty-framed per- formance incentives led to more shirking among sellers than equivalent bonus-framed offers, and argue that this is because the penalty contracts are perceived as less fair.

14For each text string I compute the Levenshtein distance (the minimum number of single character insertions, deletions, or swaps needed to convert string A into string B) between the worker’s response and the correct answer, and divide by the length of the correct answer, then average over all answers. This then roughly corresponds to the probability of error per character. In the regressions I use the natural log of this measure since it is is heavily skewed.

15I received two pairs of near identical emails, each pair within a couple of minutes, strongly suggesting that one person was operating two accounts simultaneously. The

percent accuracy in the stage 1 typing task (of the six of these who re- turned for stage 2, five scored zero percent again). Results are robust to including these 9. 398 workers were recruited for experiment 3, of which 267 completed stage 2 and 245 completed stage 3.

2.5 Randomization

I stratified the randomization on the key variables on which I anticipated selection: stage 1 performance, rejected lotteries and reservation wage.

In case some workers might know one another (for example, a couple who both work on MTurk), the treatments were randomized and standard errors clustered at the zipcode-experiment level.¹⁶ In robustness checks that drop workers who share a zipcode this is equivalent to using robust standard errors, since then each cluster is of size one.

As a graphical check of balance, Web Appendix B.2 plots the CDFs by treatment and the associated Mann-Whitney U-test p-values for key observables, confirming good balance on these variables. Web Appendix Table B3 presents the results of the statistical balance tests. There is good mean balance on all characteristics with the exception of the minimum fair wage, where the difference comes from differences between experiments 1 and 2, and the number of MTurk HITs completed, where the difference is driven by a small number of outliers.

3 Results

This section discusses the effect of the penalty frame on workers’ willingness to accept the contract, on the types of workers who select into the contract, and on performance on the job. For this analysis I pool the data from experiments 1 and 2 to increase power. I then discuss the follow-up survey, the coin toss experiment 3, and effect persistence.

third pair was revealed by the fact they typed identical nonsense in the second stage typing task.

16In experiment 1, 179 individuals, in experiment 2, 8 and in experiment 3, 30 indi- viduals reported the same zipcode as another worker.

3.1 Acceptance

Figure 1 graphs the rates of acceptance of the stage 2 job offer by treatment.

The striking pattern is that penalty framed contracts were much more likely to be accepted than equivalent bonus framed contracts. This result is particularly notable because it directly contradicts model Prediction 3.

In addition, acceptance is substantially higher under higher fixed pay, while the relationship between variable pay and acceptance appears weak at best.

The basic regression specification is a linear probability model with dependent variable Accept_i ∈ {0, 1}, individuals indexed by i:

Accept_i = β₀+β₁∗P enalty_i+β₂∗HighF ixed_i+β₃∗HighV ariable_i+X_i⁰β₄+_i P enalty is a dummy equal to 1 if the contract is penalty framed and zero if bonus framed. HighF ixed is a dummy indicating fixed pay equal to $2 (alternative: $0.50). HighV ariable is a dummy indicating variable pay of

$3 (alternative: $1.50). Since I do not have a group with both high fixed and variable pay, the comparison group in each case is the group with low fixed and low variable pay. X_i is a vector of variables measured in stage 1. In particular, I include accuracy and time spent on the stage 1 effort task, to jointly proxy for ability and intrinsic motivation, and dummies for the set of items assigned to be typed by that worker (10 possible sets per stage). Note that the main specifications estimate the average effect of the penalty frame across all incentive pairs to increase power.

Table 3 presents the main results. I find that switching from bonus to penalty framing increases the acceptance rate by approximately 10 per- centage points. This implies a 25 percent higher acceptance rate under the penalty frame than the bonus frame (the acceptance rate under the bonus frame was 42 percent), a large effect for a simple framing manipulation.

High fixed pay increases acceptance by around 15-16 percentage points, or around 36 percent (the acceptance rate in the comparison group, low fixed and variable pay, was 42 percent). Surprisingly, the effect of high variable pay is positive but much smaller at 3 percentage points greater take-up, and not statistically significant. The results are robust to dropping work- ers who made inconsistent choices in stage 1, outliers on time on the first task, reservation or fair wages, and those from zipcodes with more than one respondent. Near-identical average marginal effects are obtained using

logistic instead of linear regression.

Column (5) of Table 3 interacts the penalty treatment with the high fixed and high variable pay treatments, to estimate the differential effect of penalties under these regimes. The point estimates suggest that the effect of the penalty frame on acceptance was smaller for high fixed pay and larger for high variable pay, however neither estimate is statistically significant.

In addition, the point estimate on “high variable pay” is essentially zero for workers under the bonus frame, implying that the potential for a $3 bonus as opposed to a $1.50 bonus did not make the job offer significantly more attractive. It is important to note that the smaller difference in acceptance rates between bonus and penalty under high fixed pay does not mean that the framing effect necessarily shrinks with the level of pay. It is entirely consistent with the fact that as the level of compensation increases we move into the right tail of the reservation wage distribution, which leads acceptance rates to converge. For example, if the fixed pay was $1000 one would expect all workers to accept.

Additionally, penalty contract recipients were 9 percentage points more likely to click on the link in the job offer email (p=0.001, 52 percent clicked under the bonus treatment). Clicking took them to a page that replicated the text of the offer email. Workers are coded as accepting the offer if they then proceeded to the task. They were 6 percentage points more likely to proceed conditional on clicking (p=0.014, 80 percent proceeded under the bonus treatment).

Web Appendix Figure B7 compares the distributions of key observables between those who did and did not accept the job offer. Workers who performed better in stage 1 were significantly more likely to accept the stage 2 job offer, as is clear from Web Appendix Table B1. This is consistent with the common finding that performance pay differentially selects more able or motivated workers, which I discuss further in Web Appendix B.11.

Workers with a higher reservation wage were significantly less likely to accept the offer. The coefficient on “minimum fair wage” is small and not statistically significant, suggesting that fairness concerns (as measured by this variable) were not of primary importance for willingness to accept the contract. The number of rejected lotteries is not predictive of acceptance, whether or not I drop workers who made inconsistent choices in the lottery

questions. This is surprising as the stage 2 contract is risky, so one would expect more risk/loss averse workers to be less willing to accept.

3.2 Selection

Now I turn to the effect of the penalty frame on the types of workers that select into the contract. Figure 2 plots CDFs of stage 1 task perfor- mance, time spent on stage 1 task, rejected lotteries, reservation wage and fair wage, comparing those who accepted the bonus frame with those who accepted the penalty frame. Surprisingly, the distributions are barely dis- tinguishable for all variables except for reservation wages, consistent with no selection on these variables. I do observe suggestive evidence that the penalty contract attracted workers with higher reservation wages on aver- age, as would be expected from the higher acceptance rate. However the correlation between reservation wages and other characteristics is small.

Table 5 tests for selection effects of penalty framing by regressing the key observables on contract terms, conditional on acceptance. The coefficient on “penalty frame” is interpreted as the difference in the conditional mean of the outcome in question between penalty and bonus workers.¹⁷ The results confirm what we saw in the graphs: the differences between bonus and penalty workers are small and not statistically significant. Focusing on task 1 accuracy (the strongest predictor of task 2 performance), the point estimate implies 0.2 percentage points higher task 1 accuracy among penalty contract acceptors. Multiplied by the estimated coefficient on task 1 accuracy in the main performance regressions (0.72, see Table 4 column (2)), this implies less than 0.2 percentage points higher performance under the penalty contract explained by selection on task 1 performance, less than 5 percent of the estimated treatment effect. I discuss selection effects again when analyzing coefficient stability in the next section.¹⁸

17In Web Appendix table B.3 I regress acceptance on characteristics interacted with a penalty dummy, estimating to what extent that characteristic differentially predicts acceptance under the penalty contract. The results are very similar. A joint test fails to reject the null that all interaction coefficients are equal to zero (p=0.90).

18As for the other covariates of note, penalty acceptors were 6 percentage points more likely to be male than bonus acceptors (p=0.13), and were 5 percentage points more likely to “mainly work on MTurk to earn money” (p<0.01, note that 93 percent of workers gave this response). They had also previously completed around 10-30 percent more HITS in the past, depending on the specification (this variable is heavily skewed with

It is surprising that there seems to be no selection effect of penalty framing. One possibility is that selection is hard to detect in this context.

It is reassuring therefore that we do observe a standard selection effect as discussed above: workers who scored higher accuracy on stage 1 were more likely to accept the stage 2 job offer, just not differently between frames.

3.3 Performance

The primary focus of the paper is on the effect of the penalty contract on job offer acceptance and selection, and the ideal experimental design for estimating incentive effects eliminates selection by removing the option of rejecting the contract, as in previous studies. It is nevertheless instructive to compare performance between framing treatments. First, I can check for higher performance under penalty contracts to replicate the existing litera- ture. Second, it allows a further check for selection into penalty contracts.

Let Y_i be a measure of effort or performance. The basic regression is:

Y_i = δ₀+ δ₁∗ P enalty_i+ δ₂∗ HighF ixed_i+ δ₃∗ HighV ariable_i+ X_i⁰δ₄+ _i. In general the estimates of δ1, δ2 and δ3 will be biased by selection: if the workers that accept one contract are different from those that accept another, then performance differences may simply reflect different types rather than different effort responses to incentives. However as already documented, I do not observe differential selection on observables between frames, which would bias the estimate of the key coefficient of interest, δ₁. Moreover, since I have stage 1 measures of type, I can control for selection on observables by including these.

Figure 3 presents the mean performance on the stage 2 task by treat- ment group. I find that at each incentive level, performance is higher under the penalty than under the bonus frame (although not always statistically significantly so), consistent with the existing experimental studies. Pooling the framing treatment, in Figure 4 I plot CDFs of the accuracy measure, the log distance measure (recall that this is interpreted as the log of the per-character error rate) and time spent, and find that performance and

large outliers and the difference is not significant). None of these results is consequential for performance, as illustrated by the lack of selection on performance measures and the evidence presented below.

effort is higher under the penalty frame right across the distribution.¹⁹ Table 4 presents the main results. Accuracy under the penalty frame was 3.6 percentage points (around 0.18 standard deviations or 6 percent of the mean accuracy of 0.59) higher than under the bonus frame, statistically significant at 5 percent without and 1 percent with controls. The coeffi- cient estimate is robust to dropping workers who made inconsistent lottery choices, workers from zipcodes with multiple respondents, and outliers on the reservation and fair wage questions. Crucially, the point estimate does not change with the inclusion or exclusion of controls, consistent with the contract frame not inducing outcome-relevant selection on observables. For selection to explain the results, there would have to be a substantial unob- served driver of performance that is differentially selected under the penalty frame and orthogonal to the set of controls included in the regressions.²⁰

High fixed pay increased accuracy by around 2-4 percentage points, significant at 5 percent when including controls. The point estimate dou- bles when controls are included, indicating adverse selection induced by the higher fixed pay and giving further comfort that the lack of observed selection between bonus and penalty reflects a true lack of selection in the data.²¹ High variable pay increases accuracy by around 1.4-2.5 percentage points, although this is never significant at conventional levels. Column (5) interacts the penalty dummy with high fixed and high variable pay to estimate the differential effect of penalties under each financial incen-

19Web Appendix table B5 presents corresponding regressions.

20Oster (2013) points out that this stability heuristic is not valid unless paired with information on R-squared movements, and provides a formula that bounds the estimate of the treatment effect using the estimates with and without controls plus assumptions on a) what would be the R-squared if all relevant unobservables were included as controls (Rmax) and b) the strength of the relationship between the treatment and unobservables relative to the strength of the relationship between the treatment and observables (δ).

Even under extreme assumptions (Rmax = 1, δ = 4), i.e. including unobservables would explain all the variation in performance and the unobservables are four times as

“important” as the observables (Oster recommends setting δ equal to 1), the coefficient on Penalty changes just barely, to 3.8 percentage points.

21Workers were not forced to complete every item of the data entry task (blank re- sponses were counted as incorrect). 93 percent of bonus workers and 95 percent of penalty workers completed the task (difference p=0.2). However, high fixed pay workers were 7 percentage points more likely to complete than low fixed pay (p=0.01). Drop- ping those who did not complete the task, the point estimate of the penalty effect falls slightly to 3.2 percentage points (p<0.01). The coefficient on high fixed pay drops to 1.3 percentage points and loses significance. Most of the effect of high fixed pay is explained by the higher completion rate.

tive. High fixed seems to have the same effect under both frames, while high variable pay has a smaller effect under the penalty frame. However, neither estimate is significant.

As for the other key variables, performance in the first stage very strongly predicts performance in the second stage, while the coefficient on time spent in the previous task is negative, small in magnitude and not significant. A higher reservation wage is associated with poorer per- formance, while fair wage has no effect on performance. In this stage the number of rejected lotteries is negatively associated with performance and significant. A one standard deviation increase in the number of rejected lotteries is associated with around 1 percentage point worse performance.

Table 6 reports estimates of heterogeneous effects of the penalty treat- ment by the main variables. There is little evidence of strong heteroge- neous effects. Focusing on rejected lotteries, the overall relation is nega- tive, which is possible in the extended model outlined in Appendix A.1, but is more strongly negative (not significant) under the penalty frame, contrary to Model Prediction 2. I lack power to dig into this relationship in depth. I do however perform one simple exercise. Appendix Figure B2 non-parametrically plots accuracy against rejected lotteries separately under bonus and penalty frame, after partialling out the other variables, dropping workers with inconsistent choices and those who rejected or ac- cepted all lotteries. Over much of the range of rejected lotteries the slopes are approximately the same, but there is a strongly negative relationship between performance and rejected lotteries for the workers who rejected most of the lotteries.

To conclude this section, I note that performance improved between the flat paid stage 1 and performance paid stage 2. This improvement reflects three things: selection, effort response to performance pay, and learning (workers get better at the task). In Web Appendix B.11 I discuss how to separate these, and estimate that the combined effort and learning effects improved performance by around 10-11 percentage points or 22-23 percent.

Workers’ effort and selection response to performance pay was consistent with typical findings, suggesting that the surprising response to penalty framing should be taken seriously.

3.4 Follow-up survey

All workers from experiment 2 were invited to complete a short survey for a fixed payment of $2. 83 percent did (128 of 153 bonus workers and 124 of 151 penalty workers).²² Note that questions were unincentivized and conducted after the completion and payment of stage 2.

Workers were first reminded of the job offer they received in stage 2, then asked a series of questions about it. Results are presented in table 7. Workers were asked to indicate agreement on a 1-7 scale to whether their job offer or task was fun, easy, well paid, fair, was a good motivator, earning $3.50 was achievable,²³ understandable, and whether the principal could be trusted. Results are presented in Panel A. They were then asked to what extent they agreed that the offer was attractive because of good pay, because they would be elated to receive $3.50, and because it encour- aged effort, and to what extent it was unattractive because it was risky, because they would be disappointed to receive $0.50, and because it was difficult. Third, they were asked to guess the acceptance rates of workers who received the same job offer as they did, and the fraction who received the maximum pay of $3.50.

For most questions I find no significant differences between frames.

However the penalty offer was rated significantly higher for good pay and more attractive due to good pay. Estimated acceptance rates and success rates were not significantly different between bonus and penalty frames, in fact penalty contract recipients thought workers were 1.4 percentage points less likely to receive the bonus (see Web Appendix Figure B4 for the dis- tributions of responses to this question). Penalty workers also responded more negatively on the achievability of earning the bonus.²⁴

22Workers who accepted in stage 2 were more likely to complete the survey (96 percent vs 73 percent, p-value < 0.001), probably reflecting that some non-participation in stage 2 is driven by workers who did not see my emails.

23“If a worker worked hard on the task, he or she can be confident that they would answer the checked item correctly.”

24Workers were also asked how willing they would be to accept their contract again and for an amount of money that would make them indifferent between the contract and the money (WTA). I find no significant differences between bonus and penalty workers (WTA values were very noisy with large outliers). They were also presented with the alternative contract (the one they did not receive) and asked to rate it on various scales. I find no significant differences in ratings between bonus and penalty recipients, i.e. when asked to consider the alternative contract, penalty recipients do not

3.5 Experiment 3 and effect persistence

As previously described, experiment 3 followed the same design as experi- ments 1 and 2, but the task was changed to guessing 50 coin tosses rather than typing 50 strings. This design therefore ensured that performance did not depend on effort.²⁵ It also added a stage 3 to test for effect persistence.

In stage 2 the penalty contract was significantly more likely to be ac- cepted than the bonus contract. 62 percent of bonus workers and 72 percent of penalty workers completed the task (difference p=0.043).²⁶ These results are presented in Table 8, Panel A and Web Appendix Figure B5, Panel A.

The effect size, 10 percentage points, is very close to the average effect in experiments 1 and 2, but hard to compare as the bonus acceptance rate is higher in experiment 3. To compare them, I calculate the corresponding log odds ratios (logit coefficients), obtaining 0.437 when pooling experiments 1 and 2, and an almost identical 0.436 for experiment 3.

None of the main observables predicts acceptance. Importantly, task 1 accuracy does not²⁷ (unlike in experiments 1 and 2) and workers did not spend more time on the task under the penalty contract²⁸ implying they understood that effort and skill cannot influence performance on this task. I again find no evidence of selection on observables into the penalty contract, see Web Appendix Table B8.

Stage 3 tested whether the popularity of the penalty contract wore off.

Consider the workers who accepted in stage 2. If the effect wore off between stages 2 and 3 we would expect a lower acceptance rate under the penalty frame in stage 3. The findings are presented in Table 8, Panel B and

rate the bonus contract more or less favorably than bonus recipients rated the penalty contract. The strong modal response was that the other contract was “the same” on each scale. Interestingly, both penalty and bonus workers rated the other contract on average slightly less attractive, fair, generous, trustworthy and achievable, and slightly more motivating, than their own. One could interpret this as an endowment-type effect.

These results are presented in Web Appendix Table B6.

25To ensure this, workers were told they must guess all 50 tosses to be paid. Recall that previously workers were considered acceptors if they partially completed the task.

26Under the previous definition of acceptance (including those who partially completed the task) the figures were 67 and 75 percent respectively, difference p=0.073.

27In experiments 1 and 2 a 1 s.d. improvement in stage 1 performance is associated with a 6 percentage point higher acceptance rate. In experiment 3 it is associated with a 0.7 percentage point lower acceptance rate.

28Mann-Whitney U p-values 0.35 and 0.80 for stages 2 and 3 respectively.

Web Appendix Figure B5, Panel B. Once again, the penalty contract was significantly more popular overall. It was more popular among those who did and those who did not accept the offer in stage 2. Among those who completed the task in stage 2, it was more popular both among those who were lucky (guessed correctly and received the bonus) and among those who were unlucky. Note that although the point estimates are not statistically significant in the latter four cases, the key question of interest was whether any would be significantly negative, and I see no evidence of this.

Finally, I also examine within-task persistence by looking at item-by- item performance in experiment 1 to see if the performance difference be- tween bonus and penalty frames disappears over the course of the task. It does not. Regressing a dummy for whether a given item was entered cor- rectly on the item number, penalty dummy and their interaction (plus con- trols) I find that the interaction term is a precisely estimated zero, whereas convergence would imply a negative interaction. See Web Appendix Figure B6 and Table B10. Hossain and List (2012) also present evidence that their framing treatment did not wear off over the course of several weeks.

4 Mechanisms

While loss aversion can explain the higher effort provision under the penalty contract, it cannot explain the higher acceptance rate. Loewenstein and Adler (1995), Van Boven et al. (2000) and Van Boven et al. (2003) find that people underestimate the endowment effect: for example, they predict a lower willingness to accept to give up a mug when asked to imagine being endowed with it than when actually endowed with it (see also Loewen- stein et al. (2003)). However, at best this predicts equal acceptance rates.

This section discusses possible explanations for the main result, drawing on evidence from all three experiments.

4.1 Misunderstanding of contracts

Experiment 1 was motivated by the prediction that the penalty contract would be unpopular. A first reaction to the surprising opposite finding was that perhaps workers were inattentive when reading the job offers. Since

these prominently mentioned the base pay, followed by details of potential bonuses or penalties, perhaps some workers stopped reading at the base pay and wrongly perceived the penalty contract as better paid. Or perhaps they misunderstood and perceived the base pay as a level below which they could not go. Or perhaps they misunderstood and thought that pay under the penalty contract would be reduced to $X instead of reduced by $X.

Experiment 2 rephrased the job offer to first emphasize that pay de- pended on performance via a bonus (or penalty), and that this would in- crease pay above or reduce it below the base pay respectively. Then the pay information was given in a single sentence, to force workers to absorb all of it. The penalty contract acceptance rate was 12 percentage points (32 percent) higher (p = 0.037). The penalty contract was also more popular (albeit not significantly so) in stage 3 of the coin toss experiment among both lucky and unlucky workers from stage 2. The unlucky workers are now certainly informed how low the pay can go, so if they had been previously mistaken they should now be less willing to accept, but I see no evidence of this. Inattention or misunderstanding do not explain the popularity of the penalty contract.²⁹

4.2 Inference

As previously discussed, a primary concern in the design of the experi- ment was that workers might have different beliefs about the likelihood of receiving the bonus depending on the contract they received.

At the start of stage 2 of experiments 1 and 2, workers who had ac- cepted the job offer estimated the average typing accuracy rate from stage 1.³⁰ Bonus workers estimated a mean stage 1 accuracy of 57.4 percent (s.d.

29Further evidence: 1) over the course of all three experiments, 362 people were at some point told they had been unsuccessful and paid the minimum. Not one of them contacted me to complain that they had been expecting more; 2) reading “reduced by”

as “reduced to” would predict lower take-up under the penalty contract in the high fixed pay treatment. This is because “$2... increased by $1.50” cannot be misunderstood as

“$2... increased to $1.50”, while “$3... reduced by $1.50” might be understood as “$3...

reduced to $1.50”. The penalty was again more popular in this treatment, albeit not significantly so.

30They were asked about stage 1 accuracy because this estimate should only depend upon their recollection of stage 1 and the contract they received, it is not confounded by different beliefs about effort provision in stage 2 induced by the framing treatment.

In principle the estimates could be biased by selection: the penalty frame could increase

19.4) and penalty workers estimated 57.9 percent (s.d. 18.1). The differ- ence was not significant (p=0.75). In the follow-up survey Penalty workers rated success under their contract as slightly less achievable (“If a worker worked hard on the task, he or she can be confident that they would answer the checked item correctly”), significant at the 10 percent level, and more difficult (not significant). When asked to estimate the fraction of workers who received the maximum pay in stage 2, again there was no significant difference between treatments (Table 7, Panel B, and Web Appendix Figure B4). The strongest evidence comes from Experiment 3, where the chance of success was transparently 50 percent, and yet the magnitude of the effect on acceptance was not diminished.

Three other forms of inference might plausibly explain the results.

Workers might believe 1) the bonus contract was less trustworthy (i.e. I would be less likely to honor the contract); 2) I might choose not to pay at all for “inadequate” work (rejection);³¹ or 3) the base pay is a “default”, to be deviated from only in exceptional circumstances.

Several pieces of evidence are inconsistent with these explanations.

First, the follow-up survey asked workers to rate the trustworthiness of their contract (“The requester can be trusted to do what they say they will do”). The penalty contract was rated as slightly (not significantly) less trustworthy (the overall mean response to this question was 5.9 out of 7).

Second, as noted above, survey respondents thought that effort was slightly less likely to be rewarded under the penalty contract. Third, the guessing task leaves no room for exceptional circumstances or inadequate work that might be rejected, the outcome is unambiguous and binary.

A simple back-of-the-envelope calculation can illustrate how wrong be- liefs would have to be to explain the results. Conservatively assuming that workers expected the typing task to take 30 minutes (in practice it usually

workers’ beliefs but cause a large mass of pessimistic workers to select in, leading to no change in the mean. This seems unlikely, first because the full distributions of estimates are not distinguishable between bonus and penalty workers (Web Appendix Figure B3, Panel A). Second, because the coefficient remains small, precise and not significant when including controls in the regression (Web Appendix Table B7). Experiment 3 workers were not asked due to a concern that this would be unnatural (as it would be equivalent to asking the probability of heads), and make them suspect the coin tosses were not fair.

31Rejection is a feature of MTurk. Requesters can reject inadequate work from work- ers, which may partly explain why workers worked hard on stage 1. At no point did I mention any possibility of rejection and no work was rejected.

took longer), the high fixed pay treatment increased hourly earnings by $3 and the acceptance rate by around 15 percentage points, or 5 percentage points per $1/hr. Using this figure I calculate the difference in (risk neu- tral) perceived success probabilities between bonus and penalty workers required to explain the acceptance rate effects observed in Figure 1. I then compare them to the true distribution of success probabilities: accuracy in task 2. In the baseline treatment, penalty workers would need to believe their chance of success was 72 percentage points or 3.6 s.d. higher; under the “high variable” treatment the values are 51 percentage points/2.5 s.d.;

under “high fixed”, 36 percentage points/1.3 s.d. and in experiment 2, 39 percentage points/2 s.d.. Applying the same method to the 10 minute coin toss experiment, penalty workers would need to believe their guessing ac- curacy was 32 percentage points higher than bonus workers did, which is 4.6 s.d. of the true accuracy distribution.³² In other words, penalty work- ers would have to expect to guess 2/3 of the tosses correctly, and bonus workers 1/3. Such large differences in beliefs are not plausible and would surely show up in the various attempts to elicit beliefs.

4.3 Rational expectations of higher earnings

Perhaps workers anticipate they will work harder and earn more under the penalty contract, which makes it attractive. This argument requires three parts. First, that the loss when unsuccessful is not anticipated as being sufficiently painful to discourage acceptance, for instance because the “planner” self who accepts the offer has not yet updated her reference point. Second, workers must be aware that they will work harder under the penalty frame. Third, effort provision under the bonus frame must be sub-optimally low, such that being motivated to work harder is attractive.

In other words, it requires a self-control problem, where penalties act as a commitment device. In Web Appendix A.4 I extend the model along these lines to illustrate the resulting preference for penalties.

It is plausible that workers might like penalty contracts as a commit- ment device. Kaur et al. (2013) find that workers select into a strictly dom-

32The number of correct guesses is binomial distributed, 50 draws, p = 0.5, so the standard deviation of the accuracy rate isp(0.5(1 − 0.5)/50) = 7.1 percentage points.

inated financial incentive scheme that acts as a commitment to higher effort provision. However, the evidence suggests that this is not what is driving the differences in acceptance rates between bonus and penalty frames.

First, consider the effect of the penalty contract on earnings in experi- ments 1 and 2. For workers who accepted the contract I compute expected earnings as w + e ∗ b, where e is their realized stage 2 accuracy. I also com- pute monetary surplus by subtracting the time spent on the task multiplied by their reservation wage.³³ I regress these measures on contract terms in Web Appendix table B9. The penalty frame increased expected pay by 6 cents on average, and increased take-up by 11 percentage points. High fixed pay increased expected pay by $1.53, and take-up by 16 percentage points. High variable pay increased expected pay by 91 cents but take-up by only 3 percentage points.³⁴ Rational expectations about earnings do not explain the patterns in acceptance rates.

However, the strongest evidence comes from the coin toss experiment.

Since earnings cannot depend on effort, if workers like penalty contracts for their motivational power, I should see no difference in acceptance rates in this experiment. Again I find a strong preference for penalties.

4.4 Salience

I propose a salience mechanism to explain the popularity of the penalty contract. Workers focus on the salient “base pay”, which is higher under the penalty contract than the bonus contract, and underweight the contingent pay component when deciding whether to accept. They act as if they have preferences over base pay and contingent pay, rather than over outcomes.³⁵ To paraphrase Mazar et al. (2013), workers “focus on something other than the total benefit that the [contract] confers to them.” Thus the base pay both shifts their reference point (driving differences in effort) and their

33This measure ignores the cost of effort and intrinsic motivation. Intrinsic motivation may explain why the mean surplus is negative - reservation wages were elicited for a general job on MTurk, perhaps workers preferred my task to the typical task.

34When considering surplus the picture is even more stark. Surplus was 29 cents lower on average under penalty than bonus contracts, as workers spent more time and those with higher reservation wages selected in. It was $1.25 higher under high fixed pay and 83 cents higher under high variable pay.

35The idea is intuitively similar to “failure to reduce compound lotteries.”

valuation of the contract. The intuition is similar to how a $30 concert ticket can feel more expensive than a $25 ticket with a $5 processing fee.

Several studies find that (revealed) preferences are affected by outcome- neutral changes in description, even where subjects are aware of all as- pects of the prospect. Consumers under-weight sales taxes unless taxes are made salient, i.e. their demand is more responsive to variation in salient prices than non-salient taxes, even though they are informed about the tax (Chetty et al., 2009). “The customers know what is taxed, but focus on the posted price when shopping.” In online auctions the total sale price (win- ning bid plus shipping cost) is increasing in the shipping cost (Hossain and Morgan, 2006). Revenue equivalence predicts that the total price should not depend the proportion labeled as shipping costs. Valuations of a good are influenced by the distribution of possible prices, and not simply due to rational inference about the good’s value or context dependent preferences (Mazar et al., 2013). Valuations of goods and experiences can be altered by first asking subjects to consider whether their valuation is higher or lower than a transparently uninformative random number (Ariely et al., 2003).

The ideal test would vary the salience of the base pay to see whether the acceptance gap closes. However it difficult to generate ex-ante con- vincing variation in salience without also varying the frame. Nevertheless, the follow-up survey evidence is consistent with this theory. The penalty contract was rated as significantly more generous, and significantly more at- tractive due to good pay. Even among the 41 workers who were unsuccessful in stage 2, the penalty contract scored higher for good pay (p=0.071). The evidence suggests penalty workers had a more positive subjective view of the pay, despite having the same objective information about the terms.³⁶ In Web Appendix A.5 I sketch a model consistent with the salience mechanism. A derives utility from the reference point (replacing expected income in the original setup), plus gain-loss utility. Salience of the reference point is captured by weighting of the gain-loss payoffs. I discuss whether it makes more sense to think of A weighting probabilities or payoffs (i.e. does she underweight the likelihood of the bonus or penalty, or their magnitude),

36Note that the second question followed a question asking them to guess what fraction of workers received $3.50 and right alongside asking how they would feel to receive the max ($3.50) and min ($0.50) payments.