An Analysis of Tit for Tat in the Hawk-Dove Game

(1)

School of Education, Culture and Communication

Division of Mathematics and Physics

BACHELOR’S DEGREE PROJECT IN MATHEMATICS

An Analysis of Tit for Tat in the Hawk-Dove Game

by

Felicia Modin

MAA322 — Examensarbete i matematik för kandidatexamen

DIVISION OF MATHEMATICS AND PHYSICS

MÄLARDALEN UNIVERSITY SE-721 23 VÄSTERÅS, SWEDEN

(2)

School of Education, Culture and Communication

Division of Mathematics and Physics

MAA322 — Bachelor’s Degree Project in Mathematics

Date of presentation:

4 June 2021

Project name:

An Analysis of Tit for Tat in the Hawk-Dove Game

Author: Felicia Modin Version: 8th June 2021 Supervisor: Fredrik Jansson Reviewer: Masood Aryapoor Examiner: Lars Hellström Comprising: 15 ECTS credits

(3)

Abstract

In Axelrod’s tournaments of the Prisoner’s Dilemma, carried out in the 1980s, a strategy called Tit for Tat was declared the winner, and it has since then been thought of as the strategy to use to do as well as possible in different situations. In this thesis, we investigate whether Tit for Tat will still do as well if we change the game to the Hawk-Dove Game. This is done by comparing Tit for Tat to other strategies – All C, All D, Joss and Random – one at a time. First we analyse under which conditions each strategy will be an Evolutionary Stable Strategy, then if it is possible for a population of these two strategies to end up in a stable polymorphism, and finally, if we have a finite population instead of an infinite one, under which conditions selection will favour the fixation of each of the strategies. This leads to the conclusion that how well Tit for Tat will do depends a lot on the different conditions on the game, but in general, the more times that a pair of individuals will meet, and the higher the value of the resource is compared to the cost of fighting, the better Tit for Tat will do.

(4)

Acknowledgements

First, I would like to thank my supervisor, Fredrik Jansson, for being the one who got me interested in game theory in the first place, for his inspiration in the choice of the topic of my thesis, and most of all for his help during my work on my thesis. I would also like to thank my reviewer, Masood Aryapoor, for his help in further improving my thesis.

Finally, a special thanks to Ebba Frejd, my friend and fellow student, for her unwavering support. I’m not sure I could have done this without her.

(5)

3.1.1 TfT vs. All C . . . 20 3.1.2 TfT vs. All D . . . 21 3.1.3 TfT vs. Joss . . . 23 3.1.4 TfT vs. Random . . . 31 3.1.5 Results . . . 34 3.2 Stable Polymorphism . . . 44 3.2.1 TfT vs. All C . . . 44 3.2.2 TfT vs. All D . . . 44 3.2.3 TfT vs. Joss . . . 48 3.2.4 TfT vs. Random . . . 54 3.2.5 Results . . . 58 3.3 Finite populations . . . 59 3.3.1 TfT vs All C . . . 59

(6)

3.3.2 TfT vs All D . . . 59 3.3.3 TfT vs Joss . . . 61 3.3.4 TfT vs Random . . . 68 3.3.5 Results . . . 69 4 Conclusion 73 4.1 TfT vs. All C . . . 73 4.2 TfT vs. All D . . . 73 4.3 TfT vs. Joss . . . 75 4.4 TfT vs. Random . . . 76 4.5 General Conclusion . . . 77 4.6 Future Work . . . 78 A Matlab Code 80

(7)

List of Figures

3.1 Values of v and c for which TfT is an ESS against Joss for m = 1 . . . 38

3.7 Values of v and c for which Joss is an ESS against TfT for m = 1 . . . 41

3.13 Values of v and c for which TfT and All D are in a stable polymorphism for m= 1 . . . 46

3.16 TfT and Joss in stable polymorphism for m = 25 . . . 51

3.17 Values of v and c for which TfT and Joss are in a stable polymorphism for m= 101 . . . 51

3.18 Values of v and c for which TfT and Joss are in a stable polymorphism for m = 2 52 3.19 Values of v and c for which TfT and Joss are in a stable polymorphism for m = 26 52 3.20 Values of v and c for which TfT and Joss are in a stable polymorphism for m= 100 . . . 53

3.21 Values of v and c for which TfT and Random are in a stable polymorphism for m= 1 . . . 55

3.24 Values of v and c for which selection favours the fixation of TfT over Joss for m= 1 . . . 61

(8)

3.25 Values of v and c for which selection favours the fixation of TfT over Joss for m= 15 . . . 62 3.26 Values of v and c for which selection favours the fixation of TfT over Joss for

m= 151 . . . 63 3.27 Values of v and c for which selection favours the fixation of TfT over Joss for

m= 150 . . . 64 3.30 Values of v and c for which selection favours the fixation of Joss over TfT for

(9)

List of Tables

2.1 Payoff matrix for a general hawk-dove game . . . 11

2.2 Payoff matrix for a general two-player two-strategy game . . . 11

3.1 Payoff matrix for a game between TfT and All C . . . 20

3.2 Payoff matrix for a game between TfT and All D . . . 21

3.3 The probability of cooperation for each move in a game between JA and JB. . 23

3.4 The probability of cooperation for each move in a game between TfT and Joss 25 3.5 Payoff matrix for a game between TfT and Joss . . . 31

3.6 Payoff matrix for a game between TfT and Random . . . 32

(10)

Introduction

1.1 Background and Research Question

“When should a person cooperate, and when should a person be selfish, in an ongoing interaction with another person? Should a friend keep providing favors to another friend who never reciprocates? Should a business provide prompt service to another business that is about to be bankrupt? How intensely should the United States try to punish the Soviet Union for a particular hostile act, and what pattern of behavior can the United States use to best elicit cooperative behavior from the Soviet Union?” [1, p. vii]. These were questions that a political scientist named Robert Axelrod wanted to find the answers to in the late 1970s/beginning of the 1980s. It turns out that these types of situations can often be modelled by the Prisoner’s Dilemma [1].

The Prisoner’s Dilemma is a two-player game with two different strategies, cooperate and defect. A player will get the highest possible payoff if they take advantage of the other player by defecting when they cooperate. The second highest payoff will be gained by both players cooperating with each other, while both players will do worse if they both defect. Finally, if a player gets taken advantage of by cooperating when the other player defects, they will get the lowest possible payoff. This is an interesting game, because analysis shows that the best move a player, regardless of what the other player does, will always be to play defect. However, in reality, people often choose to cooperate anyway [4].

Now, when should you cooperate, and when should you defect, to always get the best possible payoff? In 1980, Axelrod invited game theorists to send in different agents to compete in a tournament of the Prisoner’s Dilemma. Every agent got to play against each other agent once, receiving payoff for each move in each game. These payoffs were then summed up, to see who did the best in general and received the most payoff, and thus won the tournament. The result was that a strategy called Tit for Tat – always begin by cooperating, and then always copy your opponent’s last move – won. However, Axelrod himself could find other strategies that would have won over Tit for Tat in this specific setting. After analysing the results, he let people know how the different strategies had done in the tournament, his conclusions, as well as informing about the strategies that were seemingly even better than Tit for Tat [1].

After that, he invited anyone who wanted to participate in a second tournament of the Pris-oner’s Dilemma. This time, not only game theorists participated, but also computer hobbyists, as well as professors in computer science, physics, and evolutionary biology. Knowing that Tit for Tat did so well in the first tournament, a lot of them tried to improve upon this strategy, as well as on the strategies Axelrod had found would have beaten Tit for Tat in the first tournament. However, perhaps surprisingly, the original, simple, Tit for Tat ended up being the winner of this second tournament too [1].

(11)

both tournaments, it has been seen as the strategy to use to get people to cooperate with you as much as possible – it always initiates cooperation, and rewards cooperation from the opponent with cooperation, while it does not let the opponent take advantage of it, and instead retaliates defection.

However, the conditions under which Axelrod concluded that Tit for Tat was the best strategy were very specific. One of the major assumptions he used were the payoffs, which were modelled after the Prisoner’s Dilemma, and kept constant on specific values. If those values were changed – or if the game was changed entirely, changing not only the specific values of the payoffs, but also the relations between the different payoffs – what would happen then? Would Tit for Tat still be the best strategy?

In this thesis I am going to investigate exactly that – is Tit for Tat still the best strategy if you change the conditions of the game, using the Hawk-Dove Game instead of the Prisoner’s Dilemma?

1.2 Literature Review

In 1984, Robert Axelrod first published his book The Evolution of Cooperation [1] where he analysed which strategies would be the best to get as much cooperation as possible in the Prisoner’s Dilemma. Here, Tit for Tat was first proposed to be the best strategy to achieve this.

Axelrod’s tournament and its results have inspired a lot of research in the area. In [11], they looked at Axelrod’s tournament, and specifically analysed if Tit for Tat would still be the best strategy in the Prisoner’s Dilemma if the conditions changed. Some people have instead tried to find other strategies that will do better than Tit for Tat in the Prisoner’s Dilemma, such as [9]. Other people have moved on from the Prisoner’s Dilemma to other games, such as [3], where they looked at cooperation in the Hawk-Dove Game through simulations. I have, however, not been able to find any research that has looked at how Tit for Tat would do in the Hawk-Dove Game, which is why I thought that would be interesting to analyse. The research that has been the closest has also always been done through simulations, and thus I will instead look at the problem analytically.

In this thesis, the game that will be analysed is the Hawk-Dove Game. An excellent explanation of how this game works and how to model it is provided in [2]. One of the ways to analyse strategies in a two-player game is by looking at the Evolutionary Stable Strategy. This concept was first defined in [6], and an explanation of the theory behind the Evolutionary Stable Strategy as well as how to use it can be found in [10]. In this last source is also an explanation on how the analysis will change if you instead of infinite populations have finite populations. Another way of analysing the strategies is by looking at stable polymorphisms, which has been explained in [7]. Finally, some explanations of other necessary terminology or formulas have been found in [4], [5] and [8].

(12)

1.3 Overview of The Thesis

I have switched the game from the Prisoner’s Dilemma to the Hawk-Dove Game, and to avoid limited results, I have kept variables for the payoffs instead of choosing specific numbers. Furthermore, I have chosen to look at this problem from different perspectives, since there is no one way of defining what being the “best strategy” actually means.

In the first chapter, I will begin by describing the Hawk-Dove Game, as well as describing how to model it. I will continue by, in detail, explaining the three different ways that I will be looking at the problem – the concept of an Evolutionary Stable Strategy, what it takes for a population to be a stable polymorphism, and finally what will happen if we have finite populations instead of infinite ones – and how they will be used in the thesis. The chapter will be concluded with a description of the strategies that will be used in the analysis.

In the next chapter, I will use the model described in the previous chapter to compare Tit for Tat to one other strategy at a time – All C, All D, Joss, and Random. First, I will investigate whether either of them can be an Evolutionary Stable Strategy, and if so under which conditions, to see how well Tit for Tat is at invading other strategies, and if it can withstand the invasion of other strategies. I will then take this one step further, to see if Tit for Tat can coexist – that is, be in a stable polymorphism – with either of the other strategies, or if one of the strategies will take over completely. In our final analysis, we will move to finite populations. Here, we will find the conditions for when selection will favour the fixation of either strategy, to see if Tit for Tat will end up completely taking over a mixed population, or if it will instead be wiped out.

(13)

Model

To be able to analyse how well Tit for Tat will do if we have the Hawk-Dove Game instead of the Prisoner’s Dilemma like Axelrod, we need to understand what the Hawk-Dove Game actually is. Thus, we will begin this chapter by explaining what it is, how it works, and how to model it. However, we also need some methods to be able to analyse it, so the rest of this chapter will be spent looking at the methods we will use. First, we will introduce the concept of an Evolutionary Stable Strategy, then we will look at what it means for a population to be a stable polymorphism, and finally we will move our analysis from infinite populations to finite populations to see if that will change things.

2.1 The Hawk-Dove Game

We imagine that we have a population with two different types of individuals, where each type of individual uses one specific strategy when they fight against any other individual. When we look at the Hawk-Dove Game, these individuals can be thought of as birds, more specifically hawks and doves – hence the name of the game. These birds fight over some kind of resource, such as territory, where we will simply call the hawks’ strategy “hawk” and the doves’ strategy “dove”. The hawk begins by displaying hostility, and always escalates the battle until one of them gets injured and retreats, leaving the other individual to claim the resource. The dove on the other hand also begins by displaying hostility, but it will retreat immediately if the opponent escalates. This means that if two doves meet, they will both back down and simply share the resource equally. If a hawk and a dove meet, then the dove will immediately back down, and the hawk will take the resource [2].

Now, what will the payoffs be in a game like this? We let v denote the value of the resource. If a hawk and a dove meet, then we already know that the hawk will take the entire resource, meaning that the hawk will get payoff v while the dove will get payoff 0. If instead two doves meet each other, they will simply share the resource, leaving both individuals with payoff v/2. If two hawks meet each other however, it will be a bit more complicated. Both hawks are equally likely to win and thus receive the whole resource. However, the individual who does not win will instead get hurt, which also needs to be taken into consideration. We let c denote the cost of getting hurt in the fight. The mean payoff for both individuals will then be v/2 − c/2, or, written a bit more nicely, (v − c)/2 [2].

Now that we have all the payoffs, we can create a payoff matrix for the Hawk-Dove Game. This payoff matrix is shown in Table 2.1.

What we want to do in this thesis is to investigate how good Tit for Tat is if you want as much cooperation as possible. Thus, we must adjust our model of the Hawk-Dove Game to include cooperate as a term. In our model, we always want the resource to be worth a positive value, to make sure the individuals will actually try to gain the resource by participating in

(14)

Hawk Dove Hawk v−c₂ v

Dove 0 v₂

Table 2.1: Payoff matrix for a general hawk-dove game

the game. We also want the cost to be a positive value, to make sure the individuals will lose something by fighting. That is, we have v > 0 and c > 0. Since the hawk will always try to take advantage of the other individual, while the dove is the “nice” one, and two individuals will gain more from both being doves than both being hawks, we will in this thesis use the terminology from the Prisoner’s Dilemma and use the term defect instead of hawk, and cooperate instead of dove.

2.2 Evolutionary Stable Strategy

This entire section is based on [10]. That is, all of the formulas, as well as the basic theory, have been taken from this source. I have, however, added a lot of explanations as well as added some calculations, to make everything more easy to understand.

We look at a general two-player game with two strategies, A and B. For the current analysis, we imagine that we have an infinitely large population of A individuals, and introduce mutant players of one specific type, which we will call B. For a game between A and B, the payoff matrix is given by Table 2.2.

A B A a b B c d

Table 2.2: Payoff matrix for a general two-player two-strategy game

We let xA and xB be the frequency of A and B individuals, respectively. That is, if the

meetings are random, then xA and xB will denote the probabilities of interacting with an A or

B individual, respectively. The expected payoffs for an A individual will be the payoff for a meeting with another A player times the probability that they will meet an A individual, plus the payoff for a meeting with a B individual times the probability that they will meet a B individual. We can find the expected payoffs for a B individual in the same way. If we let fAand fBdenote

the expected payoffs for A and B individuals, respectively, then these will be given by

(15)

f_B= cxA+ dxB. (2.2)

These are also called the fitness functions for A and B. Fitness is here defined as the rate of reproduction.

Now, we want to avoid B, the mutant, being able to invade A. As we have already stated, we imagine that we have an infinitely large population of A individuals, and so we assume that we only have an infinitesimal number of invading individuals, B. This means that we can say that the frequency of B, xB, will be ε, and that the frequency of A, xA, will be 1 − ε. If we want

Bnot to be able to invade A, then A will need to have higher rate of reproduction than B – that is, A will need to have greater fitness than B. Using the fitness functions defined in Formula 2.1 and Formula 2.2, we see that this will be the case if

a(1 − ε) + bε > c(1 − ε) + dε. (2.3) We know that the population consists mostly of A players, which means that a and c will affect this inequality the most. Thus, we will focus on those two payoffs. Since we do not have any conditions on the different payoffs’ relations to each other, we can either have a 6= c or a = c. We begin by looking at the case for which a 6= c. Since we have an infinitely large population with an infinitely small number of invaders, we can let ε → 0:

lim ε →0 (a(1 − ε) + bε) ≥ lim ε →0 (c(1 − ε) + dε) a(1 − 0) + b · 0 ≥ c(1 − 0) + d · 0 a≥ c.

However, we already know that a 6= c, so we will need to have a > c.

Now, what happens if we instead have a = c? Going back to Formula 2.3, we see that we get

a(1 − ε) + bε > c(1 − ε) + dε

c(1 − ε) + bε > c(1 − ε) + dε a= c bε > dε

b> d.

Since these are the only two possible cases, we see that we will have the two possible conditions

a> c (2.4)

and

a= c and b > d. (2.5)

These are the conditions for a strategy to be an Evolutionary Stable Strategy (ESS) first defined by Maynard Smith and Price in 1973 [6]. We can write this as the following lemma: Lemma 1 (Evolutionary Stable Strategy). We have a general two-player game with strategies A and B, and payoffs given by Table 2.2. If we have an infinite population consisting of both A

(16)

individuals and an infinitesimal number of mutant B individuals, then A will be an Evolutionary Stable Strategy if a> c or a= c and b > d holds.

Using this method, we will be able to see if Tit for Tat, abbreviated as TfT, will be able to invade any of the other strategies, which would point towards it being a good strategy. We will also be able to see how good it is at avoiding invasion by the other strategies.

2.3 Stable Polymorphism

This entire section is based on a section about stable polymorphism in [7]. I have, however, adapted this to include a variable m for how many times two different individuals will meet each other, instead of assuming that they will only meet each other one time.

We make a number of assumptions. To begin with, we assume that the population is large. Furthermore, all of the individuals in the population have the same abilities, and when they fight against each other for a resource, they will have the same benefits and costs, and the roles in the game are symmetric, which means that there are no labelling differences, such as some individuals being male and some being female. When the individuals pair up with another individual to fight over a resource, the pairing is completely random. They also choose their move without knowing what the other individual is going to choose, and without being able to change their move once they find out what move the other individual is making. This last condition can be thought of as both individuals doing their move at the exact same time.

We assume that the individuals in a population each use one of two strategies, A and B. We have already looked at, in Section 2.2 what the conditions will be for strategy B to be able to invade strategy A, if the population consists of mostly individuals with strategy A, with an infinitesimal number of individuals with strategy B. However, what happens after a while? Will they reach a point where strategy B will no longer increase, and they will instead coexist? If that is the case, then the population will have reached a stable polymorphism.

Let us say that a proportion q of the population will use strategy A, while the rest of the population will use strategy B. This means that every time an individual pairs up with another individual to fight, the probability that the other individual will use strategy A is q, and the probability that the other player will use strategy B is 1 − q. We assume that we have the same payoffs as in Table 2.2. We also assume that if two individuals meet, they will meet a total of m times. Then the average payoff for an individual with strategy A will be

W_A(q) = qma + (1 − q)mb, and the average payoff for an individual with strategy B will be

(17)

Now, if the payoff for one of the strategies is greater than the payoff for the other strategy, then that strategy will do better and will increase. For the proportions to be stable, we instead need the payoffs to be equal; that is, WA(q) = WB(q). Rewriting this equality, we get an

expression for q, and know that if the proportion of A individuals is equal to this q, then we have a stable polymorphism.

This will be able to show us if TfT is able to completely take over the populations it invades, or if it will be forced to coexist with the other strategy, as well as if any other strategy is able to completely wipe out TfT.

2.4 Finite populations

This section is based entirely on [10], unless explicitly stated otherwise. I have, however, added or expanded on some of the explanations, as well as added the steps in the calculations instead of just providing the final formulas, to make it easier to understand what exactly is happening.

So far, we have only looked at populations with an infinite number of individuals. However, since infinite populations is a simplifying assumption, it is also interesting to see what will happen if we instead have finite populations. Here, we can calculate fixation probabilities, to see which – if any – strategy selection favours. That is, if a population has only one individual with the strategy A and the rest have strategy B, then the fixation probability of A is the probability that A will take over the entire population. In this section, we are going to find a formula for when selection favours the fixation of A.

Say that we have a population of N individuals, out of which i are of type A and N − i are of type B. First, we want to find the expected payoff for both A and B. For each individual, we know that there are N − 1 other individuals. For each A individual, there are i − 1 other A individuals, and the number of B individuals will be N − i. Thus, the probabilities that an A individual will interact with an A or B individual will be (i − 1)/(N − 1) and (N − i)/(N − 1), respectively. In the same way, for each B individual, there are N − i − 1 other B individuals, and the number of A individuals will be i. This means that the probability that a B individual will interact with an A or B individual will be i/(N − 1) and (N − i − 1)/(N − 1), respectively. We use the same payoffs as in Table 2.2.

Using the probabilities for A and B individuals to interact with A or B individuals, and combining that with the payoff matrix, we can find the expected payoffs for A and B. Letting Fi

and Gibe the payoffs for an A and B individual, respectively, in a population with N individuals,

out of which i are A individuals, we find that Fiand Giwill be:

F_i= a · i− 1 N− 1+ b · N− i N− 1 = a(i − 1) + b(N − i) N− 1 , G_i= c · i N− 1+ d · N− 1 − i N− 1 = ci+ d(N − i − 1) N− 1 .

Usually, we interpret the expected payoff as the fitness. However, here we want to be able to vary how much the payoff influences the fitness. This is so we can have a base fitness that all individuals will have, regardless of how well they do in the game, and to avoid the game having to be the sole decider of the individuals’ reproduction rates – fitness – if we do not want

(18)

it to be, since in reality it often is not. We do this by introducing a new parameter w, which is a number between 0 and 1 and will measure the intensity of selection. We let fiand gidenote the

fitness of A and B, respectively, under the same conditions as above. Then the fitness of A and B, respectively, will be given by

fi= 1 + w(Fi− 1) = 1 − w + wFi,

g_i= 1 + w(G_i− 1) = 1 − w + wG_i.

As we see, if w = 0, then the fitness will always be 1, and the payoffs of the game will have no relevance at all for the future of the population. If w = 1 on the other hand, then the fitness will only depend on the payoffs.

We now imagine that we have a Moran process, that is, a process used in finite populations of constant size, to describe stochastic evolution; at each time step, one random individual is chosen for reproduction with a probability proportional to its fitness, and one random individual is chosen for death by uniform distribution [5].

We want to find the probability that the the number of A individuals in the population will move from i to i + 1, as well as the probability that it will move from i to i − 1. If the number of Aindividuals is going to increase, then the reproduction will have to lead to a new A individual, and the individual chosen for death will have to be a B individual. If it is going to increase, then it has to be the other way around – reproduction leads to a new B individual, and an A individual is chosen for death.

First we want to find the probability that the reproduction at this time step will lead to an Aindividual. We know that if an A individual is chosen, then reproduction will result in a new A individual, and the other way around if a B individual is chosen. We also know that an individual is chosen with a probability proportional to its fitness. This means that to find the probability that an A individual is chosen, we will have to look at the total relative fitness. Since we have i number of A individuals, each with fitness fi, and the total population consists

of these individuals plus N − i number of B individuals with fitness gi, the probability that an A

individual will be chosen for reproduction will be i f_i i fi+ (N − i)gi

.

In the same way, the probability that the new individual will be a B individual will be (N − i)gi

i f_i+ (N − i)gi

.

The probability that the individual chosen for death will be an A or B individual is simply the probability that an A or B individual will be chosen randomly out of the entire population, since we assume that death is completely random. Hence, the probability that the individual chosen for death will be an A individual is

i N,

(19)

and the probability that will be a B individual is N− i

N .

Combining these, we see that the probability that the number of A individuals in the population will change from i to i + 1, denoted by pi,i+1, will be

pi,i+1=

i f_i i f_i+ (N − i)g_i

N− i

N , (2.6)

and the probability that the number of A individuals will change from i to i − 1, denoted by

p_i,i−1, will be pi,i−1= (N − i)gi i f_i+ (N − i)gi i N. (2.7)

We can note that the only possible outcomes are that the number of A individuals increases by one, decreases by one, or does not change at all. Thus, the probability that we will have the same number of A individuals in one time step as we have now, denoted by pi,i, will be

pi,i= 1 − pi,i+1− pi,i−1

= 1 − i fi i f_i+ (N − i)gi N− i N − (N − i)g_i i f_i+ (N − i)gi i N = 1 −i(N − i) fi− i(N − i)gi

N(i fi+ (N − i)gi)

= N(i fi+ (N − i)gi) N(i fi+ (N − i)gi)

−i(N − i) fi− i(N − i)gi N(i fi+ (N − i)gi) = i 2_f i+ N2gi− 2Nigi− i2gi N(i fi+ (N − i)gi) = i 2_f i+ (N − i)2gi N(i f_i+ (N − i)g_i).

We can also note that if there are no A individuals then there will obviously still be no A individuals in the next time step, and similarly, if there are only A individuals in the population, then that will be true in the next time step as well. This means that p0,0= 1 and pN,N = 1,

which is also seen from the formula for pi,i. If any population reaches either of these cases,

then they will always stay there, and thus we clearly have two absorbing states, where any mixed population will always eventually end up.

To find out when selection favours the fixation of A, we can use the fixation probability of A. We are going to use Equation 6.13 in [10], which, after changing the names of the index of summation and the index of the product to better fit our usual notation, is

ρA=

1

1 + ∑N−1_k=1 ∏ki=1γi

, (2.8)

where γi= βi/αi, and βiis the probability of a transition from i to i − 1 and αiis the probability

(20)

That is, γi= pi,i−1/pi,i+1. However, we already have formulas for pi,i−1and pi,i+1– Formula

2.7 and Formula 2.6, respectively – and using those here, we see that we get γi= pi,i−1 pi,i+1 = (N−i)gi i fi+(N−i)gi i N i fi i fi+(N−i)gi N−i N = (N − i)gii i f_i(N − i) = gi f_i.

Now, substituting this into Formula 2.8, we get that the fixation probability for A will be ρA= 1 1 + ∑N−1_k=1 ∏ki=1 gi fi . (2.9)

Looking at this directly is a bit complicated. However, we can make it a bit easier by considering the limit of weak selection. If we do a Taylor expansion of Formula 2.9 for w → 0, we will get ρA≈ 1 N 1 1 − (αN − β )w/6, (2.10) where α = a + 2b − c − 2d and β = 2a + b + c − 4d [8].

Now, we need to know for which values of ρA selection favours the fixation of A. We

know that we have a population with N individuals of the types A and B. Obviously, all the A individuals will need to have the same fixation probability, and all the B individuals will need to have the same fixation probability. Say that both the A and the B individuals will have the same fixation probability. If we add together the fixation probabilities of all N individuals, the result must be 1. For this to hold, each individual will need to have fixation probability 1/N. It is now easy to see that if we want selection to favour the fixation of A, then we simply need the fixation probability of each A individual to be greater than 1/N. That is, selection will favour the fixation of A if ρA> 1/N. Using this condition along with the approximate formula for ρA,

Formula 2.10, we see that we get 1 N 1 1 − (αN − β )w/6 > 1 N 1 1 − (αN − β )w/6 > 1.

Clearly, the left hand side of the above inequality has to be positive. Since the numerator is positive, the denominator has to be positive too. That is, we need to have 1 − (αN − β )w/6 > 0. If we assume that this holds, we can multiply both sides of the inequality by 1 − (αN − β )w/6

(21)

without changing the inequality. This gives us

1 > 1 − (αN − β )w/6 0 > −(αN − β )w/6 0 < (αN − β )w/6, but we already know that w > 0, and thus we get

α N − β > 0 α N > β .

If we then substitute back α and β for their formulas, we see that this will be the same as (a + 2b − c − 2d)N > 2a + b + c − 4d

a(N − 2) + b(2N − 1) > c(N + 1) + d(2N − 4). (2.11) That is, if Formula 2.11 holds, as well as 1 − (αN − β )w/6 > 0, then selection will favour the fixation of A. This last condition, however, can be rewritten as 1 > (αN − β )w/6. In our model, N is finite since we have finite populations, and so are α and β , since we cannot have infinite payoffs. However, w → 0, so (αN − β )w/6 will be infinitely small. That is, this condition will always hold, and we will only have to look at Formula 2.11 to see if selection will favour the fixation of A.

2.5 Strategies

In this section we will introduce the different strategies that will be used in the analysis. These are strategies taken from Axelrod’s tournament, chosen because they did well there or because they could lead to interesting analyses.

2.5.1 Tit for Tat

Tit for Tat (TfT) is the strategy that won both of Axelrod’s tournaments, and thus we will try to find how well it does against the other strategies in this game, the Hawk-Dove Game. This will allow us to look into if TfT is a very good strategy in general to promote cooperation, or if it was only the specific properties of the Prisoner’s Dilemma, as well as the setup of Axelrod’s tournaments, that made it do so well in the tournaments.

The strategy of Tit for Tat is to always begin playing cooperate. After that, it will always copy the last move of the other player. That is, if the other player plays defect on move n, then Tit for Tat will play defect on move n + 1, and if the other plays cooperate on move n, then Tit for Tat will play cooperate on move n + 1. This means that TfT will reward cooperation, while also retaliating when the other player plays defect to avoid being taken advantage of.

(22)

2.5.2 All C

The first strategy that we will compare TfT to is the very basic strategy to always be kind – or weak, depending on how you look at it. This strategy is called All C and simply always cooperates, regardless of what the other player does. We can note that in the Hawk-Dove Game terminology, this strategy would simply always be a dove. This is a good strategy for comparison.

2.5.3 All D

The All D strategy is the opposite of All C – it always tries to take advantage of the other player, by defecting regardless of what the opponent does. In the Hawk-Dove Game terminology, this strategy would always be a hawk. In the same way as All C, this will be a very good strategy to compare the results of the other strategies to.

2.5.4 Joss

Joss is one of the strategies sent in to Axelrod’s second tournament by a Swiss mathematician named Johann Joss – hence the name of the strategy. It is a version of Tit for Tat in that is always begins by cooperating, and then copies the opponent’s last move for the rest of the game. However, when the opponent plays cooperate, there is a 10% probability that Joss will play defect anyway, to try to take advantage of the other player. In Axelrod’s tournament, Joss ended up at place 29 out of the 63 players, meaning it was not bad, but it also did not do well [1]. However, it is an interesting strategy, and it might be better at taking advantage of its opponents in the Hawk-Dove Game. Therefore this strategy will be included in the analysis in this thesis.

2.5.5 Random

In Axelrod’s tournaments, Axelrod included a random strategy that he let play against all competing strategies, to see what will happen if a player always just chooses their moves completely randomly [1]. This is a very interesting strategy and definitely relevant to include, since it works as a base line. By comparing how well the other strategies do to how well Random does, we can get an idea of if the other strategies are actually good, or if they would be just as well – or better – off by just choosing their moves randomly. The way this strategy works is to, on every move, choose either defect or cooperate completely randomly, each with probability 0.5.

(23)

Mathematical Analysis

In this chapter, the model defined in the previous chapter will be used to analyse the problem analytically. First, the concept of an ESS will be used to see when (if) TfT will be able to invade the other strategies, and when (if) the other strategies will be able to invade TfT. Then, the analysis will move on to seeing if TfT can coexist with another strategy – that is, be in a stable polymorphism – or if either of the strategies will end up completely taking over the population. Finally, the infinite populations from the first two parts of the chapter will be turned into finite populations, to see if it is possible to predict which of the populations will end up taking over the population in that case.

3.1 Evolutionary Stable Strategy

In the Prisoner’s Dilemma, specifically in Axelrod’s two tournaments of it, Tit for Tat was found to be the best strategy. Now, what happens if we change the game to the Hawk and Dove game? In this first subsection, we will investigate this by letting TfT play against each of the other strategies – All C, All D, Joss, and Random – one at a time. We will then analyse under which conditions TfT will be an ESS against each of these strategies, and under which conditions each of the strategies will be an ESS against TfT. This will allow us to see if, first of all, a population with strategy TfT can be invaded by another strategy, and second of all, if TfT can invade already existing populations of other strategies.

3.1.1 TfT vs. All C

We imagine that we have a population with only TfT and All C individuals. To be able to see if either of them can be an ESS, we first need to find their respective payoffs. Since All C always just cooperates and TfT begins by cooperating and then copies its opponent’s last move, they will both always just cooperate when playing against each other. TfT will also always cooperate against itself, since it will never be the first to defect, and All C will obviously also always cooperate. Hence the payoff matrix for a game between TfT and All C, where two individuals who meet will meet a total of m times, will be as in Table 3.1.

TfT All C TfT mv₂ mv₂ All C mv₂ mv₂

(24)

By Lemma 1, TfT will be an ESS if mv/2 > mv/2, which is obviously not true, or if mv/2 = mv/2 and mv/2 > mv/2, which is again not true. Apparently, TfT cannot be an ESS when playing against All C, and can thus always be invaded by All C. However, we see that all the conditions will be exactly the same for for All C, which means that All C cannot be an ESS either. Consequently, both strategies can theoretically be invaded by each other.

3.1.2 TfT vs. All D

We now imagine that we have a population of only TfT and All D individuals. Once again, we want to find their respective payoffs. When TfT plays against itself, we know that the payoff will be mv/2 since it will always cooperate. When All D plays against itself, both players will always defect, and so the payoff will be m(v − c)/2. Now, what about when TfT plays against All D? On the first move, TfT will cooperate while All D defects, but after that TfT will copy All D and defect, while All D still defects, and then they will keep defecting until the end of the game. Thus the payoff for TfT when it plays against All D will be (m − 1)(v − c)/2, and the payoff for All D when it plays against TfT will be v + (m − 1)(v − c)/2. Now that we have all the payoffs, we can create the payoff matrix, which can be seen in Table 3.2.

TfT All D

TfT m₂v (m − 1)v−c₂ All D v+ (m − 1)v−c₂ mv−c

2

Table 3.2: Payoff matrix for a game between TfT and All D

Now, when is TfT or All D an ESS in this case? We begin by looking at TfT. First, by Lemma 1, we know that it is an ESS if

mv 2> v + (m − 1) v− c 2 mv 2> v + m v 2− v 2− m c 2+ c 2 0 > v −v 2− m c 2+ c 2 mc 2> v 2+ c 2 m> v+ c c m> v c+ 1. However, it might also be an ESS if

mv 2 = v + (m − 1) v− c 2 m= v c+ 1

(25)

and (m − 1)v− c 2 > m v− c 2 mv− c 2 − v− c 2 > m v− c 2 −v− c 2 > 0 v− c < 0 v< c. This means that TfT will be an ESS against All D if

m≥ v c+ 1 and, if we have an equality,

v< c.

Now, can All D be an ESS? According to Lemma 1, All D will be an ESS if mv− c 2 > (m − 1) v− c 2 mv− c 2 > m v− c 2 − v− c 2 0 > −v− c 2 v− c 2 > 0 v− c > 0 v> c. It will also be an ESS if

mv− c 2 = (m − 1) v− c 2 v= c and v+ (m − 1)v− c 2 > m v 2 m< v c+ 1. Since we already know that v = c, this becomes

m<v c+ 1 =

c

c+ 1 = 1 + 1 = 2, and since m > 0 and m is an integer, that gives us m = 1.

That is, All D will be an ESS against TfT if v≥ c and, in case of equality,

(26)

3.1.3 TfT vs. Joss

We are now going to look at a population of TfT and Joss individuals. We know that when TfT plays against itself the payoff is mv/2, but we need to find the payoff for Joss when it plays against itself. We will begin by doing this.

We call the two players, both with strategy Joss, JA and JB. On the first move, both JAand

J_B will play cooperate. We know that Joss always defects the next move after the opponent defects, and cooperates with probability 0.9 the next move after the opponent cooperates. This means that on the next move, JAwill cooperate with probability 0.9, and defect with probability

0.1. However, so will JB. Now, the probability that JA will cooperate on the third move is

the probability that JB cooperated on the second move multiplied by the probability that JA

cooperates when the other player cooperates, that is, 0.9 · 0.9 = 0.92. If we just keep going like this, remembering that JA and JBwill, probabilistically, behave in the same way, we can write

out their moves in Table 3.3. The probability that they will defect will always just be 1 minus the probability that they will cooperate, so to keep the table from getting too messy I have only included the probability that the individuals will cooperate on each move.

J_A 1 0.9 0.92 0.93 . . . 0.9m−1 J_B 1 0.9 0.92 0.93 . . . 0.9m−1

Table 3.3: The probability of cooperation for each move in a game between JAand JB

Now, we can try to find the expected payoff for JA (which will obviously be identical to

the expected payoff of JB). On the first move, they will both cooperate, so the payoff will

be v/2. On the next move, they will both cooperate with probability 0.9 · 0.9, which gives payoff v/2, J_Acooperate and JBdefect with probability 0.9(1 − 0.9), giving payoff 0, JAdefect

and JBcooperate with probability (1 − 0.9) · 0.9, giving payoff v, and finally both defect with

probability (1 − 0.9)(1 − 0.9), giving payoff (v − c)/2. If we keep going like this, we can find the expected payoff. If we let P(A, B) denote the expected payoff for a strategy A in a game against a strategy B, then the expected payoff for Joss when it plays against itself will be

P( j, j) = v 2+ 0.9 2_·v 2+ (1 − 0.9) · 0.9v + (1 − 0.9) 2_·v− c 2 + 0.94·v 2+ 1 − 0.9 2_{· 0.9}2_v_{+ 1 − 0.9}22 ·v− c 2 + 0.99·v 2+ 1 − 0.9 3_{· 0.9}3_v_{+ 1 − 0.9}32 ·v− c 2 + . . . + 0.92(m−1)·v 2+ 1 − 0.9 m−1_{· 0.9}m−1_v_{+ 1 − 0.9}m−12 ·v− c 2 = v 2+ m−1

∑

i=1 0.92i·v 2+ 1 − 0.9 i_{· 0.9}i_v_{+ 1 − 0.9}i2 ·v− c 2 .

(27)

m−1

∑

i=1 0.92i·v 2+ 1 − 0.9 i_{· 0.9}i_v_{+ 1 − 0.9}i2 ·v− c 2 = m−1

∑

i=1 0.92i·v 2+ 0.9 i_v_{− 0.9}2i_v_{+ 1 − 2 · 0.9}i_{+ 0.9}2i_·v− c 2 = m−1

∑

i=1 v 2− c 2 1 − 2 · 0.9 i_{+ 0.9}2i = v 2 m−1

∑

i=1 1 −c 2 m−1

∑

i=1 1 − m−1

∑

i=1 2 · 0.9i + m−1

∑

i=1 0.92i ! = v 2(m − 1) − c 2 (m − 1) − m−1

∑

i=1 2 · 0.9 · 0.9i−1 + m−1

∑

i=1 0.92· 0.92i−1 ! = v 2(m − 1) − c 2 m− 1 − 2 · 0.9 ·1 − 0.9 m−1 1 − 0.9 + 0.81 · 1 − 0.81m−1 1 − 0.81 = v 2(m − 1) − c 2 m− 1 − 18 1 − 0.9m−1 +81 19 1 − 0.81 m−1 = v 2(m − 1) − c 2 m−280 19 + 18 · 0.9 m−1₋81 19· 0.81 m−1 = v 2(m − 1) − c 2 m−280 19 + 20 · 0.9 m₋100 19 · 0.81 m . Thus the payoff for Joss when it plays against itself will be

P( j, j) = v 2+ v 2(m − 1) − c 2 m−280 19 + 20 · 0.9 m₋100 19 · 0.81 m = v 2· m − c 2 m−280 19 + 20 · 0.9 m₋100 19 · 0.81 m .

Now we want to find what happens when TfT plays against Joss. We know that on the first move, both players will cooperate. On the second move, TfT will cooperate, but Joss will only cooperate with probability 0.9. On the third move, TfT will cooperate if Joss cooperated on the previous move, which it did with probability 0.9, and Joss will again cooperate with probability 0.9, since TfT cooperated on the previous move. On the fourth move, TfT will cooperate with the same probability that Joss cooperated on the previous move, which is 0.9, and Joss will cooperate with probability 0.9 if TfT cooperated on the previous move, which it did with probability 0.9. That is, Joss will cooperate with probability 0.92. With the same reasoning, you can find the probabilities of TfT and Joss cooperating for all future moves. We write them out in Table 3.4. Just like before, we only write out the probabilities of the strategies cooperating, but the probability of them defecting is just the probability of them cooperating subtracted from 1.

Now, remembering the different payoffs depending on if the players cooperate or defect, and noting that the probability of TfT defecting on the second move is 1 − 1 = 1 − 0.90= 0, we can use the table to write out a formula for TfT’s payoff against Joss.

(28)

TfT 1 1 0.9 0.9 0.92 0.92 0.93 . . . Joss 1 0.9 0.9 0.92 0.92 0.93 0.93 . . .

Table 3.4: The probability of cooperation for each move in a game between TfT and Joss

P(t, j) = v 2+ 0.9 · v 2+ 1 − 0.9 0_{· 0.9v + 1 − 0.9}0_{(1 − 0.9) ·}v− c 2 + 0.92·v 2+ (1 − 0.9) · 0.9v + (1 − 0.9)(1 − 0.9) · v− c 2 + 0.93·v 2+ (1 − 0.9) · 0.9 2_v_{+ (1 − 0.9) 1 − 0.9}2_·v− c 2 + 0.94·v 2+ 1 − 0.9 2_{· 0.9}2_v_{+ 1 − 0.9}2 1 − 0.92 ·v− c 2 + 0.95·v 2+ 1 − 0.9 2_{· 0.9}3_v_{+ 1 − 0.9}2 1 − 0.93 ·v− c 2 + ...

We now want to try to find a formula for this. We could write is as a sum as P(t, j) = v 2+ n

∑

i=1 0.92i−1·v 2+ 1 − 0.9 i−1_{· 0.9}i_v_{+ 1 − 0.9}i−1 1 − 0.9i ·v− c 2 + 0.92i·v 2+ 1 − 0.9 i_{· 0.9}i_v_{+ 1 − 0.9}i 1 − 0.9i ·v− c 2

for some n, but we see that this sum will take two of the players’ moves at a time, so this will only work when m is odd, and n will then be (m − 1)/2. However, we cannot be sure that m will actually be odd, so we need to add another term that represents the last move in the game when m is even. We let this term be multiplied by a variable k defined by

k= ( 0 when m is odd, 1 when m is even. We also define n by n= (_m−1 2 when m is odd, m−2 2 when m is even.

Using these k and n, we can write the payoff for TfT when it plays against Joss as P(t, j) = v 2+ n

∑

i=1 0.92i−1·v 2+ 1 − 0.9 i−1_{· 0.9}i_v_{+ 1 − 0.9}i−1 1 − 0.9i ·v− c 2 + 0.92i·v 2+ 1 − 0.9 i_{· 0.9}i_v_{+ 1 − 0.9}i 1 − 0.9i ·v− c 2 + 0.9m−1·v 2+ 1 − 0.9m−12 · 0.9m+12 _v₊ 1 − 0.9m−12 1 − 0.9m+12 ·v− c 2 .

(29)

Now, we can simplify this expression. We begin with the sum and then do the second part of the expression, to make it less messy.

n

∑

i=1 0.92i−1·v 2+ 1 − 0.9 i−1_{· 0.9}i_v_{+ 1 − 0.9}i−1 1 − 0.9i ·v− c 2 + 0.92i·v 2+ 1 − 0.9 i_{· 0.9}i_v_{+ 1 − 0.9}i 1 − 0.9i ·v− c 2 = n

∑

i=1 0.92i−1·v 2+ 0.9

i_v_{− 0.9}2i−1_v_{+ 1 − 0.9}i_{− 0.9}i−1_{+ 0.9}2i−1_·v− c

2 + 0.92i·v 2+ 0.9 i_v_{− 0.9}2i_v_{+ 1 − 2 · 0.9}i_{+ 0.9}2i_·v− c 2 = n

∑

i=1 v+ 0.9i·v 2− 0.9 i−1_·v 2− 1 − 0.9 i_{− 0.9}i−1_{+ 0.9}2i−1_·c 2 − 1 − 2 · 0.9i+ 0.92i ·c 2 = n

∑

i=1 v 2 2 + 0.9 i_{− 0.9}i−1₋c 2 2 − 3 · 0.9 i_{− 0.9}i−1_{+ 0.9}2i_{+ 0.9}2i−1 = n

∑

i=1 v 2 2 + 0.9 i−1_{(0.9 − 1) −}c 2 2 − 0.9 i−1_{(3 · 0.9 + 1) + 0.9}2i−2 _0.92_{+ 0.9} = n

∑

i=1 v 2 2 − 0.1 · 0.9 i−1₋c 2 2 − 3.7 · 0.9 i−1_{+ 1.71 · 0.9}2i−2 = v 2 n

∑

i=1 2 − 0.1 · 0.9i−1 −c 2 n

∑

i=1 2 − 3.7 · 0.9i−1+ 1.71 · 0.92i−1 = v 2 n

∑

i=1 2 − n

∑

i=1 0.1 · 0.9i−1 ! −c 2 n

∑

i=1 2 − n

∑

i=1 3.7 · 0.9i−1 + n

∑

i=1 1.71 · 0.81i−1 ! = v 2 2n − 0.1 1 − 0.9 n 1 − 0.9 −c 2 2n − 3.7 1 − 0.9 n 1 − 0.9 + 1.71 1 − 0.81 n 1 − 0.81 = v 2(2n − 1 + 0.9 n_{) −}c 2(2n − 37 (1 − 0.9 n_{) + 9 (1 − 0.81}n₎₎ = v 2(2n − 1 + 0.9 n_{) −}c 2(2n − 28 + 37 · 0.9 n_{− 9 · 0.81}n_{) .}

(30)

Now to the second part of the expression. 0.9m−1·v 2+ 1 − 0.9m−12 · 0.9m+12 _v₊ 1 − 0.9m−12 1 − 0.9m+12 ·v− c 2 = 0.9m−1·v 2+ 0.9 m+1 2 _v− 0.9m_v₊v 2 1 − 0.9m+12 − 0.9m−12 _{+ 0.9}m −c 2 1 − 0.9m+12 − 0.9 m−1 2 _{+ 0.9}m = v 2 1 − 0.9m+ 0.9m−1+ 0.9m+12 − 0.9 m−1 2 −c 2 1 − 0.9m+12 − 0.9 m−1 2 + 0.9m = v 2 1 + 0.9m−1(−0.9 + 1) − 0.9m−12 (−0.9 + 1) −c 2 1 + 0.9m− 0.9m−12 (1 + 0.9) = v 2 1 + 0.10.9m−1− 0.9m−12 −c 2 1 + 0.9m− 1.9 · 0.9m−12 . Now, putting these together again, we can write the expression for P(t, j) as

P(t, j) = v 2+ v 2(2n − 1 + 0.9 n_{) −} c 2(2n − 28 + 37 · 0.9 n_{− 9 · 0.81}n₎ +v 2 1 + 0.1 0.9m−1− 0.9m−12 −c 2 1 + 0.9m− 1.9 · 0.9m−12 = v 2 1 + 2n + 0.9n+ 0.10.9m−1− 0.9m−12 −c 2 −27 + 2n + 37 · 0.9n− 9 · 0.81n+ 0.9m− 1.9 · 0.9m−12 .

We have now found the payoff for TfT when it plays against Joss, so now we want to find the payoff for Joss when it plays against TfT. Recalling Table 3.4, we can see that the payoff will be P( j,t) = v 2+ 0.9 · v 2+ (1 − 0.9) · 0.9 0_v_{+ (1 − 0.9) 1 − 0.9}0_·v− c 2 + 0.92·v 2+ (1 − 0.9) · 0.9v + (1 − 0.9)(1 − 0.9) · v− c 2 + 0.93·v 2+ 1 − 0.9 2_{· 0.9v + 1 − 0.9}2_{(1 − 0.9) ·}v− c 2 + 0.94·v 2+ 1 − 0.9 2_{· 0.9}2_v_{+ 1 − 0.9}2 1 − 0.92 ·v− c 2 + 0.95·v 2+ 1 − 0.9 3_{· 0.9}2_v_{+ 1 − 0.9}3 1 − 0.92 ·v− c 2 + ...

We see that this is really similar to the payoff for TfT against Joss, so we can go about finding a formula for this payoff in the same way. As above, we define

k= (

0 when m is odd, 1 when m is even,

(31)

and n= (_m−1 2 when m is odd, m−2 2 when m is even,

and the payoff for Joss when it plays against TfT will then be P( j,t) = v 2+ n

∑

i=1 0.92i−1·v 2+ 1 − 0.9 i_{· 0.9}i−1_v_{+ 1 − 0.9}i 1 − 0.9i−1 ·v− c 2 + 0.92i·v 2+ 1 − 0.9 i_{· 0.9}i_v_{+ 1 − 0.9}i 1 − 0.9i ·v− c 2 + 0.9m−1·v 2+ 1 − 0.9m+12 · 0.9m−12 _v₊ 1 − 0.9m+12 1 − 0.9m−12 ·v− c 2 .

(32)

Just as we did before, we will begin by simplifying the sum: n

∑

i=1 0.92i−1·v 2+ 1 − 0.9 i_{· 0.9}i−1_v_{+ 1 − 0.9}i 1 − 0.9i−1 ·v− c 2 + 0.92i·v 2+ 1 − 0.9 i_{· 0.9}i_v_{+ 1 − 0.9}i 1 − 0.9i ·v− c 2 = n

∑

i=1 0.92i−1·v 2+ 0.9

i−1_v_{− 0.9}2i−1_v_{+ 1 − 0.9}i−1_{− 0.9}i_{+ 0.9}2i−1_·v− c

2 + 0.92i·v 2+ 0.9 i_v_{− 0.9}2i_v_{+ 1 − 2 · 0.9}i_{+ 0.9}2i_·v− c 2 = n

∑

i=1 v− 0.9i_·v 2+ 0.9 i−1_·v 2− 1 − 0.9 i_{− 0.9}i−1_{+ 0.9}2i−1_·c 2 − 1 − 2 · 0.9i+ 0.92i ·c 2 = n

∑

i=1 v 2 2 − 0.9 i_{+ 0.9}i−1₋c 2 2 − 3 · 0.9 i_{− 0.9}i−1_{+ 0.9}2i_{+ 0.9}2i−1 = n

∑

i=1 v 2 2 + 0.9 i−1_{(−0.9 + 1) −}c 2 2 − 0.9 i−1_{(3 · 0.9 + 1) + 0.9}2i−2 _0.92_{+ 0.9} = n

∑

i=1 v 2 2 + 0.1 · 0.9 i−1₋c 2 2 − 3.7 · 0.9 i−1_{+ 1.71 · 0.9}2i−2 = v 2 n

∑

i=1 2 + 0.1 · 0.9i−1 −c 2 n

∑

i=1 2 − 3.7 · 0.9i−1+ 1.71 · 0.92i−1 = v 2 n

∑

i=1 2 + n

∑

i=1 0.1 · 0.9i−1 ! −c 2 n

∑

i=1 2 − n

∑

i=1 3.7 · 0.9i−1 + n

∑

i=1 1.71 · 0.81i−1 ! = v 2 2n + 0.1 1 − 0.9 n 1 − 0.9 −c 2 2n − 3.7 1 − 0.9 n 1 − 0.9 + 1.71 1 − 0.81 n 1 − 0.81 = v 2(2n + 1 − 0.9 n_{) −}c 2(2n − 37 (1 − 0.9 n_{) + 9 (1 − 0.81}n₎₎ = v 2(2n + 1 − 0.9 n_{) −}c 2(2n − 28 + 37 · 0.9 n_{− 9 · 0.81}n_{) .}

(33)

Now to the second part of the expression. 0.9m−1·v 2+ 1 − 0.9m+12 · 0.9m−12 _v₊ 1 − 0.9m+12 1 − 0.9m−12 ·v− c 2 = 0.9m−1·v 2+ 0.9 m−1 2 _v− 0.9m_v₊v 2 1 − 0.9m−12 − 0.9m+12 _{+ 0.9}m −c 2 1 − 0.9m−12 − 0.9m+12 _{+ 0.9}m = v 2 1 − 0.9m+ 0.9m−1− 0.9m+12 _{+ 0.9} m−1 2 −c 2 1 − 0.9m+12 − 0.9 m−1 2 + 0.9m = v 2 1 + 0.9m−1(−0.9 + 1) + 0.9m−12 (−0.9 + 1) −c 2 1 + 0.9m− 0.9m−12 (1 + 0.9) = v 2 1 + 0.10.9m−1+ 0.9m−12 −c 2 1 + 0.9m− 1.9 · 0.9m−12 .

We now combine these simplifications to get a simplified formula for the entire payoff for Joss when it plays against TfT.

P( j,t) = v 2+ v 2(2n + 1 − 0.9 n_{) −} c 2(2n − 28 + 37 · 0.9 n_{− 9 · 0.81}n₎ +v 2 1 + 0.1 0.9m−1+ 0.9m−12 −c 2 1 + 0.9m− 1.9 · 0.9m−12 = v 2 3 + 2n − 0.9n+ 0.10.9m−1+ 0.9m−12 −c 2 −27 + 2n + 37 · 0.9n− 9 · 0.81n+ 0.9m− 1.9 · 0.9m−12 . Finally, we can sum up our results in a payoff matrix, Table 3.5.

Now, we want to find first when TfT is an ESS, and then when Joss is an ESS. Using Lemma 1, Table 3.5 gives us that TfT will be an ESS if

mv 2 > v 2 3 + 2n − 0.9n+ 0.1 0.9m−1+ 0.9m−12 −c 2 −27 + 2n + 37 · 0.9n− 9 · 0.81n+ 0.9m− 1.9 · 0.9m−12 . It can also be an ESS if

mv 2 = v 2 3 + 2n − 0.9n+ 0.10.9m−1+ 0.9m−12 −c 2 −27 + 2n + 37 · 0.9n− 9 · 0.81n+ 0.9m− 1.9 · 0.9m−12 and v 2 1 + 2n + 0.9n+ 0.10.9m−1− 0.9m−12 −c 2 −27 + 2n + 37 · 0.9n− 9 · 0.81n+ 0.9m− 1.9 · 0.9m−12 > v 2· m − c 2 m−280 19 + 20 · 0.9 m₋100 19 · 0.81 m .

(34)

TfT Joss TfT m₂v v 2(1 + 2n + 0.9 n −0.10.9m−1− 0.9m−12 −c₂(−27 + 2n + 37 · 0.9n −9 · 0.81n_{+ 0.9}m −1.9 · 0.9m−12 Joss v 2(3 + 2n − 0.9 n +0.10.9m−1+ 0.9m−12 −c 2(−27 + 2n + 37 · 0.9 n −9 · 0.81n_{+ 0.9}m −1.9 · 0.9m−12 v 2· m − c 2 m− 280 19 +20 · 0.9m−100₁₉ · 0.81m

Table 3.5: Payoff matrix for a game between TfT and Joss On the other hand, Joss will be an ESS if

v 2· m − c 2 m−280 19 + 20 · 0.9 m₋100 19 · 0.81 m > v 2 1 + 2n + 0.9n+ 0.10.9m−1− 0.9m−12 −c 2 −27 + 2n + 37 · 0.9n− 9 · 0.81n+ 0.9m− 1.9 · 0.9m−12 . It can also be an ESS if

v 2· m − c 2 m−280 19 + 20 · 0.9 m₋100 19 · 0.81 m = v 2 1 + 2n + 0.9n+ 0.10.9m−1− 0.9m−12 −c 2 −27 + 2n + 37 · 0.9n− 9 · 0.81n+ 0.9m− 1.9 · 0.9m−12 and v 2 3 + 2n − 0.9n+ 0.10.9m−1+ 0.9m−12 −c 2 −27 + 2n + 37 · 0.9n− 9 · 0.81n+ 0.9m− 1.9 · 0.9m−12 > mv 2.

3.1.4 TfT vs. Random

Now, we are going to look at what happens if we have a population of Tit for Tat and Random players. If TfT plays against itself, we already know that the payoff will be mv/2. If Random

(35)

plays against itself, then both players always have a 1/2 probability of cooperating. Note that the probability of them defecting will always be 1 − 1/2 = 1/2. Thus, for each move, there will be a probability of (1/2)2= 1/4 that both players cooperate, the same probability of both defecting, and finally the same probability of the first or the second player being the only one to cooperate. Therefore the expected payoff will be

P(r, r) = m 1 4· v 2+ 1 4· v− c 2 + 1 4v+ 1 4· 0 = mv 2− c 8 .

Further, what will happen when TfT and Random play against each other? On the first move, we know that TfT will cooperate. Random on the other hand will only cooperate with probability 1/2. On the next move, TfT will copy Random’s last move and thus cooperate with probability 1/2, while Random once again will cooperate with probability 1/2. Thus the probability of any combination of defect and cooperate for the two strategies will be 1/2 · 1/2 = 1/4. The exact same thing will happen on the third move, and the fourth move, and all other subsequent moves. Thus, the payoff for TfT will be

P(t, r) = 1 2· v 2+ 1 2· 0 + (m − 1) 1 4· v 2+ 1 4· v− c 2 + 1 4v+ 1 4· 0 = 1 2· v 2+ (m − 1) v 2− c 8 = c 8− v 4+ m v 2− c 8 . The payoff for Random will instead be

P(r,t) = 1 2· v 2+ 1 2v+ (m − 1) 1 4· v 2+ 1 4· v− c 2 + 1 4v+ 1 4· 0 = 3v 4 + (m − 1) v 2− c 8c = c 8+ v 4+ m v 2− c 8 .

We can now create the payoff matrix for a game between TfT and Random, Table 3.6.

TfT Random TfT m₂v c 8− v 4+ m v 2− c 8 Random c 8+ v 4+ m v 2− c 8 _m v 2− c 8

(36)

We now want to find when TfT and Random are ESSs. We begin with TfT. Using Lemma 1, we see that it will be an ESS if

mv 2 > c 8+ v 4+ m v 2− c 8 m> c 4v+ 1 2+ m 1 − c 4v m− m1 − c 4v > c 4v+ 1 2 m 1 − 1 − c 4v > c 4v+ 1 2 m> c 4v+ 1 2 c 4v m> 1 +2v c . It will also be an ESS if

mv 2 = c 8+ v 4+ m v 2− c 8 m= 1 +2v c and c 8− v 4+ m v 2− c 8 > mv 2− c 8 c 8− v 4 > 0 c> 2v. This means that TfT will be an ESS if

m≥ 1 +2v c and, in case of equality,

c> 2v. Random on the other hand will be an ESS if

m v 2− c 8 > c 8− v 4+ m v 2− c 8 0 > c 8− v 4 2v > c. It will also be an ESS if

mv 2− c 8 = c 8− v 4+ m v 2− c 8 0 = c 8− v 4 2v = c

(37)

and c 8+ v 4+ m v 2− c 8 > mv 2 m< 1 +2v c . Since we already know that 2v = c, this will be

m< 1 +2v c = 1 +

c

c= 1 + 1 = 2, and since m > 0 and m is an integer, this means that m = 1.

That is, Random will be an ESS against TfT if 2v ≥ c and, in case of equality,

m= 1.

3.1.5 Results

We have now found the conditions under which TfT will be an ESS when playing against the other strategies, as well as the conditions under which each of the other strategies will be an ESS when playing against TfT. We sum up our results in Table 3.7.

All C All D Joss Random

TfT is ESS Never m≥ v c+ 1, if equality then also v < c ? m≥ 1 + 2v c, if equality then also 2v < c ESS against TfT Never v≥ c, if equality

then also m = 1

?? 2v ≥ c, if equality then also m = 1 Table 3.7: Results for when each strategy will be an ESS

? TfT will be an ESS against Joss if mv 2 ≥ v 2 3 + 2n − 0.9n+ 0.10.9m−1+ 0.9m−12 −c 2 −27 + 2n + 37 · 0.9n− 9 · 0.81n+ 0.9m− 1.9 · 0.9m−12 , where, if we have an equality, the following also has to hold:

v 2 1 + 2n + 0.9n+ 0.10.9m−1− 0.9m−12 −c 2 −27 + 2n + 37 · 0.9n− 9 · 0.81n+ 0.9m− 1.9 · 0.9m−12 > v 2· m − c 2 m−280 19 + 20 · 0.9 m₋100 19 · 0.81 m .

(38)

?? Joss will be an ESS against TfT if v 2· m − c 2 m−280 19 + 20 · 0.9 m₋100 19 · 0.81 m ≥ v 2 1 + 2n + 0.9n+ 0.1 0.9m−1− 0.9m−12 −c 2 −27 + 2n + 37 · 0.9n− 9 · 0.81n+ 0.9m− 1.9 · 0.9m−12

where, if we have an equality, the following also has to hold: v 2 3 + 2n − 0.9n+ 0.10.9m−1+ 0.9m−12 −c 2 −27 + 2n + 37 · 0.9n− 9 · 0.81n+ 0.9m− 1.9 · 0.9m−12 > mv 2.

We remember that when TfT is an ESS, the other strategy cannot invade it, and if the other strategy is an ESS then TfT cannot invade it. We see that TfT and All C will both always be able to invade each other. The other strategies are a bit more complicated.

We know that TfT will be an ESS against All D if m > v/c + 1. Now, we know that m>v

c+ 1 > 1, since v > 0 and c > 0. If we let v < c, then

v c+ 1 <

c

c+ 1 = 1 + 1 = 2.

This means that m will be greater than a formula that is greater than 1 and less than 2. Since m is an integer we only care about its smallest possible integer value, so this means that we will have m ≥ 2. We also know that we can have m = v/c + 1 if we have v < c, which we have in this case, but this does not change our result.

If we instead have v = c, then we will have m> v

c+ 1 = c

c+ 1 = 1 + 1 = 2.

That is, if v = c, we will instead need to have m > 2 for TfT to be an ESS against All D. If v> c however, then the greater v is in proportion to c, the greater m will have to be for TfT to be an ESS.

Now, what about when TfT is able to invade All D? Here we have that as long as v < c, All D can never be an ESS and TfT will always be able to invade it. If v = c however, it will be an ESS if m = 1, and if v > c, it will always be an ESS and thus resist invasion.

Combining when TfT will be an ESS against All D with when All D will be an ESS against TfT, we get some interesting results. If the value of the resource is less than the cost of fighting, v< c, each pair of individuals will only have to meet at least twice for TfT to resist invasion from All D, while TfT will always be able to invade All D. If the value of the resource is instead the same as the cost of fighting, v = c, then each pair of individuals will only have to meet

(39)

more than two times for All D not to be able to invade TfT, while TfT will always be able to invade All D as long as each pair of individuals meet more than once. Finally, if the value of the resource is greater than the cost of fighting, v > c, TfT will be able to resist invasion from All D as long as m > v/c + 1, but as we see the greater v is compared to c, the more times each pair of individuals will have to meet for it to be possible. TfT, on the other hand, can never invade All D.

Clearly, the higher the cost of fighting is compared to the value of the resource, the better TfT will do. This makes sense, since All D always defects, and thus will end up fighting (both individuals play defect) both with other All D and TfT individuals, having to take the high cost of fighting, while TfT will cooperate with the other TfT individuals and thus in those cases avoid this cost.

We can now move on to TfT and Random. The condition for when TfT is an ESS against Random is very similar to when it was an ESS against All D; the only difference is that the v term is multiplied by a constant 2. Thus we can do the same analysis as above. We know that if m> 1 + 2v/c, then Random cannot invade TfT. We also know that

m> 1 +2v c > 1, since v > 0 and c > 0. If we let 2v < c, then

1 +2v c < 1 +

c

c= 1 + 1 = 2.

Once again we see that m will be greater than a formula that is greater than 1 and less than 2, so for the same reason as above, we will need to have m ≥ 2 for TfT to be able to resist invasion from Random. Since we have 2v < c we can also have m = 1 + 2v/c, but this will not change our result.

If we instead have 2v = c, then we will have m> 1 +2v

c = 1 + c

c= 1 + 1 = 2.

That is, if 2v = c, we will instead need to have m > 2 for TfT to be an ESS against Random. If 2v > c however, then the greater v is in proportion to c, the greater m will have to be for TfT to be an ESS.

Now, we want to look at when Random will be an ESS against TfT, we again have very similar conditions to when All D will be an ESS against TfT; as before, the difference is only that the v term in this case has been multiplied by a constant 2. We see that if 2v < c, Random can never be an ESS. If 2v = c, it will be an ESS if m = 1. If instead 2v > c, it will always be an ESS.

As we did with All D, we can now combine the results for when TfT and Random each will be an ESS against each other to get some more interesting results. If the value of the resource is less than half the cost of fighting, 2v < c, then each pair of individuals only have to meet each other two or more times for TfT to be able to resist invasion of Random, while TfT will always be able to invade Random. If the value of the resource is equal to half the cost of fighting, 2v = c, each pair of individuals only have to meet each other more than two times for

An Analysis of Tit for Tat in the Hawk-Dove Game

School of Education, Culture and Communication

Division of Mathematics and Physics

BACHELOR’S DEGREE PROJECT IN MATHEMATICS

An Analysis of Tit for Tat in the Hawk-Dove Game

by

Felicia Modin

MAA322 — Examensarbete i matematik för kandidatexamen

DIVISION OF MATHEMATICS AND PHYSICS

School of Education, Culture and Communication

Division of Mathematics and Physics

Acknowledgements

Contents

List of Figures

List of Tables

Introduction

1.1

Background and Research Question

1.2

Literature Review

1.3

Overview of The Thesis

Model

2.1

The Hawk-Dove Game

2.2

Evolutionary Stable Strategy

2.3

Stable Polymorphism

2.4

Finite populations

2.5

Strategies

2.5.1

Tit for Tat

2.5.2

All C

2.5.3

All D

2.5.4

Joss

2.5.5

Random

Mathematical Analysis

3.1

Evolutionary Stable Strategy

3.1.1

TfT vs. All C

3.1.2

TfT vs. All D

3.1.3

TfT vs. Joss

∑

∑

∑

∑

∑

∑

∑

∑

∑

∑

∑

∑

∑

∑

∑

∑

∑

∑

∑

∑

∑

∑

∑

∑

∑

∑

∑

∑