Artificial Life, A Model

(1)

Artificial Life, A Model

MASTERTHESIS INCOMPUTERSCIENCE, COURSECODE: CDT504 ADVISOR& EXAMINER: BARANCÜRÜKLÜ,BARAN.CURUKLU@MDH.SE

WRITTEN BY

J

ONATAN

T

REIJS

jts05002@student.mdh.se

2014

M

ÄLARDALEN

U

NIVERSITY

S

CHOOL OF

I

NNOVATION

, D

ESIGN AND

E

NGINEERING

(2)

Abstract

The model of this thesis simulates a simple artificial eco-system in which evolving and learning agents try to survive by consuming balls of energy and surviving attacks by other agents. The study finds that the model indeed manages to evolve surviving, and in some cases very aggressive, agents. The thesis presents similar conclusions to that of the study of Polyworld by Yaeger [16]; that an evolving population only facilitates a need for complexity set by the world it evolves in and stagnates when the population has reached this level of complexity. If the populations are to evolve further, the world it lives in must first demand a higher level of complexity. Various problems with simulating artificial life are also discussed along with the more specific obstacles of simulating artificial life in Breve and NEST integrated. The physical world of the model is built in the Breve simulation environment and the neural networks are simulated in NEST through integrate-and-fire neurons and spike-timing dependent plasticity synapses.

1. Introduction

Man-made systems that exhibit characteristics of natural living systems has been a popular subject of fiction for a long time but only recently has it been given a scientific field of its own called Artificial Life(A-Life). The aim of this field is to study not only life as we know it but also life as it could be.

Research within A-Life often include models of artificial eco-systems ranging from populations of viruses to simulations of single autonomous agents. Although similar in field of research, the two extremes differ greatly in where lies the topic of research. Some models concentrates on the evolution of the population as a whole, while some are more interested in the learning of the individual or investigates some other function of life-like character. Most if not all models of A-Life focuses on some emergent property; similar to the famous Miller-Urey experiment in 1953, A-Life researchers sets a starting state that satisfies their theory, initiates the simulation and watch what happens. However, while Miller and Urey focused on the origins of life, A-Life researchers tend to focus on what comes after that, once life is created.

The model proposed in this thesis simulates a world populated by agents with neural networks simulated in NEST [5] that are evolving and competing for survival and reproduction. The model is evaluated by examining two sets of data. The first set consists of statistical data regarding the activity of the agents such as the amount of energy consumed or number of other agents attacked during an agents lifespan. The second set consists of the neurological connections of the brains of the agents. These connections will be compared to a set of rules that logically should be present in the brain of a successful agent.

Section 2 will discuss the ethical aspect of this thesis. Section 3 will provide a background to this thesis in the form of a discussion of the previous research within A-Life. Section 4 will provide a short list of some special terminology used in this thesis. Section 5 will describe the model of this thesis complete with a detailed description of the simulation process, the simulated world and the agents within. Section 5 will also provide a explanation of the neural networks of the agents, their genome and how the evolutionary operators modifies the genome. Section 6 will discuss some technical obstacles that were faced during the development of the model. Section 7 will present the results of the thesis and Section 8 will discuss these results.

2. Related Work

The field of A-Life first got its description in 1989 at the first conference held by Cristopher Langton [9]. There are however research done earlier than that which are considered related to A-Life, the most famous being the attempts to create a self-reproducing automata by John Von Neumann.

A big part of life is the ability to reproduce itself, hence self-reproduction has been subject of much research within A-Life. John Von Neumann is often cited as one of the earliest researchers within artificial life in modern age and is recognized for his work on the theory of reproduction with self-reproducing automatons. In 1952-1953 Neumann worked on a massive self-reproducing cellular automata in an attempt to move the logic of life from the physical-mechanical realm into the world of pure logic by building a self-reproducing cellular automata [4]. Neumann was unsuccessful in finding the design for the automata, but other subsequent researchers has built on his work and found simpler cellular automatons that can self-reproduce [10].

An interesting notion is that the models proposed by Neumann were later found to consist of the same basic functions as our real living cells. The fact that Neumann’s automata behaved similar to the principles of molecular genetics has been

(4)

described as supporting the principle assumption made in artificial life, that the material basis of construction of an organism can be separated from its logical form. That is, being alive comes from the form not the material [4].

A much known cellular automaton originally called simply "Life" but now widely known as "The Game of Life" invented by the mathematician John Conway has some interesting emergent properties often much appreciated among A-Life enthu-siasts. The game is played on a 2d matrix consisting of cells that can be in either an "on" or "off" state. The game starts with a user defined initial state where some cells are chosen and turned on by the player and as the game progress, the next state of the game is calculated from the previous state according to two rules:

1. A cell is turned on if three of its neighbors are turned on.

2. A cell remains on if two or three of its neighbors are also on; otherwise it is turned off. [3]

Different initial states evolve into diverse patterns, some that dies fast, some that bloom into elaborate patterns only to die away some states later and some that just keep on growing much like the rises, falls and alterations of real life. The Game of Life is often cited in Artificial Life context as it embodies some of the principles of computation as the more advanced models of artificial life and also has the power of a universal Turing machine [4]. Just recently in fact, a self-replicating pattern was discovered in the Game of Life by Andrew J Wade in 2010 [6].

The model in this thesis resembles greatly the model designed and built by Larry Yeager [16] called Polyworld (PW). Yeager describes the goal of his model in one sentence; "Polyworld attempts to bring together all the principle components of real living systems into a single artificial (man-made) living system". The model of this thesis does not go this far but is kept relatively simple so that any reason for success of any successful agent can be deduced.

The main difference between the model of this thesis and the one of PW is that PW evolves different physical traits for its individuals such as size, speed, color etc. The model of this thesis does not concern itself with evolving the body and the physical characteristics of the individuals but instead concentrates solely on the evolution and learning of the neural networks of the individuals. This creates a distinguishing difference between the model of this thesis and the one of PW, namely, in PW a physically strong agent can theoretically survive while behaving relatively dumb. In the model of this thesis all agents have the same physical body, hence, only the neural connections of an agent decides its survivability.

The thought experiment described by Braitenberg [1] shows how fairly simple connection schemas can exhibit rather complex behaviors. Even the very simplest versions of Braitenberg vehicles exhibits behaviors that can be perceived as intelligent, without any need of memory or a representation of the world it is exploring. This shows the rather interesting notion that for an agent to be successful in an environment, an internal representation of the environment might not be necessary. Instead, a proper match between sensory input and reaction can be sufficient and very successful, at least to a certain degree and complexity of the world. This sets a lower bound for the success of the model of this thesis. For a population in the model of this thesis to be successful and survive, all it has to do is mimic the connection schemas of the simplest versions of braitenberg vehicles.

The most simple Braitenberg vehicles only involve two connections, one from each sensor connected to one of the two motors, but it is enough to make the vehicle move around and find what it is looking for. A simple schema that should be easily found through evolution by the model of this thesis if the number of possible synapse connections and weight strengths would be kept small.

An important ethical aspect of developing accurate ALife models is how simulations of agents that resemble animals can in the long run replace animals in many, especially behavioral, experiments. Thus, there is an obvious positive effect of developing advanced models. Biologically plausible models of nervous systems are also important for replacing painful experiments with computer models.

3. Terminology

This section will describe some of the more specialized terminology used in this thesis.

3.1. Neural Network

A neural network is a net of neurons connected by synapses. The term neural network is used to describe both real-life neural networks, like those of the human brain, and artificial ones like the neural networks used in this thesis. Typically, an artificial neural network is built up through layers of groups of neurons. The first layer that recieve stimuli is commonly called the input layer. All successive layers are termed Hidden Layers except for the last one that is commonly called the Output

(5)

Layer, its name stems from it being the only layer that produces an output to the external environment. A synapse acts as the connections between two neurons and carries the current that a neuron produces to that of the recieving neuron. A synapse in an artficial neural network is typically associated with a weight that modifies the output of the transmitting neuron making it either stronger of weaker, or even shifting its sign, turning a positive output to a negative one and vice versa. The simplest forms of artificial neural networks use the weights of synapses as their only method of adaptation to stimuli, while more sophisticated artificial neural networks includes more parameters, making them more true to their natural correspondent.

3.2. Minicolumn

The minicolumn is a small component of neural organization that has been observed in the cortex of mammals [12]. In reality a minicolumn can consist of up to 120 neurons where each can be connected to each other and also have synapses connected with neurons in other minicolumns. For processing speed purposes the number of neurons in the minicolumns in the model of this thesis has been reduced to five where only two neurons are allowed to act as receiver for synapses from other minicolumns and only one neuron in the minicolumn is allowed to have an external axon connected to other minicolumns. This smaller version of a minicolumn, though able to map fewer patterns than the larger version, should behave like its larger versions in other respects.

3.3. Search-Space

The search-space of an optimization problem is the set of all possible solutions, good or bad, to that problem. The goal of an optimization algorithm like an evolutionary algorithm is to narrow this set down to a subset of more fit solutions.

4. The Model

This section will describe the simulated world and the simulation process. The body, neural networks and genome of the agents will also be described in detail. The model of this thesis consists of a simulated world populated by balls of energy and evolving agents. The agents has puck-shaped bodies with actuators to move around in the world, to kill other agents and to consume energy balls. All agents have one neural network each that consists of 6 minicolumns. Each neural network is stimulated by input from a camera attached to the front of the body of the agent. The neural network in turn stimulates the actuators of the body to make it move around and interact with the world. Each agent has a set of genes that describes the layout of the neural networks making up the genome of the agent. The genome is crossed over and mutated during reproduction to create the genome of the next generation and in that way evolving the population.

4.1. The World

This section will describe the world of the model.

The world is simulated in the Breve environment [8] which is a physical 3D package designed to simulate rigid bodies, movement and cameras that can be used to simulate vision.

The world of the model consists of a flat rectangular ground surrounded by blue walls, see figure 1 for an image of a running simulation. A source of energy is available in the world in the form of red balls of energy. The number of energy balls available in the world is constant as a new ball of energy pops up at a random position whenever a ball is consumed. The agents that are the main subject of this thesis has green puck-shaped bodies and brains consisting of neural networks connected to the actuators of their bodies to allow them to move around and interact with the world. The neural networks also recieve input from the world in the form of visual stimuli. The agents has to maneuver the world in search for energy to restore their ever decaying energy storage. An agent can do this simply by consuming red balls of energy or it can kill other agents in which case the body of the dead agent is replaced by a new ball of energy that can in turn be consumed.

The model is designed so that no direct rules exist that specifies a good agent, the only guides are the indirect rules that the agents has to learn in order to survive. Rules like "energy balls need to be consumed", "energy balls are red", "I cannot go through walls", "walls are blue" etc. Hence, any behaviors that are observed in the simulations are purely emergent, and the product of evolution and learning.

4.2. The Agents

(6)

Figure 1. Snapshot of an active simulation. The vision fields of the agents are displayed in the lower left corner.

4.2.1 Body and Physiology

Each agent consists of a puck shaped green body that has a wheel on the right and left side that it can turn either clockwise or counter-clockwise to move around in the world. The agents can also activate the behaviors Feed and Attack which it needs to learn to utilize in order to survive in the world. At birth an agent is created with 20.000 in energy. The energy of the agent is then affected throughout the simulation by various actions and situations such as energy ball consumption or being attacked. An agent can die either by running out of energy, old age or because space was needed to make room for new offspring. Every action costs energy, even just being alive costs energy, hence an agent can run out of energy just by moving around or standing still. The energy decrease is calculated and subtracted from the current energy level at each iteration by the formula:

D = Sc+ Bc+ (F ∗ Fc) + (A ∗ Ac) + (M ∗ Mc)

where the static energy consumption Sc= 10.1. The brain consumption Bc = (0.0002 * numberof minicolumns2) which

in our model with 6 minicolumns evaluates to 0.0072 for all agents. F = 1 if the agent is feeding, 0 otherwise. The feeding cost Fc = 3. A = 1 if the agent is attacking, 0 otherwise. The attacking cost Ac= 3. M = the amount of movement of the

agent ranging from 0 to 35. The movement cost Mc= 0.05.

The energy consumption constants where chosen so that each agent has to find a ball of energy during its lifetime, if it does not it will die before reproduction and will not be allowed to reproduce. For instance, an agent that never moves, never attacks and never consumes anything will use up 10.1 energy units during each iteration (ignoring the small brain consumption that really only effects larger brains). For an agent to reach reproduction it will have to survive for 2000 iterations. This minimal consumption of 10.1 for 2000 iterations will result in a total energy consumption of 20200 which is 200 more than the start energy of 20000, hence the agent will die before reproduction.

The relatively high constants for feeding, attacking and moving adds a demand of complexity on the agents brains. Spiking on all behaviors at the same time as the agent is moving forward could be a successful survival strategy save for the high cost of moving, feeding and attacking. Therefore the agents has to learn when to feed and when to attack, the trivial solution of doing everything at the same time is excluded. If the energy level of an agent sinks below 0 the agent dies and is removed from the simulation and its body is replaced by a ball of energy. An agent can also die of old age if it reaches the age of 4100 iterations. This means that an agent can reproduce at most two times before it dies of old age, hence the model can be said to contain a form of elitism [15].

The agents has to consume balls of energy to be able to stay alive. To consume a ball of energy an agent has to run into it with the front of its body at the same time as it is performing the behavior Feed. Doing this consumes the ball of energy and adds its energy to the energy of the agent enabling it to stay alive and move around for a while longer. A normal ball of energy when consumed gives the agent an energy boost of 5000 while a ball of energy that was created from the body of a dead agent gives an energy boost of 10.000.

(7)

An agent can kill another agent by running into it with the front of its body and perform the behavior Kill. This will drain all energy from the attacked agent, kill it and replace its body with a ball of energy which can then be consumed.

4.2.2 Neural Networks

This section will describe the neural networks of the agents both on a conceptual level and in terms of objects simulated in NEST. See figure 2 for an overall view of the connections.

The neural networks of the agents are simulated in NEST which is a tool for simulating biologically plausible large networks of neurons. The neural networks of the agents consists of a main neural network that receives input from a layer of sensory cells which simulates a vision field, a sense of hunger and a random value input inspired by the same use of random values in Polyworld [16] to simulate free will. The output from the network is retrieved from 6 dedicated minicolumns that each are connected to motor-cells. All delays between neurons in the network are set to 10 ms with no distinction between the delays of synapses between neurons in the same minicolumn and the delays of synapses between different minicolumns. All neurons in the model are simulated in NEST as integrate-and-fire Neurons, which is a rather simple neuron model that accumulates potential until a set threshold at which time it transmits a spike, resets its potential and after a refractory period starts accumulating potential again. All synapses within a minicolumn are simulated in NEST as static synapses exhibiting no alternating plasticity but remains static throughout the whole simulation. All other synapses between minicolumns and between the sensory layer and minicolumns are simulated in NEST as dependent synapses. Using spike-timing-dependent synapses adds the possibility for synapse weights to change during an agents lifetime, effectively adding a form of hebbian learning to the model. All weights of synapses in the model are chosen within ranges that limits the neurons to spike at a maximum rate of around 35 Hz and all neurons in the model receives random stimulation that results in spontaneous spike rates at a maximum of 5 Hz, this to keep as close a relationship with real biology as possible.

Feed

Attack

Left Wheel

Right Wheel

Energy

Random

Visual Field

Input

Figure 2. Overall connections of the neural networks of the agents.

The cortex layer is where the main processing should be taking place in the neural network of an agent. The cortexes of the agents of this thesis each consists of 6 minicolumns, each minicolumn initially connected to all other minicolumns of the

(8)

same agent. Each minicolumn consists of four excitatory neurons and one inhibitory neuron, all connected to each other. One of the excitatory neurons is the designated input neuron for the minicolumn and all incoming excitatory synapses connects to this neuron. There is also a designated output neuron from which the only external axon connects to other minicolumns. The inhibitory neuron also has incoming connections from other minicolumns. See figure 3 for an explanation of the neural topology of a minicolumn.

Figure 3. Minicolumn Topology. Green color describes an excitatory neuron or synapse, red de-scribes an inhibitory one.

The Sensory Layer stimulates the neural network of an agent with input from the simulated world.

Each agent has a field of vision that spans 100 degrees left to right in front of its body. This vision field consists of three layers of cells, each layer having three sensory cells each. Each layer has cells of only one kind that reacts to a specific color, either red, green or blue, providing the possibility for agents to differentiate between the red energy balls, blue walls and the green color of other agents. Each sensory cell in the vision field takes input from one third of the whole span of the vision field. In effect, this gives the agents one sensory cell each for the directions right, left and in front.

The agents also has a sense of hunger that gets stronger the less energy they have, this is simulated by one sensory cell in the sensory layer. Inspired by the use of random numbers in Polyworld to simulate free will, a random number is also presented to a sensory cell in the sensory layer.

The sensory layer is simulated in NEST as poisson generators connected to parrot neurons that in turn propagates their signal on to the cortex layer. There is one poisson generator connected to one parrot neuron for each sensory cell. During simulation the stimuli is presented to the neural networks by adjusting the rate of the poisson generators. The rate is calculated by the formula:

Rate = Stimuli ∗ M ultiple

where Stimuli is a decimal number between 0 and 1 and Multiple is 800. For the vision sensory cells the stimuli is calculated by measuring the percentage of designated color in the designated field of vision. For the hunger sensory cell the stimuli is calculated by dividing the current energy of the agent by the maximum allowed energy level 200.000. The free will sensory cell stimuli is calculated and updated every tenth iteration by choosing a random number between 0 and 1. For example, a vision sensory cell whose designated field is "right" and color is "red" would produce a fully stimulated response of 1 if a red ball of energy would be positioned to the right of this agents vision field and take up the entire right part of the vision field, a 0 if no red were detected at all and 0.5 if red was detected on 50% of the right part of the vision field.

(9)

The Motorcells and Behaviors provides the agents with the possibility to move around the simulated world and to interact with the objects in it.

Each agent has two wheels, one on either side of its puck shaped body which it can use to move forwards and backwards and turn left or right by rotating the left or right wheel backwards or forwards. It also has two other actions that it can perform, it can consume a ball of energy and it can kill another agent. This amounts to 6 different actions; feed, kill, right wheel forwards, right wheel backwards, left wheel forwards and left wheel backwards.

The behaviors are realized in the simulation as 6 motorcells that each are connected to the output-neuron of 6 different minicolumns, in NEST the motorcells are simulated by spike-detectors. The spike-detectors counts the number of spikes during an iteration and propagates this onwards to its specified behavior. The feed and kill behaviors are binary switches, which means that the behavior is activated whenever the number of detected spikes exceeds the threshold of 10 spikes. The movement behaviors are continuous in the sense that the number of spikes detected correlates directly to how much power is transferred to the specified wheel and direction, more spikes results in more power.

The 4 movement actuators are logically divided into 2 groups, one for each wheel, where each motorcell correlates to either clockwise or counter-clockwise rotation. This in turn means that each motorcell in each group competes with the other in the same group for excitation. Ex: A spike train of 10 spikes on the counter-clockwise motorcell and a spike-train of 5 spikes on the clockwise motorcell will result in a counter-clockwise rotation of 5.

4.3. The Simulation Process

At initiation the simulations populates the world with 20 agents and 20 balls of energy. The initial genome for each agent is created by randomly selecting values within the allowed ranges for each gene. The genome is then decoded to build the neural networks of the agents.

After the initiation, the simulation activates the sensory neurons of the agents which stimulates their neural networks and will eventually activate the muscle-cells of their bodies which will make them move around and interact with the world. The time of the simulation is divided into 250 milliseconds long chunks, or iterations. Within each iteration the sensory neurons are stimulated with their specific input. The neural networks are then activated for 500 milliseconds and the number of spikes by the muscle-cells are read and then propagated on to their specific behavior that allows the agents to move, consume energy and attack each other.

The population is reproduced after 2000 iterations. When reproduction occurs two parents are randomly selected from the pool of currently living agents, their genomes are crossed over and the result mutated in order to create the genome of an offspring. This creation of new offspring is repeated 10 times or more until the population is at its max of 30 agents. If there is not enough room for the new offspring because too many agents has survived, the model makes room by removing the agents with the least amount of energy. This is done in order to keep the number of active agents in the simulation down within a manageable size and is rather regrettable as this indirectly creates a rule that states that agents must always try and keep their energy levels high, and an agent that does not do so can be deemed unfitting and be removed. This kind of rule might discriminate some survival strategies that concentrates on conserving energy and find new energy only when in great need. These kinds of rules should be avoided but were here unavoidable because of limitations in computer power and time. After the creation of the offspring the new agents are added to the population and the simulation is again activated with the new generation. The simulation continues to simulate 150 generations. See algorithm 4.1 for a description of the simulation steps.

Algorithm 4.1: SIMULATIONSTEPS(void) generation1= randomizeN ewP opulation()

for generation ← 0 to 150 do            for iteration ← 0 to 2000 do N est.simulate(500ms) Breve.simulate(250ms) of f spring = Crossover(generationn)

(10)

4.4. Genes and Phenotypes

This section will describe the genome of the agents and discuss its layout.

The genome maps directly to weights and positions of synapses and neurons in the neural networks and is divided into two large groups, one group that is being evolved and one that is not. The part of the genome that does not evolve is the part that describes the inner connections of the minicolumns and the part that is evolved describes the connections between the sensory layer and the cortex layer as well as all connections between minicolumns within the cortex layer. The rationale behind having a non-evolving part is to decrease the search space the evolutionary operators has to maneuver. The spike patterns of real life minicolumns has been the focus of a lot of research [12] [2] and it has been found that the minicolumns in the cortex of mammals exhibit great homogeneity. Therefore, the model of this thesis satisfies with simply creating minicolumns that act similar to real life minicolumns at initiation of the simulation and then passing them on to the next generation untouched. The six non-evolving internal minicolumn groups of connections can therefore be seen as building blocks which the part that is being evolved has to find a fitting connection scheme for. See figure 4 for an explanation of the mappings between genes and neuronal connections.

The non-evolving part describes the inner weights of the 6 minicolumns in the cortex layer. All neurons connects to all other neurons inside the same minicolumn, hence 20 values are needed to describe all the weights in one minicolumn. The non-evolving part of the genome can therefore be divided into 6 smaller lists, each 20 values long. Out of these 20 values the first 16 maps to the synaptic weights of the synapses from the 4 excitatory neurons in the minicolumn and the last 4 are the weights of synapses from the inhibitory neuron. The 4 synapses of a neuron in a minicolumn maps to 4 values in the genome. The 4 values of an excitatory neuron starts with 3 values that ranges between 0 and 550 and maps to the weights between this excitatory neuron and the other 3 excitatory neurons in the same minicolumn. The 4th value of the genome of an excitatory neuron maps to the weight of the synapse connected to the inhibitory neuron and ranges between 0 and 450. The 4 values of an inhibitory neuron maps to the weights of the synapses going from the inhibitory neuron to the excitatory neurons and ranges between 0 and 550.

The specific ranges of weights between different neurons are necessary to force the minicolumns to behave as in real life and to keep their spike-rates within 5-35 Hz. The relatively high weights are necessary because of the low amount of neurons in each minicolumn and the fact that only one neuron gets external stimuli. Had the minicolumns been bigger with more neurons and with more external stimuli, lower ranges could have been used.

The evolving part of the genome describes the synapses between the sensory layer and the cortex layer as well as the synapses between the minicolumns within the cortex layer. Because of the synapses going from each sensory neuron to minicolumns and each minicolumn to other minicolumns does not have an all to all connection relationship, this part of the genome has to contain some sort of pointer to which minicolumn that should receive the different synapses. This is achieved by adding an index gene to the genome that describes which minicolumn will be the receiver at the other end of the synapse. The target node index together with the weight, which here can be either positive or negative, is sufficient to describe an initial starting point for a synapse, its weight, the receiving minicolumn and depending on the sign of the weight whether the synapse should be connected to the excitatory input neuron or the inhibitory neuron of the minicolumn. The maximum weight of these synapses has been constrained to 100.

The layout of the genome considers the notion of schemas [13], or the idea that the main advantage of the crossover operation is ability to combine blocks of functioning genome together to form an even better genome. For this to work adjacent parts of the genome has to map to functions in the resulting phenotype that work together. For instance, in the model of this thesis, genome subparts that specifies connections for minicolumns that interact with each other should be adjacent to each other. Likewise, genome that specifies connections from the sensory layer to the same minicolumn should be adjacent to each other. In the model of this thesis all minicolumns are initially connected to all other minicolumns. Hence a specific ordering of the genome based on these connections would give no benefit, the only ordering that can be inferred from the connections of the minicolumns and the minicolumns themselves are the functional properties of the behaviors of the agents themselves. As mentioned earlier a relationship between the clockwise and counterclockwise rotations of each wheel exists in that they can effectively diminish each others activation by spiking at the same time, hence the genome for these connections should be placed adjacent to each other. The same thing can be said regarding the genome for the connections of movement as a whole. An agent might have evolved to be very good at moving around and running into balls of energy but not be as good at consuming that energy, hence the genome for the connections of movement as a whole and the connections for the feeding and attacking behaviors should be separate to allow the replacement of the feeding and attacking behaviors while keeping the movement genome intact. The genome that specifies the connections from the sensory layer to minicolumns however are all grouped together based on which minicolumn they interact with, creating a theoretic block of genome that

(11)

can be inherited in full as a functioning block. ! " ! # ! $ ! % ! & ! !

'

(

Figure 4. Genome Mapping

4.5. Evolution

This section will describe the evolutionary operators implemented in the model.

Reproduction in the model occurs at each 2000th iteration. At this time all living agents are put in a pool and randomly chosen for reproduction. The selection at this time is truly pseudo-random and no fitness values of any kinds are used. Using this kind of selection instead of a more commonly used version within Artificial Intelligence such as roulette wheel selection or tournament selection [11] might make it harder for the the evolutionary algorithm to find good solutions. However, by using a completely pseudo-random algorithm the model avoids the possibility of stating determined rules about what is good and what is bad behaviors and leaves the result and possible behaviors open for exploration by the algorithms. Had a different algorithm been used, for instance one that would incorporate how much activity that had occurred during an agents life and base the selection on that, then some emergent survivor strategy might have been ruled out, perhaps a strategy that conserves energy by staying still.

The reproduction starts by choosing two parents and using crossover produces an offspring genome. The offspring genome is then mutated to create the finished genome of the offspring. The evolution in the model of this thesis uses 2 two point crossover operators and single point mutations. One of the crossover operations occurs in the part of the genome which maps to the internal connections of the minicolumns, choosing which minicolumns that are to be used by the offspring. This crossover never divides the genome of a minicolum but only chooses which whole minicolumn genome that should be taken from which parent. Note that this is the only kind of evolution that is active on this part of the genome, no alterations of the actual connections are performed.

The other crossover operation occurs in the part of the genome that maps to the external connections between the sensory layer and the cortex layer as well as the connections between minicolumns within the cortex layer. As the genome of a

(12)

connection between two neurons in this part of the genome are divided into two parts, one describing the target of the synapse and one describing its weight, in theory this crossover operator can separate the genome of a connection in two taking the weight from one parent and the target node from the other. This part of the genome is also operated on by the mutation operator. Every gene is mutated with a probability of 0.005. A successful mutation removes the old gene and inserts a new random gene. Depending on what alleles are possible for that gene either a synaptic weight or an index which points to a synapse target is randomized within the allowed ranges. There are no constraints on the mutation when mutating the target of a connection. Which means that the mutation operation can produce double connections from one minicolumn to another. Two connections with the same receiver and transmitter will have the same effect as one connection with the weight of the two connections summarized, in effect possibly creating a connection with a weight higher than the max of +/-100 of a single connection.

4.6. Learning

This section will describe the hebbian learning used in the model of this thesis.

Hebbian learning is implemented in the model by using spike-timing dependent plasticity(STDP) synapses as connections between minicolumns and as connections from the input layer to the minicolumns. The neural networks in the model of this thesis is implemented in NEST through integrate-and-fire neurons connected by STDP synapses. Together with other forms of synaptic plasticity STDP is believed to underline learning and information storage in the brain and functions by adjusting the weights of synapses depending on pre- and post-synaptic spikes. Repeated pre-synaptic spike-trains a few milliseconds before post-synaptic spikes increases the synaptic weight, effectively leading to long-term potentiation of synapses. The opposite is achieved by decreasing the synaptic weight if repeated pre-synaptic spikes arrive shortly after post-synaptic activation, leading to long-term depression. Through this associative way, hebbian learning creates engrams, groups of neurons that are strongly connected and interacting by strengthening patterns that are already there and removing or weakening noise [14]. The STDP model used in NEST is described in detail in [7].

5 Technical Obstacles

This section will discuss the technical obstacles that comes with using NEST and Breve in a model that consists of many small brains that after some time should be considered as dead and removed from the simulation as well as any other technical challenges met during the design and implementation of the model of this thesis.

At the time of building this model NEST did not have any methods for removing already laid out neural networks. Because this model creates many agents with neural networks that only live for a short period of time and are then considered as dead and should be removed, this poses a problem. The dead brains has to be removed or else they will continue to be simulated, although not connected to anything, and consume processing power. In this model this was solved by creating a snapshot of the neural networks of all living agents and then resetting nest right before each reproduction. The neural networks of all living agents are then rebuilt according to the snapshot which results in a new clean nest environment with the dead brains removed. This has the rather regrettable side-effect of resetting all action potentials that were present at the time as there are no way of modifying such parameters in the NEST neural model without implementing changes in the NEST source code.

The author of this thesis was unable to remove a bug in the software in the model that seems to stem from the Breve environment and is possibly caused by a misuse of the Breve API. The bug causes a crash and shuts down the simulation. The solution to this problem that was used in this model was to restart from the last reproduction if such a crash occurred.

6. Results

Three large simulations are examined in this thesis, all three were executed with the same parameters. The only thing that differs them apart is the randomized values used when creating the initial genomes and when crossing over and mutating the genomes. The results of the model and its simulations can be examined by looking at two different sets of data. The first set consists of statistical information recorded during the simulations depicting the mean and max energy ball consumption, attacks and number of reproductions reached in each generation. The second set consists of the neurological connections that has been evolved in the agents brains.

(13)

6.1. Statistical Data

This section will present the statistical data recorded during each simulation.

The data concerning energy ball consumptions in figures 5, 6 and 7 shows that each population evolves towards a higher efficiency at finding and consuming energy. Population 3 manages to produce the most efficient agents with a maximum of 14 balls of energy consumed by a single agent compared to the maximum 8 of population 1 and 2. The curves are fairly edgy this is likely due to the world we are simulating. As each agent is born at a random place in the world, it might be born either close or far away from the nearest ball of energy. Therefore, the problem faced by each agent when it comes to survival differs depending on where in the world it is born and on the state of the world at that time. An agent could in effect have a perfectly functioning brain when it comes to finding energy in the closest vicinity but lack the brain power to find balls of energy that are far away. The energy consumption of such an agent would differ greatly depending on where in the world it is born producing either a spike or fall in our curve. The difference in mean and max number of consumptions in each population suggests that we have a healthy difference in the genes of the agents with a lot of diversity helping us explore all possible solutions of the search space of energy consumption efficiency, the previously mentioned fact regarding an agents birthplace might of course also affect this.

Figure 5. Population 1: The max and mean number of balls of energy consumed in each generation.

Figure 6. Population 2: The max and mean number of balls of energy consumed in each generation.

(14)

The data concerning attacks in figures 8, 9 and 10 shows that both population 1 and 2 had very few attacks throughout their whole simulations. Population 1 had an agent in the first generation that killed 3 other agents but the next generation had zero attacks suggesting that those genes did not make it through reproduction and were lost. Population 3 however managed to evolve a very aggressive species with as many as 7 kills by one agent. The number of attacks in population 3 seems to have stabilized at a mean value of around 0.6 with a slight increase to the end of the simulation. The max number of attacks stays at around 3 - 4 attacks with a few exceptions of around 6 or 7 attacks.

Figure 8. Population 1: The max number of attacks made by the most aggressive agent in each generation and the mean number of attacks in each generation.

The data concerning reproductions in figures 11, 12 and 13 depicts the number of agents in each generation that reached the first and second reproduction opportunities of their generation. An agent is allowed to reproduce at most two times during its lifetime, the first reproduction opportunity is at 2000 iterations and the second one at 4000. The chart for population 1 shows that it started off really bad with only one agent surviving to the first reproduction opportunity in generation 2. This

(15)

means that this agent is the sole ancestor of all agents after that, hence the randomness in the population genome and the much needed diversity that is so important for a good evolution is lost. Hence from population 1 we should expect great homogeneity in its genome material. In fact, the only diversity of the genome of the whole population will come only from mutations thereafter. Population 2 suffered a similar fate at both generation 2 and 4 before it found a winning concept and could produce more capable and surviving agents.

Population 3 did not suffer such a fate but managed in its initial random creation of agents to create two agents that survived to the first reproduction opportunity. This is a much better start then what the other populations suffered giving population 3 twice the startup genome to combine agents into. An interesting thing to note here is that simulation 3 fluctuate between 6 to 10 agents that reach reproduction 1 while the other populations balances out at higher values. Population 1 and 2 fluctuates at about 6 or 8 to as high as 14, even through population 3 had higher values in earlier generations.

The number of agents that reached the second reproduction opportunity is far fewer than the number of agents that reached reproduction opportunity 1. This is of course because for an agent to reach the second reproduction opportunity it has to survive twice as long and compete against its own offspring for survival. All three populations seems to have settled at about the same number of agents that reach the second reproduction opportunity.

Figure 11. Amount of agents of each generation that reached reproduction opportunity 1 and 2 in population 1.

The data of consumptions and attacks in relation to the age of all agents in a population is presented in figures 14, 15. There is a noticeable difference between the curves of population 1, 2 and population 3. The curves of population 1 and 2 stays fairly high until around age 2000 where they plummet to about half their former levels, while the curve for population 3 starts off strong and slowly decreases in a close to linear fashion. The plummeting of population 1 and 2 at iteration 2000 is probably because at this time a new generation is spawned and room has to be made in the pool of currently alive agents, hence some agents are killed. One can see a similar decrease in population 3 but it is not as extreme as the decrease in population 1 and 2. This is most probably because population 3 was so aggressive that it killed enough or near enough agents before reproduction, giving room to the new offspring on its own without any extra help. The slow decrease of population 3 most probably also has to do with the populations high aggressiveness, as some agents are killed less agents are left alive to search for energy and hence a lower curve at the higher ages.

(16)

Both population 1 and 2 had near to no attacks so their curves are of course very low here too. Population 3 has a lot of attacks though and its attack curve looks similar to its corresponding consumption curve suggesting that also here the number of attacks is related to the number of agents still alive in the generation. As more agents are killed, less agents are left to hunt and so the curve goes down. Further supporting this theory is the increase at age 2000 when the new offspring is added to the simulation and the amount of alive agents is replenished once again, adding more agents to pray on.

No increase in attacks or consumptions at the higher ages can be seen though, as could have been expected as a result of the hebbian learning. This could be an indication that the hebbian learning did not have a positive effect on either one of the attack or consumptions behaviors, or that the learning takes place in the early years of the life of an agent and balances out leaving no room for improvement in the later years.

Figure 14. Consumptions per age of all agents in each population.

Figure 15. Attacks per age of all agents in each population.

6.2. Data Concerning Neurological Connections

This section will present the results from the simulations regarding the connections in the brains of the agents. A number of behavioral rules that should have been learned by a surviving agent will be presented and compared to the actual connections of the agents to find to what extent they have been discovered and learned in the different populations. The data shown in

(17)

tables 16 and 17 are the neurological connections at an agents birth before any hebbian learning modified it. The reason for this is to see to what extent evolution was able to find the necessary connections on its own. We do not reason about temporal data such as spike timings or whether one minicolumn fires before the other, but only about whether a connection is there or not and how strong weight it has.

If an agent has evolved and survives in the world, then this agent should have learned the following set of rules, or at least a subset of them, and therefore have the corresponding neurological connections implied by the rules. The rules can be divided into two groups depending on which behavior they contribute to, either feeding or attacking. Even though the model simulates fairly small brains, the different possible connection patterns within them are many, hence to make the reasoning manageable we will only look at the more direct routes. For a rule to be considered as learned by an agent, one version of a pattern of connections must be present in its brain. Different versions of connection patterns for the same rule must be considered as a number of different patterns are possible and might produce similar behavior. In the following lists, the symbol e> depicts an excitatory connection.

Consumption rules are about finding balls of energy and consuming them. The first rule describes what an agent should do when it detects a ball of energy in front of it. If an agent has learned this rule we should see at least one of the four versions of connection patterns in its brain. Rules 2 and 3 describes what an agent should do when a ball of energy is detected on the left or right side of its vision field.

Rule 1: Whenever an agent sees a ball of energy in front of it, it should go forward and feed. Version A:

• Red In Front e> Right Wheel Forwards. • Red In Front e> Left Wheel Forwards. • Red In Front e> Feed.

Version B:

• Red In Front e> Right Wheel Forwards. • Red In Front e> Left Wheel Forwards. • Left Wheel Forwards e> Feed. Version C:

• Red In Front e> Right Wheel Forwards. • Red In Front e> Left Wheel Forwards. • Right Wheel Forwards e> Feed. Version D:

• Red In Front e> Feed.

• Feed e> Left Wheel Forwards. • Feed e> Right Wheel Forwards.

Rule 2: Whenever an agent sees a ball of energy on the right, it should go right and feed. • Red Right e> Left Wheel Forwards.

• Red Right e> Feed.

Rule 3: Whenever an agent sees a ball of energy on the left, it should go left and feed. • Red Left e> Right Wheel Forwards.

• Red Left e> Feed.

From tables 16 and 17 we can see that all populations managed to learn some of the rules.

Rule 1 was learned by both populations 1 and 2. Population 1 learned the rule through its B version and almost managed to learn version A and C too. Population 3 managed to learn three different versions of the rule through versions A, B and C and almost learned version D too. Population 2 did not manage to learn rule 1 through any version.

(18)

Rule 2 was learned by all three populations, most strongly by population 2. Rule 3 wasn’t learned by any population.

Attacking rules are about finding other agents, attacking them, eventually killing them and in the process create a new ball of energy that can later be consumed. The first rule describes what an agent should do when it detects another agent in front of it. If an agent has learned this rule we should see at least one of the four versions of connection patterns in its neural network. Rules 2 and 3 describes what an agent should do when another agent is detected on the left or right side of its vision field.

Rule 1: Whenever an agent sees another agent in front of it, it should go forward and attack. Version A:

• Green In Front e> Right Wheel Forwards. • Green In Front e> Left Wheel Forwards. • Green In Front e> Attack.

Version B:

• Green In Front e> Right Wheel Forwards. • Green In Front e> Left Wheel Forwards. • Left Wheel Forwards e> Attack. Version C:

• Green In Front e> Right Wheel Forwards. • Green In Front e> Left Wheel Forwards. • Right Wheel Forwards e> Attack. Version D:

• Green In Front e> Attack. • Attack e> Left Wheel Forwards. • Attack e> Right Wheel Forwards.

Rule 2: Whenever an agent sees another agent on the right, it should go right and attack. • Green Right e> Left Wheel Forwards.

• Green Right e> Attack.

Rule 3: Whenever an agent sees another agent on the left, it should go left and attack. • Green Left e> Right Wheel Forwards.

• Green Left e> Attack.

From tables 18 and 19 we can see that no population managed to learn any of the direct attacking rules we were looking for.

7 Conclusion

This section will discuss the two result-sets of the simulations and present the conclusion of the study.

Both the statistical data and the neurological data suggests that population 3 managed to create the best agents in respect to the amount of energy consumed. The statistical data for population 3 suggests that the population should have very strong connections related to aggressive behaviours while there are no indications of this in the neurological data. This conflict suggests that the most favorable neurological pattern, or at least the pattern most easily found by evolutionary operations, for a specific behavior is not always the most direct one.

None of the attack rules we were looking for found a match even though the statistical data clearly states that the agents in population 3 developed a very aggressive behavior. This would suggest that the population managed to find a less direct

(19)

route than the ones that were examined in this thesis. The consumptions data however did match both in the statistical and neurological data so the assumed rules and paths were accurate in some respects.

The results suggests further that although each population managed to evolve surviving agents, population 1 and 2 stag-nates at a certain level of efficiency and complexity. Similar to the findings of Yeager in the study of Polyworld, the agents simply no longer needs to evolve beyond a certain threshold to survive in the world. Population 3 did not seem to stagnate before the end of the simulation and was also the simulation that managed to produce the most consuming agents and the most most aggressive ones. In a way, that simulation managed to raise the bar by itself, increasing the difficulty of the world and therefore the evolution did not stagnate but kept going to further heights. This also tells us that it is in fact the world and the complexity and difficulty of it that sets the bar for how fit the agents in the simulation will become, not solely the complexity of the agents themselves. Of course the agents must also have the means to adapt to a more complex environment. Had the simulation incorporated an increase in difficulty there would definitively come a time when the world would become too complex for the simple construct of 6 minicolumns, resulting in the death of the whole population. It would seem that in this theoretical world as in the real world, the need for greater performance comes first and the facilitation to that need comes second, and for a model to be ever better performing it would have to consist of an ever changing and ever more complex world and agents with the possibility to adapt to that world.

Population 1 and 2 had great difficulty in finding a good start to evolve from, in the first generations all agents except for one died out in both simulations, diminishing the genome diversity to that of just one ancestor. This demonstrates a major problem that must be dealt with when building these kinds of artificial life models. The model should be designed so that any initial agents that are created randomly have a chance to survive to the point of reproduction but should also contain enough complexity to put a strain on the population and force it to evolve beyond this first threshold of barely surviving. The problem stems from the fact that these kinds of simulations jumps straight into the middle of the evolutionary ladder and initially try to randomly create semi-surviving agents. The alternative would be to start from the beginning with just one functioning cell, or even earlier that the development of the cell, and evolve agents from there on, leaving everything to evolution.

8 Future Research

A closer comparison between the new born brain and the same brain at the moment of death should reveal interesting aspects of the hebbian learning and how it affects the brain of a living agent during its lifetime. Furthermore, a similar study but within an environment with an ever increasing difficulty could create a demand for more effective agents that exhibits interesting behaviours.

9 Acknowledgments

I thank you Baran Cürüklu for helping me through these three years that the writing of this thesis needed, even though the mentoring time surely normally spans just for about half a year. I also thank you Annea Barkefors for your comments on the language and content of this thesis.

(20)

Figure 16. Connection strengths of the most feeding agent from each population and the difference between them. Green indicates a strong excitatory connection, red a strong inhibitory connection and blue a big difference between two agents.

P 1 P 2 P 3 Difference of P1 and P2 Difference of P1 and P3 Difference of P2 and P3

LWF to LWF 0 58 0 58 0 58 LWF to LWB -94 -94 66 0 160 160 LWF to RWF 100 64 117 36 17 53 LWF to RWB -51 0 0 51 51 0 LWF to Attack -30 -55 -24 25 6 31 LWF to Feed 19 0 0 19 19 0 LWB to LWF -45 84 -41 129 4 125 LWB to LWB 0 0 0 0 0 0 LWB to RWF 58 45 0 13 58 45 LWB to RWB 49 44 -68 5 117 112 LWB to Attack 3 25 81 22 78 56 LWB to Feed 0 0 0 0 0 0 RWF to LWF 14 72 0 58 14 72 RWF to LWB 1 -8 0 9 1 8 RWF to RWF 74 0 0 74 74 0 RWF to RWB -78 46 1 124 79 45 RWF to Attack 80 37 116 43 36 79 RWF to Feed 0 0 45 0 45 45 RWB to LWF -96 0 3 96 99 3 RWB to LWB 19 54 51 35 32 3 RWB to RWF 2 -42 49 44 47 91 RWB to RWB 0 75 51 75 51 24 RWB to Attack 0 75 0 75 0 75 RWB to Feed 38 0 0 38 38 0 Attack to LWF 73 89 0 16 73 89 Attack to LWB 7 0 128 7 121 128 Attack to RWF 0 129 94 129 94 35 Attack to RWB 61 77 -35 16 96 112 Attack to Attack 0 0 0 0 0 0 Attack to Feed 52 94 60 42 8 34 Feed to LWF 8 37 92 29 84 55 Feed to LWB 91 112 0 21 91 112 Feed to RWF 37 -38 31 75 6 69 Feed to RWB -84 91 51 175 135 40 Feed to Attack 0 0 -6 0 6 6 Feed to Feed 0 0 0 0 0 0 RedLeft to LWF 0 0 0 0 0 0 RedMiddle to LWF 49 0 28 49 21 28 RedRight to LWF 45 61 25 16 20 36 GreenLeft to LWF 0 -92 0 92 0 92 GreenMiddle to LWF 0 0 0 0 0 0 GreenRight to LWF 0 0 0 0 0 0 BlueLeft to LWF 0 0 0 0 0 0 BlueMiddle to LWF 0 -16 0 16 0 16 BlueRight to LWF 0 0 0 0 0 0 Energy to LWF 0 0 0 0 0 0 Random to LWF 0 0 12 0 12 12 RedLeft to LWB 0 0 75 0 75 75 RedMiddle to LWB 87 0 0 87 87 0 RedRight to LWB -69 -44 0 25 69 44 GreenLeft to LWB 0 0 0 0 0 0 GreenMiddle to LWB 0 -1 0 1 0 1

(21)

Figure 17. Connection strengths of the most feeding agent from each population and the difference between them. Green indicates a strong excitatory connection, red a strong inhibitory connection and blue a big differentce between two agents (Continued).

GreenRight to LWB 0 0 70 0 70 70 BlueLeft to LWB 0 0 0 0 0 0 BlueMiddle to LWB 0 0 0 0 0 0 BlueRight to LWB 0 0 0 0 0 0 Energy to LWB 0 0 0 0 0 0 Random to LWB 0 0 0 0 0 0 RedLeft to RWF 0 0 0 0 0 0 RedMiddle to RWF 64 0 18 64 46 18 RedRight to RWF 0 0 0 0 0 0 GreenLeft to RWF 0 59 0 59 0 59 GreenMiddle to RWF 0 0 0 0 0 0 GreenRight to RWF 0 0 0 0 0 0 BlueLeft to RWF 0 0 -85 0 85 85 BlueMiddle to RWF -89 8 57 97 146 49 BlueRight to RWF 0 0 0 0 0 0 Energy to RWF 0 0 0 0 0 0 Random to RWF 73 0 0 73 73 0 RedLeft to RWB 0 0 0 0 0 0 RedMiddle to RWB -24 -9 0 15 24 9 RedRight to RWB -79 -33 -41 46 38 8 GreenLeft to RWB 0 60 0 60 0 60 GreenMiddle to RWB 0 0 0 0 0 0 GreenRight to RWB 0 0 20 0 20 20 BlueLeft to RWB 0 0 0 0 0 0 BlueMiddle to RWB 0 0 27 0 27 27 BlueRight to RWB 0 0 0 0 0 0 Energy to RWB 0 0 0 0 0 0 Random to RWB 95 0 0 95 95 0 RedLeft to Attack 46 0 0 46 46 0 RedMiddle to Attack -34 0 0 34 34 0 RedRight to Attack 0 44 0 44 0 44 GreenLeft to Attack 0 0 0 0 0 0 GreenMiddle to Attack 0 0 0 0 0 0 GreenRight to Attack 0 0 -30 0 30 30 BlueLeft to Attack 0 -40 28 40 28 68 BlueMiddle to Attack 58 0 0 58 58 0 BlueRight to Attack 0 13 0 13 0 13 Energy to Attack 0 0 0 0 0 0 Random to Attack 0 0 40 0 40 40 RedLeft to Feed 0 0 0 0 0 0 RedMiddle to Feed 0 0 78 0 78 78 RedRight to Feed 37 82 29 45 8 53 GreenLeft to Feed 0 0 56 0 56 56 GreenMiddle to Feed 35 -10 0 45 35 10 GreenRight to Feed 0 0 0 0 0 0 BlueLeft to Feed 0 0 0 0 0 0 BlueMiddle to Feed 0 0 0 0 0 0 BlueRight to Feed 0 0 0 0 0 0 Energy to Feed 19 0 0 19 19 0 Random to Feed 0 0 0 0 0 0

(22)

Figure 18. Connection strengths of the most aggressive agent from each population and the dif-ference between them. Green indicates a strong excitatory connection, red a strong inhibitory connection and blue a big differentce between two agents.

LWF to LWF 0 0 0 0 0 0 LWF to LWB -94 29 66 123 160 37 LWF to RWF 100 1 -58 99 158 59 LWF to RWB -51 -11 58 40 109 69 LWF to Attack 63 27 -54 36 117 81 LWF to Feed 19 -62 0 81 19 62 LWB to LWF 77 69 0 8 77 69 LWB to LWB 0 0 0 0 0 0 LWB to RWF 46 28 0 18 46 28 LWB to RWB 49 59 -68 10 117 127 LWB to Attack 3 1 91 2 88 90 LWB to Feed -46 88 0 134 46 88 RWF to LWF -69 -37 0 32 69 37 RWF to LWB -47 29 0 76 47 29 RWF to RWF 0 0 0 0 0 0 RWF to RWB 46 99 1 53 45 98 RWF to Attack -76 56 150 132 226 94 RWF to Feed -1 27 45 28 46 18 RWB to LWF 95 73 89 22 6 16 RWB to LWB -3 34 51 37 54 17 RWB to RWF 8 -56 70 64 62 126 RWB to RWB 0 0 51 0 51 51 RWB to Attack -40 65 0 105 40 65 RWB to Feed 14 -19 -12 33 26 7 Attack to LWF 85 85 1 0 84 84 Attack to LWB 11 -44 99 55 88 143 Attack to RWF -46 52 0 98 46 52 Attack to RWB 83 89 -35 6 118 124 Attack to Attack 0 0 0 0 0 0 Attack to Feed 37 57 60 20 23 3 Feed to LWF 39 -67 92 106 53 159 Feed to LWB 81 61 0 20 81 61 Feed to RWF -14 29 31 43 45 2 Feed to RWB -5 83 0 88 5 83 Feed to Attack 27 90 108 63 81 18 Feed to Feed 0 0 51 0 51 51 RedLeft to LWF 0 0 0 0 0 0 RedMiddle to LWF 13 0 28 13 15 28 RedRight to LWF -32 -74 0 42 32 74 GreenLeft to LWF 0 0 0 0 0 0 GreenMiddle to LWF 0 0 0 0 0 0 GreenRight to LWF 43 0 25 43 18 25 BlueLeft to LWF 0 0 0 0 0 0 BlueMiddle to LWF 0 0 0 0 0 0 BlueRight to LWF 0 -70 0 70 0 70 Energy to LWF 0 7 0 7 0 7 Random to LWF 0 0 12 0 12 12 RedLeft to LWB 0 0 75 0 75 75 RedMiddle to LWB 17 0 0 17 17 0 RedRight to LWB -69 26 0 95 69 26 GreenLeft to LWB 0 0 0 0 0 0 GreenMiddle to LWB 0 35 0 35 0 35

(23)

Figure 19. Connection strengths of the most aggressive agent from each population and the dif-ference between them. Green indicates a strong excitatory connection, red a strong inhibitory connection and blue a big differentce between two agents (Continued).

GreenRight to LWB 0 0 70 0 70 70 BlueLeft to LWB 0 0 0 0 0 0 BlueMiddle to LWB 0 0 0 0 0 0 BlueRight to LWB 0 25 0 25 0 25 Energy to LWB 80 0 0 80 80 0 Random to LWB 0 0 0 0 0 0 RedLeft to RWF -69 50 0 119 69 50 RedMiddle to RWF 0 0 18 0 18 18 RedRight to RWF 0 65 0 65 0 65 GreenLeft to RWF 0 0 0 0 0 0 GreenMiddle to RWF 0 0 0 0 0 0 GreenRight to RWF 0 0 0 0 0 0 BlueLeft to RWF 0 0 43 0 43 43 BlueMiddle to RWF 40 0 57 40 17 57 BlueRight to RWF 0 71 0 71 0 71 Energy to RWF 0 0 0 0 0 0 Random to RWF 73 0 0 73 73 0 RedLeft to RWB 56 0 0 56 56 0 RedMiddle to RWB 0 0 0 0 0 0 RedRight to RWB -29 0 -41 29 12 41 GreenLeft to RWB 0 -78 0 78 0 78 GreenMiddle to RWB 49 0 0 49 49 0 GreenRight to RWB 0 0 20 0 20 20 BlueLeft to RWB 0 0 0 0 0 0 BlueMiddle to RWB 0 0 27 0 27 27 BlueRight to RWB 0 16 0 16 0 16 Energy to RWB 0 -4 0 4 0 4 Random to RWB 0 0 0 0 0 0 RedLeft to Attack 0 0 0 0 0 0 RedMiddle to Attack 0 75 0 75 0 75 RedRight to Attack 0 0 0 0 0 0 GreenLeft to Attack 21 -35 0 56 21 35 GreenMiddle to Attack 0 0 0 0 0 0 GreenRight to Attack 0 0 -30 0 30 30 BlueLeft to Attack 68 0 36 68 32 36 BlueMiddle to Attack 0 1 0 1 0 1 BlueRight to Attack -52 0 0 52 52 0 Energy to Attack 0 0 0 0 0 0 Random to Attack 0 0 41 0 41 41 RedLeft to Feed 26 47 0 21 26 47 RedMiddle to Feed 0 69 78 69 78 9 RedRight to Feed 0 0 22 0 22 22 GreenLeft to Feed 0 0 87 0 87 87 GreenMiddle to Feed 0 11 0 11 0 11 GreenRight to Feed 25 0 0 25 25 0 BlueLeft to Feed 0 0 0 0 0 0 BlueMiddle to Feed 0 0 0 0 0 0 BlueRight to Feed 0 0 0 0 0 0 Energy to Feed -68 0 0 68 68 0 Random to Feed 0 0 0 0 0 0

(24)

References

[1] V. Braitenberg. Vehicles: Experiments in synthetic psychology. MIT Press, 1984.

[2] D. P. Buxhoeveden and M. F. Casanova. The minicolumn hypothesis in neuroscience. Brain, 2002.

[3] R. K. G. Elwyn R. Berlekamp, John Horton Conway. Winning ways, for your mathematical plays. Academic Press, London New York, 1982.

[4] C. Emmeche. The Garden in the Machine: The Emerging Science of Artificial Life. Princeton University Press, Princeton, 1996. [5] M.-O. Gewaltig and M. Diesmann. Nest (neural simulation tool). Scholarpedia, 2(4):1430, 2007.

[6] A. P. Goucher. Oblique life spaceship created. http://pentadecathlon.com/lifeNews/2010/05/oblique_life_spaceship_created.html, May 2010.

[7] R. Gütig, R. Aharonov, S. Rotter, and H. Sompolinsky. Learning Input Correlations through Nonlinear Temporally Asymmetric Hebbian Plasticity. The Journal of Neuroscience, May 2003.

[8] J. Klein. breve: a 3d simulation environment for multi-agent simulations and artificial life. http://www.spiderland.org, Octobre 2013. [9] C. Langton. Artificial life : an overview. MIT Press, Cambridge, Mass, 1997.

[10] C. G. Langton. Self-reproduction in cellular automata. Physica D: Nonlinear Phenomena, 10(1–2):135 – 144, 1984. [11] T. M. Mitchell. Machine Learning. 1997.

[12] V. B. Mountcastle. The columnar organization of the neocortex. Brain, 1997.

[13] S. Russel and P. Norvig. Artificial Intelligence: A Modern Approach, Second Edition. 1995.

[14] J. Sjöström and W. Gerstner. Spike-timing dependent plasticity. http://www.scholarpedia.org/article/Spike-timing_dependent_plasticity, 2010.

[15] Selection(genetic algorithm). http://en.wikipedia.org/wiki/Selection_(genetic_algorithm), May 2014.

[16] L. Yaeger. Computational genetics, physiology, metabolism, neural systems, learning, vision, and behavior or polyworld: Life in a new context. In Artificial Life III, Vol. XVII of SFI Studies in the Sciences of Complexity, Santa Fe Institute, pages 263–298. Addison-Wesley, 1993.