BLEKINGE TEKNISKA H ¨OGSKOLA
BACHELOR THESIS IN COMPUTER SCIENCE
A first approach in applying Artificial Potential Fields in Car
Games
Tim UUSITALO,tim.uusitalo@gmail.com
June 12, 2011
”Sisu”
− Finnish word for having the guts
Abstract
In car racing simulation games, finishing first is the main goal. To reach that goal, it is required to go around a racing track, competing against other cars aiming for the same goal.
Implementing a bot for doing so may have its difficulties, although using a technique called multi-agent systems combined with artificial potential field, let- ting the agents take care of subtasks like keeping the car on the track, minimize how much the car turns in a curvature and basics in navigation around the track, has showed that artificial potential fields very well fit the problem of driving a car in simulated car racing in a competitive way.
Acknowledgements
I would like to thank my supervisor Dr. Stefan J. Johansson, whom has supported and supervised my work. He has been a great help on the way to my findings.
I would also like to thank my parents, for being there and supporting me when things have been both bad and good.
I would also like thank Jeremy Clarkson1, for his great enthusiasm in scream- ing ”poweeer” when accelerating in a car, which lead to the name of this car- controller, ”Powaah”.
The last acknowledgement goes to the people whom organized the simulated car championship, which have provided the basics of the client and a challenging problem.
1Jeremy Clarkson is an English broadcaster, journalist and writer and best known for his role on the BBC TV show Top Gear
Contents
1 Introduction 1
2 Research question and hypothesis 2
2.1 Research Question . . . . 2
2.2 Hypothesis . . . . 2
2.3 Verification of the hypothesis . . . . 2
3 TORCS 3 4 Using Multi-Agent Potential Fields 4 4.1 Artificial Potential Fields . . . . 4
4.2 Multi-Agent Potential Fields . . . . 6
4.3 Identification of objects . . . . 7
4.4 Finding the driving forces of TORCS . . . . 7
4.5 Placing the objects into different fields . . . . 8
4.6 The granularity of time and space . . . . 8
4.7 Creating agents from the driving forces . . . . 8
4.8 The architecture of the MAS . . . . 9
5 Strategies and techniques 11 5.1 Choosing track sensors . . . . 11
5.2 Lookahead points . . . . 11
5.2.1 Determine the length of a lookahead point . . . . 13
5.3 The Agents . . . . 15
5.3.1 Shortest path agent . . . . 15
5.3.2 Curvature agent . . . . 16
5.3.3 Track agent . . . . 18
5.3.4 Driver agent . . . . 19
5.3.5 Interface agent . . . . 21
5.4 Track analysis (Navigator Agent) . . . . 21
5.4.1 Determine left/right placement of latitudal position . . . . 21
5.5 Determine current target speed . . . . 22
5.6 Using subparts of the provided controller . . . . 22
6 Experiments 23 6.1 Brief description of the opponents . . . . 23
6.1.1 Autopia . . . . 23
6.1.2 COBOSTAR . . . . 23
6.1.3 Neil . . . . 24
6.2 The tracks . . . . 24
6.3 Preparation . . . . 25
6.4 Experiment 1 . . . . 25
6.4.1 Shortest path agent . . . . 26
6.4.2 Shortest path with track agent . . . . 26
6.4.3 Shortest path with curvature agent . . . . 26
6.4.4 All agents . . . . 27
6.4.5 Results . . . . 28
6.5 Experiment 2 . . . . 29
6.6 Experiment 3 . . . . 31
7 Discussion 34 8 Conclusions and future work 36 8.1 Conclusions . . . . 36
8.2 Future work . . . . 36
A 2011 simulated car racing championship @ Evo 38
1 Introduction
In car games, such as Torcs [2] the main goal for a computer controlled car is to finish the race at the first position while competing against opponents as well as keeping the car on the track. To accomplish such a goal we need to know our way around the track, have good lap times, obstacle avoidance (such as keeping the car on the track) as well as not crashing into the opponents.
It was Oussama Khatib [3] (1986) who introduced Artificial Potential Fields (APFs) while he was looking for a method of avoiding obstacles in realtime for manipulators and mobile robots. He applied a technique which is based on moving a manipulator through a field of forces, using an attractive pole to reach a goal position. For obstacle avoidance, he used repulsive surfaces for the manipulators parts.
Previous work in applying APFs to control a vehicle has been done by Rosset- ter [4], where he used potential fields for avoiding accidents from lane departures.
For the use of APFs in games we have the bot that was developed for the first- person shooter (FPS) game Quake 2, by Thurau et al. [8]. The bot learns reactive behaviours, where a behaviour corresponds to a potential field of an action to be taken.
Combining a technique called multi-agent systems (MAS) with APFs, a tech- nique called multi-agent potential fields (MAPFs), have shown good results, where Hagelb¨ack [6] applies this technique for his Real-Time Strategy (RTS) bot for path finding and obstacle avoidance for hundreds of units.
One may ask what a multi-agent system is? Weiss [9] begins with first de- scribing what an agent is, which is a computer system that can take autonomous actions; an action that is decided without the aid of humans or other systems. The actions are to be taken in an environment which, for example, could be static or dynamic. He describes it as having several kinds of agents is a good idea, when the problem is too complex for a single agent to solve.
The study presented here will show that applying a MAPFs solution in car racing simulation games extends the list of applications where these kind of tech- niques can be used today.
2 Research question and hypothesis
In this section we will introduce the research question, followed by the hypothesis and finally how the hypothesis is verified.
2.1 Research Question
Can APF-based solutions be competitive in simulated car racing?
2.2 Hypothesis
To apply APFs in a car game, we need different fields that take care of different sorts of driving forces to get good lap times and being competitive. For clarity, the hypothesis is divided into three parts.
Navigation By having a field for navigating on the track we will be able to reach the shortest path possible for the track which gives us the basics in track navigation.
Minimize curvature Using one field to minimize curvature of a turn will make the car go faster around a curvature which will lead to faster lap-times.
Keeping the car on the track Not using another field to prevent the car from go- ing off course will result in slower lap times and lesser competitiveness.
2.3 Verification of the hypothesis
The hypothesis will be verified by implementing a bot, using a multi-agent system that contains potential fields in them, one for basic navigation on the track, one that will minimize how much the car needs to turn in a curvature on the track, and one for keeping the car on the track. This will be based upon the simulated car racing championship software [5] using Torcs as a test bench for the experiments.
3 TORCS
Torcs, which stands for The Open Racing Car Simulator is a highly portable multi- platform for car racing simulation. This means that it contains elements like 3D visualization, a good physics engine and it takes into account the dynamics of the car such as traction, aerodynamics, fuel consumption, etc. [2]. Torcs is today being used as a research racing platform from which you can construct your own car bot with several different techniques. You can as well drive around as a human player competing against computer controlled opponents or against time.
We have chosen to construct our idea in the competition software [5], which is based upon Torcs using a client/server solution. The server is Torcs itself, which provides the different kinds of data to the clients which corresponds to a computer controlled car controller. As input it takes some sort of car control data which includes:
Acceleration value between [0,1]
Braking value between [0,1]
Steering value between [-1,1]
Clutch value between [0,1]
Gear value between [-1,6]
The client’s job is to take an action and decide values for this car control data, and the client has as well some access to different kind of data to aid the decision of how to control:
Track sensors Sensors, that can be adjusted at the start of every race, around the car between the angles [-90,90] degrees. These sensors’ job is to read a distance to the track edge (in the range of [0, 200] meters) in the direction that it is being placed at. A noisy option can be enabled for these sensors at the server side, which means that they are affected by a normal noise with a standard deviaton of 10 percent of the sensors range.
Opponent sensors These sensors work in almost the same way as the track sen- sors. The difference is that we can not change where they are placed on the car. These angles are static and are placed at every 10th degree around the span [-180, 180] degrees, i.e., around the car.
Track position Determines if the car is at the right or left side of the track’s middle. This is a value between [-1,1] where -1 means right and 1 means left.
Track angle The angle between the direction of the car, and the direction of the track axis.
Gear Provides what gear we currently are in SpeedX The speed of the car in longitudal axis.
Distance from start Provides the distance in meters measured from the start line to where the car currently is along the track line.
There is of course more sensor data to access here, but we have chosen to explain only the most necessary parts i.e., the parts that are currently in use for this car controller. If one wishes to read further on about the different sensor inputs that are not being introduced here, we refer to the competition software manual [5].
4 Using Multi-Agent Potential Fields
In order to address the problem we have chosen a technique that is called Multi- Agent Potential Fields(MAPFs); a technique that combines Multi-Agent systems with Artificial Potential Fields.
4.1 Artificial Potential Fields
Hagelb¨ack [6] illustrates a common use of applying artificial potential fields in games of today. He apply the technique in a Real-Time Strategy game, using artificial potential fields for obstacle-avoidance and path finding.
He describe an artificial potential field as a field built up by tiles where one tile represents a potential value and a small area of the game world. Charges are placed at point’s of interest, for example, a specific place in the game world where a unit would go. A field is generated around the charges, which gradually fade out to zero through out the field. A charge can be either attractive or repelling. This is illustrated in Figure 1 which describes how this is being done for an attractive charge at E, with repelling charges around mountains (brown) and units (the green and white dots).
Figure 1: Illustrates how the charges can generate potentials in a field. Brighter blue tiles contains higher potential than darker blue tiles.
So how do we move a unit from where it is being placed at, to the point E in a field? Basically look ahead on the closest tiles from where the unit is standing, choose the tile that has the highest potential and let the unit go to that area, which continues until it has arrived at point E. This is illustrated in Figure 2 where we can see how the green unit chooses the path to the point E where it is standing from now, avoiding mountains and the white units that has generated repelling charges in the field. We may see that in Section 5.2 our lookahead method differs from how Hagelb¨ack [6] uses it.
2Figure 1 is created by Johan Hagelb¨ack
Figure 2: Shows how the green unit avoids the white units and the mountains when finding a path to point E.
4.2 Multi-Agent Potential Fields
To apply the technique MAPFS, we have chosen to go through all the phases that Hagelb¨ack [6] illustrates in his study. These steps are:
1. The identification of objects
2. The identification of the driving forces (fields) of the game 3. The process of assigning charges to the objects
4. The granularity of time and space in the environment 5. The agent system, and,
6. The architecture of the MAS
3Figure 2 is created by Johan Hagelb¨ack
4.3 Identification of objects
First we begin with finding the objects of interest in Torcs, which is:
• The track, which is found by using the Track sensors.
• The car, that the bot tries to control, and,
• The Opponents who are to be found by using the opponent sensors
When we have defined these objects, we want to find out if they are static or dynamic of the environment.
1. The track is defined as a static object, since this always lay at the same place in a race.
2. The car will be a dynamic object in this definition, since this is an object that moves around the track.
3. Opponents, which are dynamic since they are objects that moves around the track.
4.4 Finding the driving forces of TORCS
Here we shall define the driving forces (fields) of Torcs, which could, in our situ- ation, be to avoid obstacles, or to base the movements on how the track sensors or the track looks like. In Torcs the main goal is to:
• Finish a race in first place
Though, to achieve that some minor driving forces are required. They are:
• Drive around the track in the right direction as fast as possible
• Stay on the road
• Keep the control of the car
• Avoid crashing into opponent cars
When we identified the fields we were inspired by Cardamone et. al. [1]
which have identified two kinds of curvatures around the track. The first curvature they are calling Shortest Path (SP), and this represents a line which the car may follow to take the Shortest Path possible around a curvature. The second path they call Minimum Curvature Path (MCP) which will try to minimize how much the car needs to turn around a curvature. What they have done is that they use
a genetic algorithm to balance these two lines together into what they call The optimal racing line. The inspiration will show when we describe how we have decided to assign the jobs of the different fields which follow as:
• Finding the Shortest Path (Field of Shortest Path, F oSP ) possible around the track, which can be compared to the SP in [1], and,
• Stay on the track which is one important part in car racing, the name for this field is Field of Track (F oT ).
• To minimize how much the car needs to turn around a curvature, which we have defined as Field of Curvature (F oC) is kind of similar to the M CP in [1].
4.5 Placing the objects into different fields
Here we identify the forces that exist in the fields, which means that we assign a position and how strong the force is relative to our own car. This is done by the agents and can be found in Section 5.3.
4.6 The granularity of time and space
Car games are one kind of game where things may happen fast, and so we need to handle the decision making as fast as possible. APFs makes just that possible, which results in making decisions every frame using MAPFs being the decision making technique. In our tests we had no problem evaluating 171 lookahead points (which we have chosen to represent as a decision) in a resulting frame rate of 150 frames per second or higher. A description of our lookahead points can be found in Section 5.2.
4.7 Creating agents from the driving forces
This is the fifth step in the design of a MAPF-based solution, which is that we shall look and create the agents out from the objects. We have chosen here to create all the agents according to the driving forces that exist in Torcs, with an exception of the opponents which have not been agentified. Basically there is one agent for curvature, one agent for finding the shortest path and finally one for keeping the car on the track.
4.8 The architecture of the MAS
The last step is to create an architecture of the MAS. We designed it in a way that makes it easy to activate or deactivate the agents; it shall also be easy to add an agent to the system if one wishes to do so. In Figure 3 we can see that the driver agent communicates with the three agents that determine the charges.
We have the curvature agent which is responsible for the field of curvature, the shortest path agent which is responsible for the field of shortest path and finally we have the track agent which is responsible for the field of track. In these three agents, calculation of potentials for the different LookAhead (LA) points are to be done, followed by sending this information to the Driver Agent. The Driver Agent sends the LA point with the highest potential to the interface agent that will send the control data to the Torcs server. From the Torcs server we can see that we are getting the sensor input, which we provide to the agents.
Figure 3: Figure over the MAS-architecture
5 Strategies and techniques
Here follows a detailed explanation of methodologies and techniques that has been used when creating this car controller. First we will take a brief look into what track sensors we have chosen, followed by how the LA points work. After that we go straight into the agents followed by how we determine the current calculated speed.
5.1 Choosing track sensors
We have chosen here to use them in the standard way that the provided client uses, which is in the degrees [-20,20] we have a sensors at every 5 degrees, and in the span [-90,-20] degrees and [20,90] degrees we have a sensor at every 15 degrees.
This means that the sensors will be more focused on what happens at the front of the car, compared to the sides of the car.
5.2 Lookahead points
For representing multiple choice of actions we use LookAhead (LA) points which represents an approximated position to go to in the next frame. This method differs from previous work done by Hagelb¨ack [6], since what he have used is a field of tiles representing the game world where each tile represents a potential in the field, using this as basic path finding and obstacle avoidance in his solution.
For our game world we do not have this exact information about how it looks like, instead we have track sensors that measures the distance to the track edge for every frame. We also have more control output to take care of, so we have chosen LA points as our lookahead method where one LA point represents an action to take;
an action that represents how much throttle and brake will be applied and how much the wheels will be turning. The LA points are each given a potential from different kind of fields and the LA point with highest potential will be the action to take until the next frame, where a new action will be chosen. To determine where these positions might be, we have a number of inputs to take care of:
• The angle at which the wheels are pointing in between [-45,45] degrees where -45 and 45 degrees means the maximum limit of how much they can turn in the left and right directions.
• How much acceleration and deceleration is used in the LA point
• Velocity
The resulting velocity we calculate as being described in Equation 2, which determines the length, i.e., how far the car can go in a direction until the next
frame. Meanwhile the direction for the LA point determines the angle of how much the wheels shall point for a lookahead point, these two combined will result in a position in front of the car. When creating the LA points, for every fifth degrees that we go through, we add 6 LA points for the usage of the accelerator (values: 0, 0.2, 0.4, 0.6, 0.8 and 1.0), and 3 LA points for usage of the brakes (values: 0.6, 0.8 and 1.0), resulting in a total of 171 LA points.
In Figure 4 we can see how the LA points looks like when going in 155 kmph, and in Figure 5 we can see them zoomed in for better details.
Figure 4: The resulting lookahead points.
4Figure 4 is made with GNUPlot
Figure 5: The front lookahead points zoomed in
5.2.1 Determine the length of a lookahead point
To know how far away to place the lookahead point we have measured some accelerations on full throttle in all of the gears as well as having the deceleration of when full brake is applied. The numbers that we came up with when measuring is presented in Table 1 and since the car has different accelerations depending on what gear we are in, we measured the mean acceleration for every gear.
The resulting acceleration is calculated as taking the acceleration for the gear we are currently in, multiply this value with the throttle value from the LA point, which results in how much we can increase the velocity from one frame to an- other. We also want to calculate how much we can decrease the velocity, which is why we add the deceleration to the resulting acceleration, which is multiplied by the value of the brake-pedal. The following formula describes the resulting acceleration in hand:
ares(agear, adec, pthrottle, pbrake) = agear· pthrottle+ adec· pbrake (1)
5Figure 5 is made with GNUPlot
Gear Acceleration (m/s2) 1 7.1666667
2 6.045 3 4.8783 4 4.05 5 2.81 6 2.04 Brake -21.16
Table 1: Describes the acceleration for a gear when the throttle is fully applied, and the deccelaration when the brakes would be fully applied.
where ares is the resulting acceleration for the LA point, agear is the acceleration depending on what gear we are in, adec is the decelaration value, pthrottle is the throttle value and pbrake is the value of the brake-pedal. Now when we have the resulting acceleration for the LA point, we can calculate the resulting velocity.
This is calculated as taking the current velocity, adding the acceleration which is multiplied by the time to the next frame. The formula for vresis used as follows:
vres(vcurrent, ares, dt) = vcurrent+ ares· dt (2) Here vcurrent is the current speed, ares is the resulting acceleration and dt is the time it takes from on frame to another, which we from now on will call the delta time.
To determine where a lookahead point might appear, we calculate a distance which the car will travel in one time frame, meaning that we multiply the resulting velocity with our delta time.
s(vres, dt) = vres· dt (3)
Where vres is the resulting velocity and dt is the delta time. Now when we know the distance it will travel for a LA point, as well as the direction (which is specified in the LA point), we can determine an approximated position in front of the car where it will travel to the next frame.
5.3 The Agents
Here we will describe how we have applied the charges in the fields, currently letting the agents being responsible for that.
5.3.1 Shortest path agent
The shortest path agent’s job is to put a charge at the field that makes the car want to go around the track in the shortest path possible, i.e., Field of Shortest Path.
The technique used for this agent is that it looks at the sensors and chooses the one that measures the longest distance, putting an attractive charge at that direction.
To determine how far away we should lay this point we have an input to this agent, namely the current calculated optimal speed for a current situation which is being described in Section 5.5. This speed is then multiplied by the delta time so that the length of this charge will appear at the distance we want to travel until the next frame. The calculation of the potential for a charge in field of shortest path is following:
fF oSP(d, c) = 60 − c · d2 (4)
Where we have chosen the value on the charge to c = 18, d in this case is the measured distance between the position of the LA point and the placement of the charge. So now when we know how the calculation are being done, as well as the placement for the charge in this field, an illustration of how this is being done is in order. If we look at Figure 6 we can see that the symbol ”SP”
is the attractive charge placed in front of the car, the car is the gray square, the green dots represents the LA points, and the blue values represents a potential for a given LA point on the road where brighter blue values means high potential values and darker blue color means low potential values.
Figure 6: Illustration of the Field of Shortest Path.
5.3.2 Curvature agent
This agent will put an attractive charge (in the field called Field of Curvature) at the right or the left side of the car to make it align on one side of the road depending on what kind of turn that is coming up against it. If it is a left turn, the charge will be placed on the right side of the road; if it is a right turn the charge will be placed at the left side of the road. For input it gets the track analysis (described in Section 5.4) that has been made on the warmup before a race, meaning that we now know what turn is coming up and can calculate the potential (for a charge in field of curvature) due to the following formula:
fF oC(w, d) = c · (1 − (d
w)) (5)
Where w is the width of the road and d is the length between the positions of the LA point and the placement of the charge. Here we can see that if we are distant to the LA point, the potential becomes greater than if we had been close to it.
The charge, c, in this case is calculated in a way so that it will start to align the car when being 250 meters away from the turn, having the maxima at 150 meters letting the function decrease when the length in this function becomes lesser than 150 meters. For simplicity we have provided a graph of how this function looks like which can be illustrated in Figure 7. The formula for calculating the charge in fF oC:
fc(df ront) = −0.001d2f ront+ 0.3df ront− 12.5 (6) where df ront is the distance of the frontal sensor placed at 0 degrees. Since we only want attractive charges from the curvature agent we have chosen only the positive values from this formula which is described as:
c = fc if fc>= 0 0 if fc< 0
Figure 7: Graph over how the charge is calculated in field of curvature
An illustration of how the charge is placed and generated in this field is illus- trated in Figure 8 where the green dots represents our LA points, the blue tiles represents a potential for a given LA point where the brighter blue tiles represents high potential values and darker blue tiles represents low potential values. The symbol ”C” is the charge for this field and the gray square is the car. In this figure we can see that this agent want to align the car to the left side of the road, since a curvature that goes to the right is coming up in front of the car.
6Figure 7 is made with Wolfram Alpha:http://http://www.wolframalpha.com/
Figure 8: An illustration of the Field of Curvature.
5.3.3 Track agent
Track Agent will try to make the car stay on the road, by having repelling charges layed out in the field (Field of Curvature). This is done by putting the repelling charge at the distance which the track-sensors measures, since they measure a dis- tance to a position we want to avoid e.g., a track edge. The formula for calculating the potential (for a charge in field of track) are following:
fF oT(d, c) = c
√d (7)
Here d is the distance between the position of the LA point and the placement of the charge, c is the charge which is equal to a value of c = −2.
We illustrate how the charges are being placed in this field in Figure 9 where the symbol ”T” is the charges placed out in this field, the gray square is the car, the
brighter blue tiles represents high potential values and darker blue tiles represents low potential values. We also have the charge ”SP” and the field it generates, where we can see that the track edge becomes more darker than it were in Figure 6.
Figure 9: An illustration of the Field of Track.
5.3.4 Driver agent
Driver agent is the agent that is responsible for the LA points. The driver agent updates the position for the lookahead points in every frame and provides them to the shortest agent, track agent, and curvature agent. Finally, when the agents have had their turn of adding potential to the LA points, we check which one of the points has the highest potential, and send it to the interface agent. To get the final potential value for a LA point, we need to sum the potential values from every
Figure 10: An illustration of all agents activated
charge in all fields. This is illustrated as following:
X
a∈A
X
c∈P F
fn(d, c) (8)
where A is the agents, fais the function in agent a to calculate a potential value, d is the distance between the LA point and the charge, c is the charge and PF is the field for the agent a.
In Figure 10 we illustrate how it looks like when having all the agents ac- tivated, combining the different fields. The brighter blue tiles represents high potential while darker blue tiles represents low potential. The symbol ”SP” is rep- resenting an attractive charge placed by the shortest path agent, the symbol ”C”
is an attractive charge placed by the curvature agent and finally the symbol ”T” is the repelling charge placed by the track agent. The green dots represents our LA
points and the gray square is our car. Here we can see that the brighest blue tile that were placed exactly on the charge at ”SP” in both Figure 6 and 9 are moved to the left side against the charge placed at ”C”.
5.3.5 Interface agent
The interface agent is responsible for taking in a LA point and convert it so that Torcs understands what we want to do, followed by sending it away to the Torcs server. It also provides the sensor data from the Torcs server to the rest of the agents. This is typically a connection between the agents and Torcs itself.
5.4 Track analysis (Navigator Agent)
In the warmup for every race we use simple analysis of the track, since we do not have any kind of knowledge of the track except for what the sensors read. So to know the track, we drive around it at very low speed, following the track axis (since we know how much the cars angle differs from the track axis, we need to turn in a way to make this value as close to zero as possible) using the front right, and front left sensors (the two sensors at -5 and 5 degrees) to know what turn is coming. To examine this we look at the k-value of the line that comes between these two points, if it is a negative value we define the turn as a left, if the k-value is positive the turn is defined as a right one. If though the front sensors are longer than the right and left, we are in a straight.
To know where on the track we are, we use the current distance from the start line (provided from the Torcs server), and saves a track point at that position, which contains if the track goes to the left or right. When we want to know the current track point, we choose the one with the shortest distance from where we are at a given distance. For knowing the closest turn that lies ahead of us, we look ahead in the track points and take the turn that has the closest distance to us.
5.4.1 Determine left/right placement of latitudal position
In Torcs, the track width have the same width overall on the track, meaning that we can use the width everywhere on the track. Since we have chosen to follow the middle of the track we know that the width must be found by adding the sensor value from the sensors that lies in the angle -90 degrees and 90 degrees. Though there is one problem here, when having the noise on the sensors activated the width would not be the correct width, which is why we have chosen to measure a width in every track point, using the width as the mean value calculated of all widths measured in our track points.
To determine how far the track edge lies from the left and the right side of the car we can use the track position value that the competition software provides us, which is described in Section 3. The calculation of the right edge is done by:
dright(trk, w) = (0.5 + 0.5 · trk) · w if trk > 0
|(0.5 + 0.5 · (trk − 1.0)) · w| if trk < 0
Where w is the width of the road and trk is the track position value. Now when we have the distance to the right edge and the width of the track, we can calculate the distance to the left:
dlef t(w, right) = w − right (9)
Where w is the width of the track and right is the right distance of the car to the track edge.
5.5 Determine current target speed
To determine the current target speed which we want to aim for, we have chosen here to use a relatively easy approach. We simply take the sensors that lies at -10, -5, 0, 5, and 10 degrees, sum the measured sensor length that they provide and then calculate the mean value of these five lengths. The formula for calculating the target speed is following:
vtarget(sll, sl, sm, sr, srr) = sll+ sl+ sm+ sr+ srr
5 (10)
Where sllis the sensor at -10 degrees, sl is the sensor at -5 degrees, smis the sensor at 0 degrees, sr is the sensor at 5 degrees and finally srris the sensor at 10 degrees.
5.6 Using subparts of the provided controller
Provided methods that comes with the competition software which we have cho- sen to use is following:
• Gear change
• What we shall do if we get outside of the track
• What we shall do if we get stuck
6 Experiments
The experiments have been made on a computer that runs the Torcs server waiting for clients to connect. On the same computer we have executed the clients that are to be connected to the Torcs server. The computer was running Ubuntu 10.10 at that time, in a runtime environment as similar to the competition environment as possible.
We begin with illustrating what happens when we activate and deactivate the different agents combined with each other. In Experiment 2 we compare the lap times with the opponents, followed by Experiment 3 where we let our bot race against the opponents.
6.1 Brief description of the opponents
For the experiments we have used the last year competitors Autopia which was the winner from 2010 simulated car racing championship, COBOSTAR which was the winner from 2009 simulated car racing championship, and Neil which finished around the middle placement in the simulated car racing championship 2010.
6.1.1 Autopia
The Autopia competitor turned out to be a very fast driver, with best racing times as well as having good race results making this driver one of the hardest com- petitors. Autopia is using a Fuzzy Architecture which bases the driving on three basic modules which is from what gear we should be in, how fast we should go and steering control. In the warmup stage it initializes a vector to 1.0, and tries to maintain that vector with as many real values as track length in meters. In case of any damage occurs while driving around in the warmup, or if the car goes outside of the track the vector positions are multiplied by 0.95, from 250 meters before the current position. To maintain a cautious function this vector is multiplied by a function F = 1 − 0.02 · [damage/1000] to make the driver more cautious.
6.1.2 COBOSTAR
COBOSTAR [7], which was one of the best drivers from the 2009 championship, is very good on fast tracks. However, on more challenging tracks (like Ruud- skogen and Wheel 2 that we will use in our experiments), it becomes more care- ful. COBOSTAR is using a parameterized controller which is optimized with a technique called CMA- ES (Covariance Matrix Adaption Evolution Strategy). For knowing where on the track it goes off the road during the warm up, COBOSTAR
saves the crash points dynamically, in that way it can be more careful next time it enters those curves.
6.1.3 Neil
The Neil driver is a careful driver, using a Feed Forward Multi Layer Neural Network, with imitation learning to control the car. To train this classifier the controller is provided with human data from the author, which differs a bit from the rest of the competitors since the other drivers somehow takes the track into account to decide their behaviors.
6.2 The tracks
The tracks that we have been using in our test is a mixture of simple fast tracks and tracks that are a more tricky to get around. CG Speedway 1 and 2 are fast and easy tracks, meanwhile Wheel 2 contains long fast corners with a mixture of hairpin bends. Ruudskogen contains a very small width, with very tricky corners that are to be taken in both faster and slower speeds. Table 2 describes some numbers about the tracks and Figure 11 illustrates how the tracks looks like.
Table 2: Some numbers about our tracks Track Length (meter) Width (meter)
CGS1 2057.56 15
CGS2 3185.83 15
Ruudskogen 3274.20 11
Wheel 2 6205.46 12
(a) CG Speedway 1 (b) CG Speedway 2
(c) Ruudskogen (d) Wheel 2
Figure 11: Pictures of the tracks that the experiments have been made on
6.3 Preparation
In preparation for every track that the races will be placed on, all car controllers were given 100000 in-game ticks of racing. Here the controller can drive around alone and learn how the track looks like.
6.4 Experiment 1
With this experiment we will illustrate how the presence of the agents affect the outcome of the best lap time followed by the total time. We chose to use single car race, where one car is placed at the pole position of the track, racing for the best possible lap time followed by the total time. The amount of laps driven around the track for this experiment were a total of 10 laps. We did not have an experiment with the shortest path agent deactivated, since our bot does not work at all without it.
6.4.1 Shortest path agent
First, the car drove around the track with only the shortest path agent activated, where Table 3 shows the best lap times and the total times for each track.
Table 3: The results of having only the shortest path agent activated Track Best lap-time Total time
CGS1 45.38 466.27
CGS2 63.86 645.77
Ruudskogen 83.99 855.45
Wheel 2 145.45 1463.98
Sum 338.68 3431.47
6.4.2 Shortest path with track agent
We added the track agent, with the shortest path agent still activated, to illustrate the difference with having the track agent switched on. The results are shown in Table 4.
Table 4: The results of having the track agent and the shortest path agent activated Race nr Best lap-time Total time
CGS1 45.16 460.57
CGS2 63.09 640.91
Ruudskogen 79.74 808.19
Wheel 2 142.53 1436.37
Sum 330.52 3346.04
6.4.3 Shortest path with curvature agent
We wanted to illustrate what happens when the shortest path agent and the cur- vature agent work together without the track agent. The results are presented in Table 57.
7The car went off the track on Ruudskogen and could not complete a single lap which is why the sumed laptime and the sumed total time is a bit off compared with Table 3, 4 and 6
Table 5: The results of having both of the shortest path and the curvature agent activated
Race nr Best lap-time Total time
CGS1 42.87 482.05
CGS2 58.41 591.72
Ruudskogen DNF DNF
Wheel 2 133.30 1353.07
Sum 234.58 2426.84
6.4.4 All agents
In Table 6 we can see the outcome of driving around 10 laps on each track with the usage of all the agents, i.e., the shortest path agent, the curvature agent and the track agent activated.
Table 6: The results of having all of the agents activated Race nr Best lap-time Total time
CGS1 42.57 420.30
CGS2 59.52 609.76
Ruudskogen 73.31 741.50
Wheel 2 128.38 1260.47
Sum 303.78 3032.03
6.4.5 Results
To be able to illustrate the difference on the lap times from each agent we have provided two figures, Figure 128which shows the lap time on the horizontal axis and the agents on the vertical axis, and Figure 138which shows the total time on the horizontal axis and the agents on the vertical axis.
Figure 12: Graph over the improvements in lap time
8Figure 12 and 13 is made with OpenOffice.
Figure 13: Graph over the improvements in total time
6.5 Experiment 2
In this experiment we will compare our car controller with the opponents, having one race with noisy sensors, and one race were the sensors noise were turned off.
We measured the fastest lap time achieved on each track where the one with the fastest time overall will be the winner. The amount of laps driven around the track for this experiment were a total of 10 laps. Table 7 and 8 describes both experiments here, showing the fastest lap time that the controllers made during the session on each track.
Table 7: The test results for Experiment 2, using sensors with no noise Competitor CGS1 CGS2 Ruudskogen Wheel 2 Sum
Autopia 40.75 57.74 73.72 120.08 292.29
Powaah 42.57 59.52 73.31 128.38 303.78
Neil 47.0 65.03 80.73 139.44 332.2
Cobostar 47.41 53.77 90.86 147.41 339.45
Table 8: The test results for Experiment 2, using sensors with noise Competitor CGS1 CGS2 Ruudskogen Wheel 2 Sum
Autopia 41.39 57.26 74.25 123.3 296.2
Powaah 43.15 61.36 76.16 131.96 312.63
Neil 45.98 65.31 80.07 138.96 330.32
Cobostar 40.75 55.11 93.63 155.1 344.59
6.6 Experiment 3
In experiment 3, the cars were aligned in different positions on the starting grid, racing each other. The order was decided in the way that every car got the chance to start at one position equal amount of times, the starting order is presented in Table 9 and the results are presented in Table 11. The race consisted of 5 laps totally, meaning that the bot that finished first when these 5 laps ended, had won the race. Here the sensors noise was turned on, as it is in the simulated car racing championship. Table 10 shows all 12 races that has been made during this ex- periment, presenting the fastest time, total time, how much damage that occured during the race, and finally the scored points from the race. An additional point were given for achieving the fastest lap time from each race, and another point were given for achieving the least amount of damage from each race.
Table 9: The starting positions of every race in Experiment 5 Position Powaah Autopia COBOSTAR Neil 2
1 1,8,12 2,7,10 3,6,11 4,5,9
2 4,7,11 1,6,9 2,5,10 4,5,9
3 3,6,10 4,5,12 1,8,9 2,7,11
4 2,5,9 3,8,11 4,7,12 1,6,10
Table 10: The test results for Experiment 5, using sensors with noise
Bot Best Laptime Total time Damage Points
Race 1
Powaah 78.12 06.37.03 855 10
Neil 72.30 +00.17.83 1555 8
Autopia 81.01 +00.45.02 1403 6
COBOSTAR 95.25 +01.27.49 2031 5
Race 2
Powaah 77.55 06.33.14 102 10
Neil 81.20 +00.25.51 778 8
Autopia 72.22 +00.33.49 2787 6
COBOSTAR 96.07 +01.32.77 2143 5
Race 3
Neil 80.77 57.26 4479 10
Autopia 74.19 +00.02.83 3404 8
COBOSTAR 96.89 +01.13.93 4608 6
Powaah 84.66 DNF (Did Not Finish) 10431 -
Race 4
Powaah 77.93 06.35.50 2454 10
Neil 80.81 +00.20.75 1418 8
Autopia 71.80 +00.31.65 2957 6
COBOSTAR 91.76 +01.30.28 467 5
Race 5
Powaah 76.91 06.36.08 5048 10
Autopia 73.09 +00.00.99 1907 8
COBOSTAR 95.78 + 1 lap 3124 6
Neil 118.37 +4 laps 6520 5
Race 6
Neil 80.41 06.58.35 447 10
Autopia 72.50 +00.02.46 4364 8
COBOSTAR 97.18 +01.09.94 5954 6
Powaah DNF DNF 10043 -
Race 7
Powaah 77.37 06.32.80 0 10
Autopia 71.90 +00.18.21 2139 8
Neil 80.44 +00.52.74 452 6
COBOSTAR 94.46 +01.24.72 831 5
Continued on next page
Table 10 – continued from previous page
Bot Best laptime Total time Damage Points
Race 8
Powaah 77.03 06.31.40 0 10
Neil 80.61 +00.19.47 318 8
Autopia 71.32 +00.24.61 1696 6
COBOSTAR 95.22 +01.31.69 9 5
Race 9
Autopia 73.52 06.49.95 6079 10
Neil 81.72 +00.56.95 3524 8
COBOSTAR 95.33 +01.17.25 6063 6
Powaah DNF DNF 10150 -
Race 10
Neil 80.50 06.49.24 0 10
Autopia 77.26 +00.05.67 1793 8
Powaah 77.35 +00.10.92 1450 6
COBOSTAR 95.76 +01.16.77 1108 5
Race 11
Powaah 77.20 06.31.97 1113 10
Autopia 72.93 +00.25.83 3778 8
Neil 82.13 +01.14.79 5112 6
COBOSTAR 96.56 +01.36.47 5250 5
Race 12
Autopia 71.40 +06.30.95 3015 10
Powaah 76.83 +00.06.91 1270 8
COBOSTAR 96.10 +01.34.31 2153 6
Neil 85.12 + 1 lap 2939 5
Table 11: The final results of Experiment 5
Controller 1st 10 2nd 8 3rd 6 4th 5 DNF FL LD Sum
Autopia 2 7 3 0 - 12 2 122
Powaah 7 1 1 0 3 0 6 96
Neil 3 4 2 3 - - 3 95
COBOSTAR - - 6 6 - - 1 68
7 Discussion
Previous work, by Rossetter [4] (2004), has shown that applying potential fields for making a vehicle avoid accidents from lane departures is appliable. One dif- ference between our studies is that he executes the experiments on a real car and not using a car racing simulator game which we have done. It would however be interesting to see if our work could be applied in a real car on a real track. If the sensors can be arranged as they are in our experiments and the car can be con- trolled in the same manner as it is being controlled in Torcs, it might very well be applied to a real car for driving around a racing track.
With our test results, we have shown that APFs very well fit the problem of driving a racing car around a track in car games.
In Experiment 1 we can see that our bot needs all of its agents to perform as good as possible. When we added the track agent with the shortest path agent the lap times became decreased meaning that the car went faster around the track with a car that became more careful when approaching the track edge. When we added the curvature agent with shortest path agent we can see that there are major differences in the lap times, though it was somewhat unstable when aligning in the curvatures and on Ruudskogen it could not complete a single lap without going off track, this problem was solved by having the track agent activated. When we had all of the agents turned on the car performed at its best, being able to stay on the track while going as fast as it can around it.
In Experiment 2 we can see that our car is close to Autopia at some points, and even beats it once on Ruudskogen when using sensors without noise. It finishes second in both results, beating Neil and Cobostar. Cobostar seemed to have some problems with tracks that were more trickier than others since on Ruudskogen and on Wheel 2 it performed quite badly compared to Autopia and our bot.
In Experiment 3 we can see that Autopia once more has beaten us, if we look at the points in the total. If we check the results of the races we can see that winning 7 out of 12 races is not that bad, though Autopia beats us by having a better mean value of points in every race, as well as having the fastest laptime in every race.
Our bot however was one of the better in the field of staying on the track, since in 6 races out of 12 we had the least amount of damage when finishing, note however that our car controller did not have any system for avoiding collisions with the opponents.
So with our experiments we have showed that having all of our agents the car drives in an effecient way around corners without going off the track. The charges for the different fields are manually assigned, meaning that the values have been tested in a race and increased or decreased trying to balancing the different fields with each other. It would be interesting however to find out wether a genetic algorithm would be better optimizing these values trying to adapt the fields for the
current track racing on, which might lead to better lap times.
To add an agent that holds a field for avoiding collisions with other cars might have changed the results in Experiment 3. Since in 3 out of 12 races we crashed so hard, mainly with other cars, so our car became so destructed it could not finish the race. This resulted in that we did not score any points at all in those races. If such an agent would have existed when the experiments were made, and it could avoid those three destructions of the car, we would have at least gotten 15 points more than we did, which would take us closer to Autopia.
One problem we have had is our lookahead points in this solution, since the points that represents acceleration and deceleration is very close to each other, the action taken by the agents more or less became full throttle or full brake.
Improvements could be done here to let the lookahead points have more space between each other so the car can have full usage of more than just the LA points with full throttle or full brake.
