Autonomous Hauler Loading

(1)

Autonomous Hauler Loading

Abstract

I det här examensarbetet har designat och implementerat komponenter till ett system för automatiserad materialhantering i anläggningsbranschen, en autonom hjullastare. Examensarbetet avslutas med en uppvisning av den prototyp som vi implementerat våra komponenter på. Vårt främsta bidrag till systemet är en ny mjukvarudesign, med det nya upplägget har det blivit enklare att integrera nya komponenter i systemet. Förutom plattformen så har vi tagit fram kompo-nenter och lösningar för era delsystem som krävs för autonom operation med hjullastaren, dessa har vi listat några av dessa nedan.

Tillståndsmaskin för hantering av övergripande uppdrag.

Bildbehandlingsalgoritmer, som utifrån LiDAR data parametriserar last-mottagaren (i detta examensarbete en dumper)

Dynamisk translationsalgoritm, denna används för att genomföra en tömn-ingssekvens på en godtycklig koordinat.

Mycket jobb har även lagts på utveckling av underliggande system och funk-tionalitet, bland annat har vi implementerat kommunikations protokoll och API för en LiDAR skanner samt ett API för en servo controller. Vi har också kon-struerat en mekanisk vagga för LiDAR skannern och servomotorn i CAD, denna konstruktion tillverkade vi senare i en CNC fräs.

Jonatan Blom

Supervisor: Torbjörn Martinsson Examiner: Lars Asplund

Volvo Ce Mölardalen University

Emerging Technologies Department of Innovation, Design and Engineering

(2)

Autonomous Hauler Loading

Jonatan Blom

Supervisor: Torbjörn Martinsson Examiner: Lars Asplund

Volvo Ce Mälardalen University

Emerging Technologies Department of Innovation, Design and Engineering

(3)

Abstract

In this thesis we have designed and implemented components for an tonomous system for material re handling in the construction business, an au-tonomous wheel loader. Our achievements was at the end of this thesis demon-strated on a prototype machine. Our main contribution to the autonomous wheel loader is a new software platform design, with this new platform future expan-sions is made easier. We have also developed several software components for this platform that solves tasks required for autonomous operation with the wheel loader the most important listed below.

Mission state machine

Vision algorithm, detect and parametrizes a load receiver (in this thesis a hauler) from LiDAR scanner data

Translation algorithm, translate the wheel loader to a dynamic target and there empty the payload

Low level development on the vision system of the autonomous wheel loader was also required to bee able to reach the target of the thesis. We have imple-mented a communication driver and a API for a LiDAR scanner and a API for a servo controller. A mechanical xture have been designed in CAD, manufacturing of this xture was achieved using a CNC mill.

(4)

List of Figures

2.1 Short load cycle . . . 5

3.1 Short load cycle simplied . . . 10

7.1 Software design overview . . . 28

7.2 State machine, short load cycle (second half) . . . 30

7.3 UML diagram, nd hauler algorithm . . . 34

7.4 Find hauler algorithm non translation mode . . . 35

7.5 Find hauler algorithm translation mode . . . 35

8.1 Vision system CAD Design . . . 56

8.2 Vision system zero return switch open . . . 58

8.3 Vision system zero return switch closed . . . 59

8.4 The vision system mounted on the wheel loader . . . 60

8.5 Vision system manufacturing . . . 63

8.6 Conventional and climb milling . . . 64

9.1 D-Latch frequency divider . . . 66

(8)

Nomenclature

Abbreviations

Ack Acknowledgement GND GrouND

NAck Negative Acknowledgement Acronyms

API Application Programming Interface CAD Computer Aided Design

CAN Controller Area Network CNC Computer Numeric Control CRC Cyclic Redundancy Check CSAM Cluster Split and Merge

DC Direct Current

DLL Dynamic Link Library ECU Electronic Calculation Unit

(9)

Nomenclature GPS Global Positioning System

GUI Graphical User Interface HMI Human Machine Interface IMU Inertial Measurement Unit

IO Input Output

LiDAR Light Detection And Ranging LMS Laser Measurement System

OS Operating System

PC Personal Computer

PSU Power Supply Unit

PWM Pulse Width Modication RPM Revolutions Per Minute

RS Recommended Standard

SLAM Simultaneous Localization And Mapping STX Start of TeXt

TCP Transmission Control Protocol UDP User Datagram Protocol UML Unied Modeling Language USB Universal Serial Bus

VCC Positive supply voltage WiFi Wireless Fidelity

(10)

Chapter 1 Introduction

A wheel loader is a reversed tractor equipped with a bucket; the main purpose of the wheel loader is to move material. The machines main domain is in factories, mining facilities and on construction sites. In factories they are for example used to move raw material from storage areas to the production facility. In mining facilities it is used to move the mined material and on construction sites they are used to reshape landscape, for example by moving blast rock and soil.

The strength of a wheel loader is its exibility, unlike a conveyor belt it can easily be redirected to move material in another area if needed. It is often used where the construction site is constantly moving, for example in a daybreak-mine. In many applications where a wheel loader is used, the task it performs is not very complex. It is adapting the task to the specic situation, being exible, that creates the need for a human operator. Such an example is an asphalt mill; trucks unload raw material from a nearby mine in huge piles in a storage area. The asphalt mill is equipped with several pockets; each pocket is assigned with a specic type of material. Each pocket is also equipped with a light that when turned on indicates that this pocket need relling. A wheel loader operator follow these signals and rells the materials that the factory needs, by driving the wheel loader back and fourth between the piles and the mill.

As mentioned each type of material have a dedicated pocket in the mill, so the unloading location is static, the diculty, or adaptation, in this task is that

(11)

around, the majority of asphalt mills are built on or close to ground that contains the minerals that are used in asphalt manufacturing. This saves transport costs, but when the mine is depleted and moves the asphalt mill have to move along with it. So the use of a wheel loader is a clear choice, it is very exible and it is a manageable economic cost.

Another example is a mine where trucks arrive to be loaded with gravel, here a wheel loader is used to ll the trucks with the requested type of gravel, it usually takes the wheel loader operator three or four buckets of gravel to ll a truck, this loading procedure is called short load cycle. This task, when it is adapted to the current environment is, as in the case of the asphalt mill, rather simple.

Both the short load cycle and the asphalt mill have potential for automation to some degree. In an economic perspective the operator represents 50% of the total cost of the wheel loader, in the cases where the task preformed is of low complexity money can be saved. An automated machine can also save money in more ways than eliminating the need for a operator, human operators tend to overload the machines causing a lot of damage and wear. The cost of this wear is many times put on the manufacturer; as it often is dicult to prove that the machine has been improperly handled, and some of this wear is thus covered by warranty. But this wear also aect the owner through more expensive service costs and wear of tires, that are very expensive on a wheel loader.

The operator also have a great impact on the fuel consumption, the eciency (measured in ton/liter), vary as much as 50% among the professional operators at Volvo Ce and theire induvidual eciency also tend to vary over time. The drive pattern of a autonomous machine can be strictly controlled, this in turn will lead to a controlable eciency.

There is a clear case for development of automation in this area, the Au-tonomous Machine project at Volvo Ce in Eskilstuna started in January 2008. This project aims to show a Volvo L120f (wheel loader) and a Volvo Ae25 (articulated hauler) that performs an autonomous short load cycle at the Intermat fair 2012.

(12)

Chapter 2 Background

2.1 Wheel Loader

The wheel loader (also known as back end loader) was introduced by Bolinder Munktell (Later Volvo Ce) in 1954. It was essentially a tractor but instead of the common conguration where the loading aggregate is placed over the front wheels the aggregate was placed over the rear wheels, allowing heavier loads and higher break out force, at the time a revolutionizing product.

Key features of a wheel loader:

Articulated steering, symmetric wheel base

Engine and a counter weight placed in the back end of the vehicle

Loading aggregate with two degrees of freedom, placed in the front end of the vehicle

Sti front axis and oscillating rear axis Four wheel drive.

(13)

2.2 Short Load Cycle or more generally a load receiver with material of a granulated kind. The material source can be of any kind among the most common are natural bank and stock, whatever the source its always an objective to perform the loading procedure with minimum eort. The procedure can generally be described by the following steps:

The operator of the load receiver drives his vehicle to an area instructed by the operator of the wheel loader, when he reaches the area he positions his vehicle approximately 50m away from the wheel loader with the rear end towards the wheel loader

The operator of the wheel loader drives his machine in to the material and loads the bucket, he then drives his machine in a v-shaped path so that it is almost parallel with the pile. He stops and raise his bucket to show the operator of the load receiver where he should position his vehicle

The operator of the load receiver moves his vehicle so that his bucket is underneath the bucket of the wheel loader, the operator of the wheel loader signals with his horn when he is satised with the position

The operator of the wheel loader then empty his bucket onto the load receiver, he then repeats the loading procedure until the adequate load is achieved, driving in about the same v-shaped path back and forth

(14)

2.2 Short Load Cycle

Figure 2.1: Short load cycle, the straight lines on the curve segment represent the front and rear body of the wheel loader in the end conditions of the cycle. As can be seen in the picture the front and rear of the wheel loaders are aligned when the wheel loader enters the pile

(15)

2.3 Autonomous Machine Project at Volvo CE

The Autonomous Machine Project was started in January 2008 at Volvo CE in Eskilstuna. The objective of the project is mainly to gain knowledge and to spawn interest within the company for autonomous solutions. Competitors are already working with autonomous solutions and research projects, especially for products within the mining segment. Volvo CE have not had any previous activities in this area, the founders of the project see this as a future key area necessary for survival within the construction equipment business. They hope that this project will be the start of development of autonomous products at the company.

The ultimate goal for the project is to demonstrate a continuous autonomous short load cycle with one wheel loader and one hauler at the Intermat fair in 2012. It was also decided that the development of these prototypes was to be done by thesis workers. This will keep the cost of the project low and as the company lack of competence within this area is will also be a good source for new recruitments. Up until the start of this thesis 5 thesis workers have been involved in the project, you can read more about their contribution to the project in

chapter:4. All eorts have so far been aimed at developing the autonomous wheel

loader as this was determined to be the most dicult and that knowledge gained in this development could be reused when developing the hauler, which would also be required to demonstrate a complete autonomous short load cycle.

(16)

Chapter 3 Thesis Description

The goal for this phase of the autonomous machine project is to perform a short load cycle autonomously. The remaining workload after the previously performed thesis works, to achieve this goal, has been divided in two thesis works.

Thesis one

Design and implement a software platform suitable for the autonomous wheel loader.

Design and implement an algorithm that nds a load receiver using a LiDAR scanner.

Design and implement an algorithm that empty the bucket of the wheel loader at this location.

Thesis two

Design and implement an algorithm that nds a suitable dig location in a gravel-pile using a LiDAR scanner.

Use the existing bucket ll algorithm to ll the bucket of the wheel loader at this position.

(17)

3.1 Demonstration

The assignment of this phase of the project also states a live demonstration of the prototype machine as a nal result. In this demonstration the two thesis works together shall show a third of an autonomous short load cycle. This is the wheel loader autonomously loading one bucket of material from the pile on to the hauler. For this demonstration the project have set the following constrains:

Wheel loader Volvo L120 Hauler Volvo Ae25

Environment, 75x75m, at, obstacle free area

Pile, convex surface, uniform gravel of size 10-16mm

Hauler positioned with the end of its bucket 1 meter from the pile and the angle between the pile and hauler should be approximately 120deg

Simplied translation, the wheel loader will do no turning while driving thus it will only be traveling on circle segments more details below

3.1.1 Simplied Short Load Cycle

So far in the project no work has been performed on how to translate a articulated vehicle to move it to a desired position. Generation of dynamic paths and the development of a controller to follow these are left to a later phase in the project. It has been agreed for this phase that the translation will be simplied to only setting new waist angles when the machine is halted, this means that the wheel loader can only travel on circle segments. This will lead to a simplication of the

short loading cycle that is illustrated by gure 3.1. One issue with this approach

is that the wheel loader will perform a bucket ll with a non-zero waist angle. This means that the force from the rear wheels will not be aligned with the force from the front wheels, and the wheel loader will twist and turn when it is forced into the pile. Also one of the requirements of the bucket ll procedure, the bucket shall be parallel with the pile when its entering the pile, can only be reached when

(18)

3.1 Demonstration the wheel loader starts in an optimal position, in most cases we will not reach this requirement. The same requirement exists for the emptying procedure, to have the bucket parallel with the hauler during the empty procedure, this will only be the case if we travel on a circle with innite radius towards the hauler which is a very unlikely scenario in the setup.

(19)

3.1 Demonstration

Figure 3.1: Short load cycle simplied, as can be seen the entire move is described by two curve segments, and as can be understood the wheel loader will only enter the pile with its bucket parallel to the pile under the illustrated optimal conditions. The same apply for emptying the bucket; the wheel loaders bucket is only parallel with the hauler under the illustrated conditions, and still only at the instance when the bucket enters the hauler

(20)

Chapter 4 Platform

To understand the actual task for this thesis we need a deeper knowledge of the existing platform, what is done, how it is done, what we can reuse and what do we need to add. We started our thesis by reviewing the existing solutions; we read the reports, viewed the source code and tested the functionality in the actual prototype. Below we have written a summary for each thesis describing the most important contributions that we depend on in this thesis. For more information we refer to the thesis reports, references are listed at the end of each summary.

4.1

Design and Implementation of HW and SW/HW Layer

The wheel loader assigned to the autonomous machine project is a pre-series machine from the development of the F series, model number L120F. The scope of the rst thesis in the project was to create a hardware platform and a control software platform to enable autonomous operation with this machine.

Hardware Interface To interface the machine the HMI is overridden electri-cally, thus the original software of the machine is not altered. This might not be the optimal way to solve this as the control signals from the autonomous system will be ltered and altered as if they are from a human operator.

A better solution would have been to alter the original software to accept ex-ternal commands. But due to issues with resources at the electronics department

(21)

4.1 Design and Implementation of HW and SW/HW Layer that develop the original software for the machine, it was decided to not involve them in this project.

With this hardware interface we can access all functionality of the machine that the operator can control except the brake which is fully analog (hydraulic brake pedal). Below is a list of the input and output (with respect to the au-tonomous system) of this interface.

Input:

CAN bus (used for internal communication between the machine ECUs) Analog angle sensors, tilt, lift and waist

Output:

Buttons digital, all button (except buttons to control the display) Levers analog, tilt, lift, waist and throttle

Aditional hardware It was also in the scope of this thesis to add and in-tegrate other necessary hardware on the machine, such as computers, sensors and actuators. Below is a summary of the additional equipment installed in this thesis.

Industrial computer PIP8 equipped with: digital and analog IO, CAN bus, RS232 bus, Ethernet etc.

Remote controlled emergency switch, with this the main power supply to the machine can be disabled, this will automatically engage the parking brake. The remote control also have 4 auxiliary buttons that in our case are used to enable/disable the interface to the HMI and to communicate with the computers in the machine.

Electric brake valve, connected in parallel with the brake pedal valve. Quadrature encoder, mounted on the main drive shaft.

(22)

4.2 Bucket Fill Regulator Pressure sensors at brake lift and tilt circuits.

Dual axis accelerometer for measuring machine angle.

Sick LiDAR scanner LMS291-s05, mounted on a temporary xture on the handlebar above the windshield.

USB camera. USB GPS.

More details about this work can be found in [3]

Software Layer The software layer of this thesis mainly aims towards the IO computer, the purpose is to develop software that enables control of the actuators of the machine in a more abstract manner.

The IO computer is running MatLab OS X PC Target, which is a real-time operating system that is capable of running a SimuLink model. In this thesis such a model was developed, this model is includes low level ramp following PI controllers for the hydraulic actuators and direct control of all other outputs (on/o or set-point).

For the high level algorithms its decided to use a standard PC running win-dows, the application is created in common c++. The communication between these two computers is enabled with a DLL supplied by MathWorks, that al-lows remote getting and setting of signals in the SimuLink model over Ethernet. A simple program is developed that demonstrates the basic functionality of the

complete system. More details about this work can be found in [3]

4.2 Bucket Fill Regulator

Filling the bucket of a wheel loader might at a rst glance seam like a trivial task. However due to machine dynamics lling the bucket is a complex task that have great impact on the machines running cost the major factors being.

(23)

4.3 Vision Fuel consumption; in a common wheel loader application ,short load cy-cle, 1/3 of the fuel is consumed when lling the bucket hence its of great importance to do this as ecient as possible

Wear of tires; when lling the bucket great force is applied to drive line and if too much force is applied the wheels will start slipping and cause wear, improper bucket lling can reduce the life time of the tires by as much as 20%. A set of new tiers for a mid size wheel loader is 100Ksek, as reference the price of a new machine is 1.7Msek

The scope for the third thesis work was to investigate how a professional operator performs a bucket ll procedure, and then design and implement a bucket ll algorithm in the SimuLink model. The algorithm that was developed was compared to a professional operator in a dened environment where productivity (time) and fuel consumption (integrated RPM) was measured. It is claimed by the author that it is comparable to a professional operator in productivity and eciency, without stating any numbers.

More details about this work can be found in [2]

4.3 Vision

To be able to perform autonomous operation the wheel loader needs to have knowledge about the environment it is operating in. The main sensor used in this project is a laser range nder; this sensor measures distances in a half circle at 1, 0.5 or 0.25deg interval. To increase the domain of this type of sensor it is in many applications used together with an actuator that tilts the sensor around its y-axis. The output of this type of system is commonly referred to as a point cloud (distance measurements in 3D space). An algorithm can then analyze the generated data to detect and parametrize, in our case, for example a pile or hauler.

The scope for the fth thesis worker is to design and implement such a system, that is add the tilting capability to the existing sensor. It is also in the scope to write an API in c++ for the sensor and actuator. As actuator for the tilting movement a DC servo from Maxon Motor is acquired, a mechanical construction

(24)

4.3 Vision is sketched but was never manufactured. A basic API for the scanner and ac-tuator is implemented but due to the fact that the mechanical construction was missing it was never tested as a complete system. Instead the thesis ended with a demonstration of the sensor without the actuator.

(25)

Chapter 5 Related Work

To gain more knowledge and inspiration for our solution, the next step in the thesis is to perform a search of existing solutions of similar problems. We found many reports and patents about implementation of autonomous vehicles of dif-ferent kind. These gave us much inspiration but we decided to focus on the most similar project in this report. A summary of this project can be found below, focusing on the task of emptying the bucket on to the load receiver.

5.1 Yamazumi No.4

This project is a collaboration between Advanced Industrial Science and Tech-nology (AIST), University of Tsukuba, Chou University, Tokyo Denki University and Hitachi. According to the papers published about the project, they have suc-cessfully implemented an autonomous short load cycle. We have also managed to nd a movie of this demonstration. This project has a more research oriented focus then the Volvo project. The Volvo project has an aim that it in the fu-ture should be possible to transfer to industrialization. This sets requirements on developing a solution with the costumer and cost in focus.

In the project the sensor setup is quite dierent then the sensor setup cur-rently used in the Volvo project. For pile detection they use a stereo vision camera

system [5]. To determine the location of the load carrier (a truck in the

demon-stration) they use a combination of sensors. The truck itself is equipped with a GPS that supplies both position and orientation, this data is transmitted with

(26)

5.1 Yamazumi No.4 WiFi to the wheel loader. Which in turn is also equipped with a GPS, with the

GPS data combined they make a rough estimation of the path, according to [7]

this system has an accuracy of about 0.5m. To dock the load carrier safely and to detect the height of the load carrier they use two LiDAR scanners, the scanners are of the same brand and model as the scanner used in the Volvo project. The scanners are mounted in a xed conguration scanning in the vertical plane, on xtures that extends from the sides of the wheel loader beyond the width of the bucket. Thus their eld of vision is never blocked by the bucket.

With these scanners they detect the actual distance to the truck, its relative angle and the height of its bucket. They describe the algorithm that process the

data from the scanners in more detail in [7], in short terms the algorithm for each

scan follow these steps:

Find the ground in the scan, the ground is assumed to be the points that belongs to a almost horizontal line

Find the line of points, among the points that dose not belong to the ground, that contains the most points. This line is assumed to belong to the surface of the load carrier bucket

Find the point in this line with the largest distance to ground, this point is assumed to be the top edge of the load carrier bucket

The distance and angle data that the two scanners and the algorithm generate are then used to align and position the wheel loader relative the truck, the height information is used as input when actuating the bucket.

In the project the authors have also attended issues as; path planning, genera-tion of scooping posigenera-tion & scooping (the later referred to as bucket ll algorithm in this thesis). But as these topics are not in the main scope of this thesis work we will omit the details about this, but more information about this can be found in [5], [6] and [4].

(27)

Chapter 6 Problem Formulation

6.1 Short Load Cycle Analyze

To further understand the requirements for our solution we performed a deeper analyze of the second part of the short load cycle. We decided to describe the process as a state machine as this would later on be easy to convert to code. The analysis is based on interviews with a professional operator employed at Volvo Ce, who also performed a live demonstration that was recorded on video for later review. Below follows the state machine that is the result of our analysis.

First state forward

Start condition: The wheel loader is halted at the reverse position between the pile and load receiver, the bucket is full of material, lowered and in a roll-back position

Actions: Waist angle is set to give a path towards the load receiver, forward gear is engaged, brake is released, the lift actuators are extended with full speed

Second state docking

(28)

6.1 Short Load Cycle Analyze Actions: Waist angle is adjusted to keep the bucket parallel with the load receiver, brake is lightly applied to lower the velocity, the lift actuators are extended with full speed, and the tilt actuator is retracted to start emptying the bucket

Third state stop

Start condition: The front wheels are 30cm away from the load receiver Actions: Neutral gear is engaged, full brake is applied to stop the wheel

loader, the lift actuators are extended with full speed, and the tilt actuator is retracted to empty the bucket fully

Fourth state raise bucket

Start condition: Bucket is empty

Actions: The tilt actuator is extended with full speed to raise the bucket edge over the load receivers bucket edge

Fifth state reverse

Start condition: The bucket edge is above the load receivers bucket edge Actions: Reverse gear is engaged, the brake is released, the lift actuators

are slowly retracted to start lower the bucket Sixth state reversing

Start condition: The bucket edge have cleared the load receivers bucket Actions: The lift actuators are retracted with full speed to lower the bucket

quickly and get better visibility Seventh state stop reversing

(29)

6.2 Work Packages Eight state nished

Start condition: The wheel loader have stopped

6.2 Work Packages

We are now ready to divide the thesis into work packages. We can describe our high level task in the classical robotics approach Sense Plan and Act.

Sense: How do we locate the load receiver

Plan: How do we adapt the task to our sensor data

Act: How do we control the wheel loader to execute our plan.

6.2.1 WP 1 Mechanics

The hardware setup of the project has already been set by previous thesis works so we do not have much inuence of the selection of the sensor setup. For sensing the load receiver we will use the existing LiDAR scanner but we will need the ability to tilt the scanner to gather more data than a single scan line. As mention the plan was that this feature was supposed to be added in a prior thesis but this failed. This feature will not only be required for this thesis but also for thesis two where the dig point will be detected from LiDAR data. But as this thesis started earlier then thesis two, the implementation of this feature is of practical reasons included in the scope of this thesis

Add degree of freedom to LiDAR scanner, design and construct a tilting cradle for the scanner

6.2.2 WP 2 Software Platform

Before we look in to the dierent algorithms that we need develop to solve our task we need to design a high level software platform. This platform must also integrate and support easy access to the functionality of the existing components.

(30)

6.2 Work Packages The vision system will be the main sensor for both theses in this phase of the project; it will also most likely be of high importance of future projects. Thus we see a need for a dynamic API that allows easy control/gathering of data from this system; this will also be developed in this thesis.

Design a new high level software platform

Integrate previously developed components in the new platform Servo controller, develop API layer software for the servo controller

LiDAR scanner, develop communication driver and API layer software for the scanner

6.2.3 WP 3 Software Algorithms

When we are able to gather data with our LiDAR sensor we need to design an algorithm that nds the load receiver from this data. Then we need to implement the state machine for the short load cycle. To execute our cycle we will use the existing controllers, but we will implement an algorithm that wrap the function-ality to a higher level. Our state machine will concern a more abstract target then only for example changing lift angle, a coordinate where to empty the load will be more feasible on this level.

Mission state machine, develop a state machine for the second half of the short load cycle

Vision algorithm, develop an algorithm that can detect and parameterizes a load receiver (in this thesis a hauler) from LiDAR scanner data

Translation algorithm, develop an algorithm that can translate the wheel loader to a dynamic target and there empty the payload

(31)

6.2 Work Packages

6.2.4 WP 4 Live Demonstration of Prototype

To complete our task it is stated that we in cooperation with thesis two shall perform a live demonstration.

(32)

Chapter 7 Software

In this chapter we will describe the software we have designed and implemented in this thesis.

7.1 Structure and Design (WP 2)

The thesis works before this have solved several fundamental problems, but they have all been solved as individual problems. It have also been the goal, set by Volvo, for all thesis to end with a live or video demonstration of the results. This have forced the thesis workers to spend time on things that will never be reused later in the project, instead of spending that time on preparing for future needs and integration. The existing structure is hence rather messy and up until now no eort have been put in developing an overall software structure. This is the major task for this thesis; design and implement a new software structure for the project. This time we also have to keep the future in mind; future extensions of the system must be possible without having to reconstruct the design. One major feature we have identied is that it should be possible to distribute the system on several computers if needed. As an example the autonomous machine project have started a research project with Örebro University, the goal of this research project is to implement more sosticated versions of the vision algorithms. The algorithms they develop are targeted for a Linux environment and thus the run

(33)

7.1 Structure and Design (WP 2)

7.1.1 Software Components

The rst task in the design was to identify the main components of a system for an autonomous wheel loader. And by main components we mean components that should be possible to distribute on separate hardware. We identied the owing main components:

Watch dog This component was requested by the project owner and is respon-sible of detecting and solving dead locks and similar errors that might occur. The functionality of this component is not in the scope of this thesis work.

Mission state machine Although the project is called autonomous machine it is still a costumer focused project and this machine will have a specic mission that it will perform. This component denes this mission, in this thesis this component is a state machine that executes a third of a short load cycle as is

described in the demonstration target (3.1). In the future of this project it will

at least be expanded to a complete short load cycle, but it could for example also be a load and carry cycle.

Vision This component controls the vision system, that is the LiDAR scanner and the actuator. It also contains the algorithms that that analyze the generated vision data, so far two algorithms, nd pile and nd hauler.

In future expansions of the project this component will most certainly become the most advanced and calculation heavy component of the software; there is a big possibility that this component will be moved to a dedicated computer. Algorithms this component could contain in the future could for example be:

Find obstacles

Find waypoint markers

(34)

7.1 Structure and Design (WP 2) Path planning and orientation This component is added for future purposes when the wheel loader needs to navigate over larger distances, for example in a load and carry cycle. This component will probably include things such as:

Dead reckoning algorithms and GPS for orientation

Maps and SLAM algorithms using data from the vision component to up-date the map

Algorithms to plan paths between coordinates in this map

X PC Host This component is an interface and abstraction layer to the SimuLink model developed in the rst thesis, that runs on the IO computer. For example it handles the more sequential processes of the machine control. Starting the ma-chine and conguring the HMI, moving the mama-chine a dened distance, these type of actions are typical constructed of many small actions in a sequence. SimuLink is of course capable of doing this as well but it is still easier in a more abstract environment like c++. It also handles the instantiation of the computer after a power on, downloading and starting the model.

7.1.2 Inter Thread Communication

These components all run on individual threads (or in future individual pro-cesses), to enable safe communication between them, we have developed a mail system. The mail object that is communicated between the components is de-signed so that it easily could be sent to another component written in a another language running on another os. Hence the mail object is quite basic it consists of a predened number of booleans, integers and doubles members. The meaning of the dierent members is agreed between the communicating components. It also contains a message and a response member which are of the same integer enumer-ation class, when a component responds to a mail it moves the message received into the response member, and the message member is set to acknowledged or not acknowledged. In a new message the response is left empty.

(35)

7.1 Structure and Design (WP 2) threads to prevent unnecessary blocking. This object contains the thread safe mechanism and also one queue for each direction, the queue holds the mail object until it is requested by the receiver, the queue follow the simple FIFO rule. Before a thread can use the mailbox for the rst time it has to connect to the mailbox, in this process the thread retrieves an unique id from the mailbox, if more then two connection attempts are made to one mailbox it will be refused. This id is used by the mailbox to determine which queue to place or retrieve a mail from.

If it is later on decided to distribute one component onto another computer it is then the idea that one should only have to modify the mailbox object, for example extend it with a simple UDP or TCP socket.

7.1.3 GUI and Console

The application also contains a console window; this is used for debugging and can easily be reached from all executing threads owned by the process. We also decided to add a GUI to display information that is of a more permanent nature and also to be able to send commands to components for testing purpose. The GUI has one tab page for each component each component also have one mailbox that is shared with the GUI. On the main tab for each component there is a message logging option that can be enabled, when this option is enabled the component will forward all outgoing mails to the GUI. The GUI will display these messages in a text box on the components main tab page. If a components tab page contains text boxes or similar that display output parameters from the component, logging option has to be enabled for this component. This is because the components are not sending any extra information to the GUI what the GUI displays is just information that the components is already communicating with another component, thus this must be forwarded to the GUI. When a mail is sent from the GUI it is treated just as if it arrived from component that usually sends the specic mail. The reason for this way of communication is to keep a lose connection between the components and the GUI, so that it can be removed without any unexpected eects when it is no longer needed.

(36)

7.1.4 Execution Control

Each component follows a template design that contains a method called main procedure, the rst statement in this method checks to se if this instance is al-ready executing or not, if not, it continues and checks all mailboxes in a set priority. If a new mail is received or all mailboxes have been checked the method again continues and executes a state machine. The content of this state machine is left to the specic component. Each component is then handed by reference as input to a system timer object that will call this method on an interval dened on the timer object instantiation. This way the components will run on a periodic scheme as far as it can be supplied by the OS and if they manage to complete their execution within the set interval. Typically the vision component will often overrun its execution time when it is analyzing vision data, as it is rather compu-tational heavy. This is a deadlock prone design but our approach to this is that all underlying code shall be non blocking, and all blocking code shall be executed within a timeout mechanism. If blocking do occur then it is a bug that has to be solved at that level.

(37)

Mission state machine Path plannning and

orientation X PC Host

Watch dog

Vision Console

:\>_ X PC Target Servo Controller IMU Translation

Counter Scanner E th e rn e t

Graphical user interface

20ms 20ms 20ms 20ms 20ms R S2 32 R S 4₂ 2 _RS23 2 RS2 32 Mailbox

System console access External buss

Figure 7.1: Software design overview, this picture shows the structure of the software design and how the components communicate

(38)

7.2 Mission State Machine (WP 3)

The mission that is in the scope of this thesis work is the second part of the short load cycle, the other part is done by another thesis worker in parallel to this thesis work. Both parts are implemented in this state machine, in this report we will only include the part that is in our scope. We have chosen to illustrate the structure of our state machine as a UML diagram.

(39)

7.2 Mission State Machine (WP 3)

Confirm machine pos

Request current lift angle Request current tilt angle

Wait for angels Request sent

Locate hauler

Request hauler position

Lift angle of limits

Change lift angle

Request new lift angle

Change tilt angle

Request new tilt angle

Wait for Lift and Tilt move Lift & Tilt reached

Lift ok & Tilt reached Lift reached & Tilt ok

Wait for vision system Request sent Lift & tilt angel ok

Goto Turn off machine Failed to send

Timeout

Failed to send || Timeout

Goto Turn off machine Timeout

Request translation

Request a bucket empty sequence at the hauler

coordinate Hauler found

Increase watchdog counter

Counter ++ Hauler not found Counter < limit

Counter > limit Failed to send

Failed to send Lift angle ok & Tilt angle of limits

Request sent & Tilt angle of limits

Request sent & Tilt angle ok

Request sent Failed to send

Failed to send

Request translation

Request a return to start position translation

Turn off machine

Request a machine power down sequence Timeout || Target reached

Wait translation

Request sent Timeout

Target reached

Figure 7.2: The state machine of the second half of the short load cycle, locating the hauler, positioning wheel loader and the bucket, emptying the bucket and return to the start position

(40)

7.3 Vision

Below we have listed the functionality our mission requires from the vision com-ponent.

Find and parametrize the load receiver Find and parametrize the pile

The contribution in this work is focused on the rst item in the list, nd and parametrize the load receiver (in our case a hauler). We have also spent much time and eort on the fundamental software of the vision component, these parts

are described on more detail in chapter:7.3.3.

7.3.1 Find Hauler Algorithm (WP 3)

As explained in more detail in5the load carrier localization in the Yamazumi No.4

project is accomplished with use of GPS and LiDAR data from two GPS receivers and two LiDAR scanners. In our project we only use one scanner combined with an actuator, thus we have chosen a quite dierent approach for the load carrier detection.

The load carrier used in our nal demonstration is a Volvo Ae25 with a slight modication the bucket have been equipped with reective markers. These markers have a signicant signature in the scan data, with these markers the bucket can be detected even when it is partly blocked by the wheel loaders bucket. Inputs to the algorithm

Two angles, start and stop angle for the actuator Desired delta angle between the scans

Desired resolution of the scan (even tough the algorithm allows for selection of resolution, any other resolution then 0.25deg is insucient to safely detect the markers, as the distance between the measuring point gets too large)

(41)

7.3 Vision Output from the algorithm

A vector that starts in the front end of the hauler bucket and end at the rear end

A vector that is orthogonal to the rst vector (in the wheel loaders ground plane), with its rst coordinate in the center point of the rst vector and its second coordinate so that it points away from the hauler (it has a preset length of 1m)

The algorithm has two modes to select between, one if the wheel loader is halted and one if it is in translation. If the wheel loader is halted, the algorithm is allowed to locate the hauler using several scans. In this mode the algorithm will try to nd both the front and the rear marker of the hauler. If this is the rst call to this algorithm, during this bucket emptying sequence, the algorithm requires that it nd both markers. If this is not the rst call and we have an approximate position for the hauler and its direction, the algorithm will complete even if it nds only one marker. The algorithm makes the rst scan at the start angle and then adds or subtracts the delta angle so that the scan angle approaches the stop angle. After each scan is completed the algorithm analyzes the scan and searches trough the scan to nd sequences of measurements with high reection values. If it nds a group of such values it calculates the distance between the boundary points in this group, it then compares this distance with the size of the markers. If it matches a marker its position is stored and the search continues, when the entire scan is analyzed the algorithm checks if both markers have been found. If they have not the algorithm requests another scan and repeats the procedure above, this is done until the scan angle have reached the stop position. If both front and rear marker have been found the algorithm aborts and returns the ndings, if the stop position is reached and only one of the markers have been found the algorithm can only proceed if the direction of the hauler is known (the algorithm sets the direction when it nds both markers). If the direction is set the algorithm use the found marker and the knowledge of the direction to calculate the position of the unknown marker.

If the algorithm is in translation mode it is not allowed to determine the haulers position from information from more then one scan, this, as the scan

(42)

7.3 Vision is not yet being transformed to compensate for movement. In this mode the algorithm stops searching if, after a scan is analyzed, one or two markers have been found. It is very unusual that both markers are visible in the same scan so in this mode the algorithm in most cases calculates the position of the second

marker. The algorithm is illustrated as a UML diagram in gure 7.3.

Figure 7.4 shows data from two actual scans, and also the results after they

have been processed by the algorithm. Figure 7.5 shows data from a single scan

(43)

7.3 Vision

Input Data Start search angle Stop search angle

Delta angle Find hauler mode

Start

Find hau ler mode

Request scan data Run marker detection

algorithm Return false S to_p re_a ch_e d M a rk_e rs n_o t f_o u_n d Return false Sto p r ea ch ed Ma rke r n ot fou nd

Calculate bucket vector and normal Determine hauler direction Marke_{rs fou} nd Return true Markers found M a rk_e r f_o u n d a n d h a u le r d ire c tio_n k n o w n

Until stop angle reached Or

Both markers found

Until stop angle reached Or

Both markers found Or One marker found

Update

haule_{r mod} e

Calculate bucket vector and normal

Use knowledge about hauler direction, the line

and the marker to calculate the position of

the second marker

Fit line to the found points and the marker

Run algorithm to find measured points that belong to hauler bucket

Figure 7.3: A UML Diagram of the nd hauler algorithm, the diagram only illustrates the general succession of the algorithm, all special circumstances have been omitted

(44)

7.3 Vision

Figure 7.4: The right picture illustrates the rst scan where the front end reex is detected, the lines in the picture is drawn with data from both scans. The left picture illustrates the rst scan where the rear end marker is detected, in this picture the path, calculated in7.4.1, to the hauler is also shown. In both pictures the blue dots represents measurements with high reectivity

Figure 7.5: The right picture is a close-up of the front reex, the blue dots and the read dots on the bucket appears to be in dierent planes, this is a measurement error due too the reex, they are actually measurements form the same surface. The blue line is generated from only the blue dots the yellow line is generated from the read dots (belonging to the bucket). The purple line is a mean from the two lines slopes and this is the output from the algorithm. The left picture shows the entire scan also the path to the hauler, calculated in 7.4.1. In both pictures the blue

(45)

7.3 Vision

7.3.2 CSAM Line Detection Algorithm (WP 3)

This algorithm is used in the nd hauler algorithm when only one marker is found, it then used to extract points that belongs to the at bucket surface. The algorithm extracts lines from scan data, it one of the most popular algorithms for line tting in scan data. All three parts of the algorithm draw benet from the natural sorting of the scan data, the data is stored in an array and the index in this array represent the scan angle.

The Cluster part of the algorithm groups the values into smaller arrays, the algorithm implements a simple version of a successive line following algorithm. It splits the array if the trend of the points in the array makes a signicant change. By a trend we mean that a point have a similar distance to the next point as is has to the previous also the direction of the distance vector is in the same direction.

The Split part of the algorithm uses a least square minimization algorithm to t a line to the set of points it have received. It then locates the point with the maximum distance to this line. If this distance is greater then a threshold the algorithm then checks if the next to this point is a point on the same side of the line who's distance to the line also is greater then the threshold. If this is not the case the point is rejected as a noisy point, the algorithm now repeats this step with the previous point excluded. If such point is found the algorithm splits the array in two and repeat the split algorithm from the beginning. If no such points are found the line is accepted, the algorithm then calculates endpoints for a vector on this line and returns it.

When the Split algorithm has nished the Merge algorithm is called. This algorithm takes the set of vectors that the Split algorithm generated and merges collinear vectors. The algorithm takes the most distant points of two vectors and creates a new vector from these if the other two endpoints lie within a threshold from this vector the new vector replace the rst. The threshold is weighted by a sine function that takes the new vectors length as argument, so for a short new vector the threshold is small while as for a long vector the threshold approaches the set threshold.

(46)

7.3 Vision

7.3.3 Vision System Drivers and APIs (WP 2)

These components where all part of the scope of the previous Vision thesis, but the developed software was insucient. The communication driver developed for the scanner was based on xed telegrams dened as constant arrays, an thus it lacked any kind of dynamic communication. The API allowed for selection between collecting a congurable number of scans or a single scan, the implementation was integrated in the control structure of a WIN32 window application. The API for the servo controller was also integrated in the same WIN32 window application and was a sequence of commands that moved the actuator between two congurable angles. No eort at all was made to solve the synchronization of the scanner and actuator as one unit. Below follows an overview of the most important methods of our implementation.

7.3.3.1 LiDAR Scanner Driver Methods

The LiDAR scanner is in most applications used as a stand-alone unit. The scanner in these applications via three digital outputs indicate whether there is or is not an object in three dierent user dened elds. These dierent elds are congured from a PC with a supplied software. In our application we need the raw scan data in the form of measured distances from the scanner. The rmware of the scanner allows several dierent congurations for how this data is scanned and processed such as resolution, scan angle, real-time indicators, reection value etc. To enable easy access and conguration of these settings and for data collection we needed to develop a communication driver and an API for the scanner. The structure of a standard telegram that is used for communication with the scanner

is described by tableA.6in sectionA.1. All telegrams transmitted to the scanner

are if the CRC sum is correct conrmed by an acknowledge telegram hex 0x06. If the CRC sum is incorrect the scanner responds with a not acknowledged telegram hex 0x15. If the telegram sent to the scanner has a correct syntax the scanner performs the action described by the telegram and sends the according response telegram. The methods below wrap the data link layer into a more easily used transport layer.

(47)

7.3 Vision Build Telegram A telegram shall contain a xed header with scanner address and STX byte, the header shall also include the length of the telegram, and it must also end with a CRC sum. The input for this method is the telegram identication byte and the dened data eld for the telegram; this method then adds the correct header and check sum.

Read Telegram This method checks if the read buer contains more then zero bytes, if it dose it reads one byte, if it does not it returns false. If the read byte is Ack or NAck this is returned by the method. If the read byte is STX the method waits until the length of the buer is equal to or more then the length of a header. The header is then read and the total length of the message is read in the header, the method now waits until the entire message is in the buer. The complete message is read and the check sum is calculated, if the check sum matches the telegrams check sum the telegram is returned. If the rst byte read is none of the above or if the check sum dose not match or if the length of the telegram is out of limits the algorithm enters recovery mode. In this mode the method locates the next telegram header, it then deletes all data prior this header and then restarts the procedure.

Write Telegram The input for the method is a telegram, the method sends the telegram to the scanner. When this is done the method calls the read method, the expected response from the read method is an Ack telegram. If this is the case the read method is called again, this time the expected response is a telegram. If the read method returns the proper telegram the write is complete and this telegram is returned. If neither Ack nor NAck is received or an Ack but no response telegram is received the method will resend the telegram, the method have a dened limit of how many attempts it will make before it returns that it have failed. If an NAck is returned from the read method the syntax in the telegram is incorrect and the write method returns false.

Calculate CRC Sum This method simply calculates the CRC sum of the telegram as is described by the data sheet, this method is private and is used by the build telegram and read telegram method.

(48)

7.3 Vision Initialization This method initiates the communication with the scanner ver-ies that standard resolution and scan angle is set. If requested it can also reset the EEPROM settings of the of the scanner to our dened standard conguration, this option should be used with care as the EEPROM have a limited number of write cycles.

7.3.3.2 LiDAR Scanner API Methods

These methods wrap the scanner communication and conguration for the most frequently used operations into an API.

Congure Scan Mode The scanner allows settings for dierent resolution both in delta scan angle and in measured values, the scanner can also be con-gured to scan 180 or 100 degrees. The delta scan angle can be set to 1, 0.5 or 0.25deg, the output of the measured values can be congured to be sent in mm or cm. The scanner also allows for dierent conguration of the output data such as eld indication and measured reectivity, information. This method allows easy conguration of these parameters, the inputs are two enumeration variables one that describes the scan angle and delta scan angle (as only certain combinations are allowed) and one that describes the conguration of the output data.

Congure Operation Mode The scanner rmware allows for several dierent operation modes, this method allows easy changing of operation mode. The most common operation modes we use:

Scan Frame at Request, as the name suggest the scanner scans a frame when its told to do so.

Scan Frames Continuously, in this mode the scanner automatically sends all scan frames.

Scan Interlaced Frames Continuously, this mode is identical to the mode above accept that start angle of next scan is shifted 0.25deg, in a loop of four steps.

(49)

7.3 Vision Installation Mode, this mode is required to make changes in the scanner

conguration parameters.

7.3.3.3 Maxon Servo Controller API Methods

The servo controller manufactured by Maxon is supplied with a DLL that wraps the communication layer into more abstract read object and write object methods. The servo controller is general used as a CAN.Open node but in this project we are communication with RS232, but the methods in the DLL have many similarities to the CAN.Open protocol. The read and write method inputs an address, sub-address and a pointer to a 8-32bit int (depending on the specic object) for data. Besides these two functions there are for every object in the controller also a wrapper function. These functions omit the address and sub-address, instead the name of the functions and its in parameters tell the user what object that will be set or get. There is also a function that initiates the communication with the controller. To make the servo controller easier to use we have built and API upon this driver, the dierent methods in our API is described below.

Servo Initialization This method has three dierent modes initialization, nd home and reset EEPROM. The user may choose to run any combination of the dierent modes, but the two last will fail if initialization have not been performed. The rst mode opens the communication, enables the actuator, sets software angle limits, clear errors and sets standard regulation mode.

Find home congures homing setup and starts the homing sequence (resetting the angle encoder).

Reset EEPROM congures all standard settings used in the project, such as tuning parameters for regulators and actuator parameters. This method should be used with care as the EEPROM has a limited number of write cycles.

Move To Angle Moves the actuator to a desired angle with the highest allowed speed and acceleration.

(50)

7.3 Vision Target Reached The servo controller has a setting for a angle window, when the actuator have a set point this window determines when the target reached bit in the command word is set. This method reads this bit in the command word and returns true when it is set.

Prepare Constant Speed Move Input for this method

Two angles, start and stop angle for the actuator Desired time in milliseconds

The method calculates, from these parameters, the desired speed of the actuator and the required distance needed to accelerate to this speed. The actuator is then moved to the new start angle, that is the acceleration distance plus the requested start angle. A warning will be displayed if the requested speed cant be reached before start angle is reached, or if deceleration will start before stop angle is reached, also if the desired speed is out of limits.

When the actuator has started to move towards the start angle the control is returned to the caller. The calling task will then have to use the target reached method to determine when to call continue constant speed move. This so that the calling task wont be blocked while waiting for the actuator to reach the start angle.

Continue Constant Speed Move This method needs no further input, but it requires that a constant speed move have been initiated. The actuator will now be set to continue its movement towards the new stop angle, the deceleration distance plus the stop angle.

Collect Captured Positions Algorithm The servo controller and the scan-ner are synchronizes by an external signal (you can read more about this in

(51)

7.3 Vision interrupt the servo controller stores its current angle. This captured angle is as-sumed to represent the horizontal angel of the scanner at the start on the latest scan. The servo controller has four registers that contain information about cap-tured angles. Three of the registers contain the three last capcap-tured angles, the angles are stored according to a FIFO algorithm. The fourth register is a counter that is increased every time a new angle is stored, so with this counter we can identify the angles. The diculties with reading these angles arise due to the fact that only one register can be read at a time, and to read one register takes approximately 4ms. The problem with only being able to read one register at a time is that the counter has to be read both before and after the angle is read. This to verify that another interrupt has not occurred when we read the angle, if this happens we may not have read the desired angle but the next.

So at rare occasions when the OS is concerned with other tasks and an in-terrupt occurs during a read, the long read time may lead to that angles are overwritten before we have had time to read them. If this happens the algorithm reports how many angles that have been lost so that the calling method may act accordingly.

As the signal toggles at a completed mirror rotation and the controller only is interrupted at a positive edge, the angle only is stored for every other mirror rotation. The controller may be congured to interrupt on both positive and negative edge but at this sample rate we do not have enough time to read the angles. Instead this algorithm interpolates the angle for the scan line in between, if a previous angle is available. If this algorithm is not called often enough and the FIFO queue in the controller starts to ll up the algorithm, when called, also reads the next angle and stores it in a buer. If there still is a angle to read in the controller, the algorithm reports this, the calling method should upon this signal call the algorithm again as soon as possible so that the angle wont be lost. If an interrupt occurs during a read the algorithm makes an immediate second attempt to read the angle it will also read the next angle to the buer. If a second interrupt occurs now the algorithm reports that it failed and that there are angles to read.

(52)

7.3 Vision Reset Position Marker Counter The angle marker counter is a 16bit register so it has a maximum value of 65536. This register is increased at 37.5hz; this means the register ips around to zero every 29minutes. To avoid a few special cases that can arise when this happens, the user should use this method prior to a move that will use the collect captured angles algorithm.

7.3.3.4 Vision System API Methods

Create Sweep Point Cloud A sweep point cloud is generated by setting the scanner to scan continuously, and while scanning tilt the scanner around its horizontal axis at constant speed.

Input for this method

Two angles, start and stop angle for the actuator Desired delta angle between the scans

Desired resolution of the scan

During the sweep, every scan is collected from the scanner and stored together with its corresponding scan angle collected from the servo controller. Counters in the collected scan and scan angle enable matching of the data. The scan angles are stored in a FIFO buer of length of three in the servo controller (the angles are stored on a trigger signal the servo controller receives from the scanner you

can read more about this in chapter:7.3.3.3), a read from this buer has to be

introduced by the host. The scanner transmits all scans automatically to the host, the receive buer for the scan data can store up to 25mb of scan data. Due to this fact the method prioritizes the collection of scan angles and let the buer handle the scans, so that no angles are lost. If one or more angles are lost, the method estimates angles for these scans with interpolation. The method will not do interpolation if the rst angle is lost, if the last angle is lost extrapolation is used. If a scan is lost during the sweep a variable indicates that it is not a valid scan.

(53)

7.3 Vision Create Discrete Sweep Point Cloud A discrete point cloud has the same input as a sweep point cloud the dierence is that the scanner is not moved during the scanning. Instead of moving the actuator at continuous speed while the scanner scans, the actuator is moved in steps. At each step the actuator stops and a request is sent to the scanner for a scan, when the scan is complete the actuator moves to the next position.

(54)

7.4 X PC Host (WP 2)

This component is an abstraction layer for the SimuLink machine control model developed in the rst thesis, hereafter in this chapter referred to as the model. The model supply basic functions like on/o and ramp control for some actuators, in this layer we have developed congurable methods that wrap these functions into more advanced patterns. As example cranking the engine and conguring the HMI (our initialization process), or translating a desired distance and at the same time actuating the bucket to a desired angle. Below we have listed the functionality our mission requires from the X PC Host component.

Instantiate the machine (crank engine, congure HMI) Stop the machine

Actuate the lift angle Actuate the bucket angle

Perform a bucket empty sequence at a desired target (translate, lift, tilt and brake in a dened pattern adapted to the given target)

Perform a reverse from load carrier sequence at to desired target (translate, lift, tilt and brake in a dened pattern adapted to the given target)

Perform a bucket ll at a desired target (not in the scope of this thesis) Perform a reverse from bucket ll to a desired target (not in the scope of

this thesis)

These functions can be requested from the component by the mail system. Each function is implemented as a state in the state machine of this component, in these states we access the functionality of the model through an API we have developed. As some of the states have much in common this made reusing and maintain code more manageable, we call this API the wheel loader class.

(55)

7.4 X PC Host (WP 2)

7.4.1 Wheel Loader Class

This class is the layer above the communication layer (section 7.4.2) , this class

contains methods with self explanatory names that hides all the code involved in reading a signal or writing a parameter in the model. Some functions in the model require a sequence of commands to be activated, these sequences have also been wrapped in these methods. We have also created more readable enumerations for the value of the signals and parameters that have a discrete nature, that is used as input and output from the methods. This also protects the user from setting a value that is not applicable in the model. The list below explains the functionality of these methods.

InterruptMovement, this method has two modes stop movement and abort movement, the rst stops all movement with a deceleration the second stops all movement immediately

LiftRamp, sets the lift ramp parameters

LiftRampComplete, returns true if a lift ramp have been initiated and the lift angle have reached its target

TiltRamp, sets the tilt ramp parameters

TiltRampComplete, returns true if a tilt ramp have been initiated and the tilt angle have reached its target

WaistRamp, sets the waist ramp parameters

WaistRampComplete, returns true if a waist ramp have been initiated and the waist angle have reached its target

ThrottleRamp, sets the throttle ramp parameters SBC, sets the Speed Brake Control parameters LiftStep, sets the lift step parameters

Autonomous Hauler Loading

Autonomous Hauler Loading

Abstract

Autonomous Hauler Loading

Jonatan Blom

Abstract

Contents

List of Figures

Nomenclature

Chapter 1

Introduction

Chapter 2

Background

2.1 Wheel Loader

2.3 Autonomous Machine Project at Volvo CE

Chapter 3

Thesis Description

3.1 Demonstration

3.1.1 Simplied Short Load Cycle

Chapter 4

Platform

4.1

4.2 Bucket Fill Regulator

4.3 Vision

Chapter 5

Related Work

5.1 Yamazumi No.4

Chapter 6

Problem Formulation

6.1 Short Load Cycle Analyze

6.2 Work Packages

6.2.1 WP 1 Mechanics

6.2.2 WP 2 Software Platform

6.2.3 WP 3 Software Algorithms

6.2.4 WP 4 Live Demonstration of Prototype

Chapter 7

Software

7.1 Structure and Design (WP 2)

7.1.1 Software Components

7.1.2 Inter Thread Communication

7.1.3 GUI and Console

7.1.4 Execution Control

7.2 Mission State Machine (WP 3)

7.3 Vision

7.3.1 Find Hauler Algorithm (WP 3)

7.3.2 CSAM Line Detection Algorithm (WP 3)

7.3.3 Vision System Drivers and APIs (WP 2)

7.4 X PC Host (WP 2)

7.4.1 Wheel Loader Class

3.1.1 Simplied Short Load Cycle