View Planning for Objects Modeling using UAVs

(1)

IN

DEGREE PROJECT COMPUTER SCIENCE AND ENGINEERING, SECOND CYCLE, 30 CREDITS

,

STOCKHOLM SWEDEN 2017

View Planning for Objects

Modeling using UAVs

MICHAEL C. WELLE

KTH ROYAL INSTITUTE OF TECHNOLOGY

(2)

View Planning for Objects Modeling

using UAVs

MICHAEL C. WELLE

Systems, Control and Robotics Date: December 17, 2017 Supervisor: Patric Jensfelt Examiner: Hedvig Kjellström

(3)

i

Abstract

(4)

ii

Sammanfattning

(5)

Introduction

The field of UAVs has gotten more and more attention in recent years. The availability and commercializing of Unmanned Aerial Vehicle (UAV) decreased the price per unit significantly. This sparked a huge interest in the research community. A modern UAV right of the shelf, like the "Parrot Bebop" [1], shows how far the development has come in recent years. The Bebop can hold its position and altitude in a satisfying manner and allows for accurate manual control, correlating with the pilots experience.

UAVs can be used in many different fields, for example as surveillance gear, inspec-tion of forests, construcinspec-tions sites. Another big applicainspec-tion sector is video recording in extreme sports, outdoor festivals and many other similar applications like drone races. The RPL department at KTH researches mobile robots regarding long time autonomy and data collection. A UAV, with its high mobility and ability to easily avoid ground obstacles has the potential to highly enrich this field. For the UAV to make meaningful contributions, a wide variety of techniques from different topics need to be combined and applied. The topics range from estimating the position in GPS-denied small environ-ments, over mapping those, sometimes partially dynamic, environments over to finding and/or to identify objects while operating in a safe manner.

This work focuses on the view planning problem (VPP). The VPP is a problem closely connected to path planning. Path planning is concerned with how to get from A to B, without colliding with obstacles in an autonomous, preferably optimized, way. View planning takes the field of view of a camera into account. The goal of view planning is to find a path and corresponding movements, which enables the camera to acquire the desired views.

An UAV has nearly omni-directional movement capabilities and can therefore posi-tion itself in nearly arbitrary ways1. If the UAV is equipped with an actuated camera, view planning gains access to even more degrees of freedom.

The method presented is kept as general as possible, to make a transfer to other fields or platforms possible. The experiment conducted to evaluate the principal method pre-sented addressing the VPP is performed in simulation and while done with a UAV, can be transferred to other platforms. The experiment performed on a real UAV are

con-ducted to show possible real-world usage. The underlying assumption is that a pre-existing map of the environment is provided and can be used in a safe way to avoid hitting

ob-1_{While a UAV can assume arbitrary posisitions in XYZ (given that there are no obstacles), it can only be}

ro-tated around the Z-axis (yaw angle) in a stable manner. A typical UAV cannot assume arbitrary rotations with respect to the roll and pitch angle.

(8)

2 CHAPTER 1. INTRODUCTION

stacles. Furthermore, the positioning problem is addressed by a GPS-sensor in the sim-ulation, and a Motion-Capturing System for the real-world trial. Additionally an initial position of the target object as well as approximate dimensions are given to the method. An example of a system that could produce this is given by related work such as Am-bru¸s et al. [2]. In this report, we focus on a solution to the view planning problem and how to execute the generated path given the above assumptions. The following research question encapsulates this.

1.1 Research Question

Given a 3D model of the environment with partial objects identified, how to plan and execute a path for a drone such that it acquires the best possible data to build a model of the objects when the uncertainty of a UAV needs to be taken into account?

1.2 Report Outline

(9)

Chapter 2

Related Work

To get a broader sense of the work already published in relevant fields, we first look at methods presented by the RPL department at KTH, to priovide the context in which our work fit and the work we build upon. Next a wider range of applicable research is re-viewed and considered, after which we zoom in on information based approaches and work directly done with UAVs.

2.1 View Planning at large

Ambru¸s et al. [2] present a method for how to build and maintain a so called "Meta-room". The Meta-room is used to recreate the static structures of cluttered office envi-ronments. In order to map dynamic environments and identify possible objects a novel method identifying changes between different instances of recorded point-clouds and ex-tracting clusters of dynamic objects from those observations is presented. The method is structured in such a way that it allows the data to be processed sequentially and there-fore later measurements can be incorporate with ease. This work also presents a way to re-identify previously identified objects over a longer time frame. In later work Faul-hammer et al. [3] build upon this method to autonomously learn object models. The pre-sented system can detect the object in a real-world scene autonomously using a mobile robot equipped with an RGB-D camera on a pantilt unit and plan paths to gather addi-tional views of the object. This is an example of a view planning problem (VPP). Faul-hammer et al. fuse the different clouds obtained from the viewpoints in an effective way to store them in an octree structure. In their work also discuss the problem of objects ob-structing the path of the robot as the mobile platform they use is quite large.

Scott and Roth [4] provide a meta study on the topic of view planning for automated three-dimensional object reconstruction and inspection in 2003. They compare and evalu-ate different methods and aspects and concluded that: “... view planning for high-quality object reconstruction/inspection has no general-purpose solution." Furthermore, their work suggests that the way to go forward is in combining multiple techniques as well as fusing multiple sensor inputs to enhance and achieve high-quality object reconstruction. Li and Liu [5] propose an information entropy-based approach to the VPP. Having a met-ric in place that measures the information gain for a possible next view point of a cam-era, they can pick the Next Best View out of a set containing all possible next viewpoints. The Next Best View is a special case of the View Planning Problem that heavily focuses on how the "next" view should look. One can string a series of Next Best Views together

(10)

4 CHAPTER 2. RELATED WORK

to get an answer to the View Planning Problem. They use a robotic arm equipped with a camera to acquire views with a CCD camera and therefore their approach is slightly different from the ones using an RGB-D camera. Scott [6] focuses on model-based view planning employing laser scanners and considering the view planning as a set covering problem. This makes it possible to tackle the problem with a greedy search algorithm[7] in an effective way. The approach used by Scott uses a multi-stage technique. First a coarse scan is conducted, the information gained in the first stage is used to perform a more precise scanning in the next stage. This reduces the complexity greatly and shows the value of model-based view planning approaches.

A twostep approach is also presented by Blaer and Allen [8] where a mobile robot is tasked with mapping an area. The first step requires a 2-D map which is used to plan covering views. The mobile platform uses a laser scanner to acquire scans at those first viewpoints. Later the map is translated into a 3-D voxel-based occupancy map. Blaer and Allen use this map to calculate Next Best Views in order to obtain a complete model of the environment.

In more recent work Lin et al. [9] present a method for planning and searching for an occluded target with a robot arm. They employ A* search with tree node sampling to find the object. Their work is a good example where the robot has active influence on the environment and can change the environment by moving objects out of the way. In the case studied by this master thesis we have no such influence and must consider this constrain.

Kaba et al. [10] present a reinforcement learning approach in order to arrive at a so-lution for the view planning problem. They employ a novel scoring function which

ex-ploits the geometry of a 3-D model. By using reinforcement learning methods like SARSA [11], Walking-Q [12] and TD [13], they can arrive at a solution to the view planning problem

which is cast as a Markovian Decision Process. In their result they show that this method is able to slightly outperform the usually used greedy search approach.

(11)

CHAPTER 2. RELATED WORK 5

2.2 View Planning with UAVs

The View Planning Problem being approached from the domain of UAVs brings many advantages with it, like the additional degrees of freedom a UAV can take advantage of in contrast to a wheeled platform or a stationary arm. Those advantages come how-ever with a price, as a framework for those platforms is usually more complex. [18] lays down a hardware design and implementation scheme that addresses many of the prob-lems arising from the engineering side.

Work directly done using UAVs comes from Schmid et al. [19] where they show how to use autonomous multicopter to build a 3-D model of outdoor structures. As their work is performed on outdoor structures they re using GPS to position the UAV. The work also focuses on model based view planning. Peng et al. [20] also discuss the setup of a UAV outside but add an obstacle avoiding system. They make use of a visual guidance system that extracts the optical flow from a series of images. Similarly, Jing et al. [21] em-ploy a freely available 2-D map of buildings to construct a rough 3-D model, inspect and refine it with a UAV. Bircher et al. [22] focuses on structural inspection in their work pre-sented where they employ a two-step optimization paradigm to find good viewpoints. The mentioned work is done in outdoor environments and the object model or inspected area is in general larger than what the target of this report is.

In the context of GPS-denied environments, Lai et al. [23] present an online path plan-ning approach in cluttered environments. The presented method plans a path first and evaluates the feasibility in the second step by dividing the problem into many two-point boundary value problems which can be solved in closed form.

The information theoretic approach, where a metric is put in place to measure how much information was employed by Wenhardt et al. [24]. This is the basis that the work in this report is built on as well. In Wenhardt et al. a camera, held by a robotic arm, is positioned so that the resulting reconstruction error for an object model is as low as pos-sible. This is made possible using the employed extended Kalman filter, which enables Wenhardt et al. to select the Next Best View with the least expected entropy.

Employing the information gain as well are Isler et al. [25]. A mobile robot equipped with a camera evaluates the expected information gain from a discrete set of possible views. They incorporate a visibility likelihood as well as the likelihood of seeing new parts of the object into their evaluation. The authors make a great effort to keep the func-tions modular to allow a more universal application of their work.

Calculating Information gain, where a huge part of the environment is still unknown requires the usage of probabilistic representation of said environment. Daudelin and Campbell [26] present an adaptable probabilistic, Next Best View algorithm. In their work they do not need any knowledge about an object a priori. Daudelin and Campbell em-ploy an information-theoretic cost function which considers self-occlusion and enables them to build a whole model of an before unknown object with a relative small num-ber of views. In addition, the algorithm provides natural termination, when the object is completed and does not need to rely on human intervention to stop.

(12)

es-6 CHAPTER 2. RELATED WORK

timate for the objects size. In [27] , we showed how a fully autonomous pipeline can be used to acquire views of an object and build a corresponding model. This thesis presents an extension to the work presented in [27].

The work by Potthast and Sukhatme [28] focuses more on the evaluation of said view-points. Potthast and Sukhatme introduced a believe model which provides a better rep-resentation of occluded space and therefore a more precise reprep-resentation of the potential information gain. In addition, Potthast and Sukhatme use Iterative Closes Point, as well as Markov fields for their map update which results in a better and more accurate repre-sentation of the world as holes in object models are automatically filled by the algorithm.

(13)

Chapter 3

Background

In this Chapter, the reader can acquire the necessary theoretical knowledge to fully un-derstand the report. Readers that are already be familiar with the topic may safely skip this chapter.

The system makes use of a wide variety of different techniques that are state-of the art and explained in more depth elsewhere. This listing restricts itself to only the rele-vant parts regarding the topic of this report. The interested reader can follow the given sources to get a broader understanding of the discussed topics.

3.1 Probabilistic Representation of an Environment

To make autonomous decisions a robot needs a model to represent the world. There is a great variation of possible world representations [29]. For this project, a special kind of 3D occupancy-grid map was used, a so called Octomap [30]. The Octomap is a proba-bilistic mapping based on octrees. The great advantage of it is that it can cluster different 3D cells, also called voxels, with the same, or very similar values and represent it with a single bigger voxel. This allows for an elegant way to trade precision versus memory. This concept can be seen in Figure 3.1.

Figure 3.1: Concept of an Octomap (adapted from [29])

The probabilistic nature of the map allows it to incorporate the uncertainty of the sen-sors and other sources of uncertainty into the representation of the world. A huge

(14)

8 CHAPTER 3. BACKGROUND

vantage of this representation is that it allows to model free space as well as unobserved space explicitly while being efficient in memory usage.

The map gets initialized with cell probability P (cell) = 0.5 which indicates an unob-served/unknown cell. The probabilities of the cells are update with every new measure-ment that gets incorporated into the map using Equation 3.1.

The framework comes with an integrated ray caster based of a 3D variant of the Bresen-ham algorithm [31].

Those features combined with a freely available c++ library makes the Octomap suit-able as world representation for this project.

3.2 3D Sensor Model

In order to evaluate a given set of viewpoints one needs a model of the used sensor to evaluate said viewpoints. In the case of an RGB-D camera a viewing frustum (a cut of pyramid) is usually used as shown in Figure 3.2.

Figure 3.2: A viewing frustum (adapted from [32])

A frustum can be defined by six points (left, right, top, bottom, near, far). To calculate a frustum, one needs the image width (i_w) and height (i_h), the minimum (min_vd) and maximum (max_vd) view-distance as well as the projection matrix P.

P =   fx 0 cx Tx 0 fy cy Ty 0 0 1 0   (3.2)

The six points can then be calculated using equations 3.3 - 3.8: lef t_{= min_vd ·} (cx− i_w)

fx

(3.3) right_{= min_vd ·} (i_w − cx)

fx

(3.4) top_{= min_vd ·} (cy− i_h)

fy

(3.5) bottom_{= min_vd ·} (i_h − cy)

fy

(3.6)

near= min_vd (3.7)

(15)

CHAPTER 3. BACKGROUND 9

3.3 Ray Casting

To simulate a camera view given a probabilistic map one must simulate what the camera would see given a position. The common way to achieve this is by ray casting like de-scribed in [31]. The more rays are cast the more accurate the estimated view will be, but every ray takes additional computational resources. A common solution is therefore to not cast a ray for every pixel the camera might receive but to only calculate a fraction of those rays.

Ray casting is mostly used to render scenes in computer generated environments, a basic implementation can be seen in Algorithm 1.

Algorithm 1Ray casting algorithm (adapted from [33])

1: Image RayCast(Camera camera, Scene scene, int width, int height) 2: {

3: Image image = new Image(width, height); 4: for(int i = 0;i < width;i++){

5: for(int j = 0;j < height; j++){

6: Ray ray = ConstructRayThroughPixel(camera, i, j); 7: Intersection hit = FindIntersection(ray, scene); 8: image[i][j] = GetColor(hit);

9: } 10: }

11: return image; 12: }

In the case of evaluating viewpoints one can use the cell value stored in the proba-bilistic map in order to infer the information gain of said view point.

3.4 Path Planning and Execution with a UAV

A UAV has more degrees of freedom than a typical ground robot but comes with more engineering challenges. [34] presents a good overview of how a UAV framework can be realized. The specific realization of the UAV configuration and framework used for this work can be seen in the appendix.

Figure 3.3 shows a generic framework. The UAV needs to have some way to know its own position, some method to know how the world looks like as well as how to con-trol the rotors to follow a desired path. For finding the path a variety of ways can be employed, from simple proximity checks to planning algorithm like A* [35] or Rapidly-exploring random treeS [36]. The resulting movements depend on the used control strat-egy, where also a wide variety of methods can be employed. From a standard PID-controller to newer model predictive control2

(16)

10 CHAPTER 3. BACKGROUND

Figure 3.3: General UAV framework.

3.5 Object Modelling

Object modelling can be roughly divided in two separate camps. The first requires the camera pose to be known and combines the different views using the position informa-tion of the camera as a starting point and techniques like Iterative closest point (ICP) [37] and Random sample consensus (RANSAC) [38] to acquire the final model. As in a lot of use cases the pose of the camera is either not accurate enough or not accessible at all to be useful. In recent years a number of interesting solutions has emerged most of them using SIFT [39] based approaches and algorithms like RANSAC to separate the models from the background. Examples of 3D modelling tools are the online based tools "recap360"[40] from "Autocad" and "Sketchfab"[41] which can be used to generate 3D models given 30-40 pictures taken by a camera. Figure 3.4 shows a model which was generated by taking 35 pictures with a DSLR-camera and processing them with recap360 + Blender[42] + Sketchfab.

(17)

Chapter 4

Method

The presented method provides an answer to the research question on how to find a Next Best View in a consecutive fashion to model 3D objects. As discussed in the re-lated work section, many approaches focus on the evaluation of the Next Best View or on the modelling aspect. The focus of this report is mostly on how to find views which contribute the most to modelling the object by refining initial views. We extend our own work presented in [27] where the focus was on how to do the actual object modeling and the view planning was simplistic. We represent the information acquired by the sensors about the world using an Octomap [30]. This is what is used to make decisions about the next view.

4.1 Method overview

We begin to give an overview of the method and then provide more details for the dif-ferent parts. The method consists of three main parts:

• Initial view generation • Evaluation of views • Optimization of best view

The method (displayed in Figure 4.1) requires an approximate pose as well as the rough dimensions of the target object. That information can be acquired using a mobile ground robot as in Faulhammer et al., for example.

To start with, initial views are generated around the target object given the camera parameters as well as the target’s approximated pose and dimensions. We generate the viewpoints on a circle around the object with a height and radius that depends on the pose and dimension of the object and the camera parameters. These viewpoints are then send to the evaluation function which scores all viewpoints using a scoring method that rewards the acquisition of new information highly given a target region enclosing the object. In our work the target region is a cuboid which is 1.5 times the size of the rough dimensions given at the start.

The viewpoint with the highest score is send to an iterative optimization scheme. There the program generates neighbouring viewpoints and evaluates them. This iterative approach continuous until no neighbouring viewpoint with a higher score is identified.

(18)

12 CHAPTER 4. METHOD

The result is a view point that is placed on a local maximum of the information gain function.

The UAV gets the command to move safely to said viewpoint to take a point cloud snapshot, traversing the circular view points on the way. This snapshot is saved and

used in order to update the Octomap that constitutes the target volume. If there are enough unknown cells inside the map the method will evaluate the circular viewpoints again to pick a new starting point for the iterative optimization. This procedure repeats itself until the map is sufficiently updated and the information gain is close to zero or another met-ric specified by the operator is fulfilled. At this point the UAV will land and the object modelling phase will begin using the saved snapshots.

Figure 4.1: Information based view planning with iterative optimization for object mod-elling. A list of view poinst is generated in the top left corner along a circle around the object. The blue thick arrow show the main part of the algorithm where the initially gener-ated view points are used to generate Next Best Views which are visited. Finally, at the end, the model is generated based on the collected views.

(19)

CHAPTER 4. METHOD 13

4.2 View point initialization

The viewpoint initialization is based on our previous work in [27]. The viewpoints are generated on a circle using the method shown in Figure 4.2.

Figure 4.2: Generating circular viewpoints (adapted from [27])

Here r is the radius and h the height of an enclosing cylinder of the target object. The field of view of the camera is denoted by α and the tilt of the camera on the UAV is named τ . The parameter β is a margin to counter positioning errors of the UAV. The height relative from the center of the object Z as well as the distance from the center of the object R can be found with the help of t−= tan(τ − (

α 2− β)) and t+= tan(τ + ( α 2− β)) as: Z = t+(2rt−+ h) t+t₋ (4.1) R= Z t+ + r (4.2)

The number of viewpoints n defines how many viewpoints are evenly created around the target. As the initial view point dictates the yaw direction, n dictates also the number of different yaw angles the method evaluates. A higher n gives a wider range of yaw angles but also means a longer evaluation time. The computational time grows linear for the evaluation of viewpoints.

4.3 Evaluation of views

To compare different viewpoints to each other a metric is required. The typical informa-tion gain which is based on the measurement of entropy left after a view has been taken. The gain is computed as:

(20)

uncer-14 CHAPTER 4. METHOD

tainty. The Octomap models the world probabilistically. Each cell can have a value rang-ing from 0 (definitely free) to 1 (definitely occupied). A cell value of 0.5 means we have the highest uncertainty. If our view reveals those cells, the score should get the highest increase while cells with values close to 0 or 1 should receive a lower score. The method also restricts the evaluation to the target volume to avoid that the UAV starts exploring the rest of the world (Figure 4.3).

Figure 4.3: Example of a enclosing target volume

The target volume is evaluated using a 3D-sensor model based on the camera param-eters and rays are cast from the viewpoint origin. Those rays traverse the cells of the Oc-tomap and score them with a Gaussian function with µ = 0.5 and σ = σ0 which fulfills

the requirements listed beforehand.

The score gets calculated for each traversed cell inside the target volume with: score+ = _√₂ 1

2πσ2 · e

−(x−µ)2_2σ2 _(4.4)

where x denotes the current cell value. Figure 4.4 shows the scoring function graphically.

Figure 4.4: The scoring function

If the traversing ray encounters a cell with a value > T the ray is stopped as it is as-sumed it encountered an occupied cell and could not traverse any further.

(21)

CHAPTER 4. METHOD 15

4.4 Optimization of best view

The motivation for this step comes from the assumption that there exists an unknown information gain function IG which describes the information gain for the entire set of possible viewpoints S. The goal is to find the viewpoint contained in S with the highest information gain. We use the heuristic that this maximum can be found starting from the initial view point with the highest score.

best_view = max(IG(S)) (4.5)

If the function IG is continuous, there should exit a number of local maximas which would constitute good viewpoints. The iterative optimization goal is to end up at one of those local maximas.

The Next Best View of the circular viewpoints (evaluated with the scoring function) is the starting point of the iterative optimization method which is displayed in Algorithm 2.

Algorithm 2Iterative view point optimization; Input: best_view, start_score, step_size new_score =start_score

2: whilestart_score<=new_score do

new_views = generate_adjacent_views(best_view, step_size) 4: new_score, new_view = get_best_view_and_score(new_views)

ifnew_score>best_score then 6: best_score=new_score best_view=new_view 8: else break 10: return(best_view)

The "get_best_view_and_score" function evaluates each view with the previous dis-cussed scoring function and returns the best view as well as the corresponding score.

The "generate_adjacent_views" function builds six adjacent viewpoints to the current view point using the step_size variable and the placement shown in Figure 4.5:

Figure 4.5: Placement for six adjacent viewpoints, 0-5. The squares as well as the layers are one "step_size" away from each other.

(22)

16 CHAPTER 4. METHOD

4.4.1 Runtime improvement

The algorithm can be improved by saving the corresponding score for each view point and skipping the calculation for the view point that corresponded to the position of the UAV in the previous calculation. As the evaluation of a view point makes up the lion share of the computational time, around 16% of calculations can be saved this way. As the UAV should preferably perform those computations on-board this improvement is recommended.

4.5 Execution of the planned path

The circular viewpoints generated as initializations and the adjacent viewpoints are sub-ject to a proximity check given a map of the environment. Viewpoints that are not reach-able are rejected. If a suitreach-able viewpoint is identified, the UAV follows the path planning method shown in Figure 4.6.

Figure 4.6: Path planning for the UAV.

(23)

Chapter 5

Experiments

In order to validate the method and show its real-world usefulness two experiments are performed. First the iterative optimization approach is evaluated to see if the made as-sumptions hold up. Next the method is deployed on a real UAV and used to build object models.

5.1 Experimental setup

We used the following parameters in all experiments: • step_size=0.1m

• σ0 = 0.1

• T = 0.8

5.2 Evaluation of iterative optimization

The basis for the iterative optimization is the assumption that there exists a continuous information gain function which has local maxima that correspond to good view points. In order to see if those assumptions are correct, the first experiment is performed.

5.2.1 Setup

A target (see Figure 5.1a and 5.1b) is positioned at X = 0.0m Y = 0.0m Z = 1.0m. Next n= 24 viewpoints are created with the circular method. The given object parameters are: object_radius = 0.1m, object_height = 0.2 which results in a cubic target area with a = 0.3m. The "step_size" variable is set to 0.10m and for each circular view point a set of 2744 viewpoints are created in a cube around it encapsulating an area of 1.4m3

. In total, the score of 2744∗ 24 = 65856 different viewpoints is recorded. After the best view point out of the 65856 evaluated viewpoints is identified, the point cloud at the best view point is used to performed the update step. This procedure is repeated until the unobserved cells in the target area are not changing in any significant way. This view point data is saved as validation set and compared to the result of the iterative approach. While the validation set is generated in simulation, the simulation still employs a model UAV as well as an imperfect positioning system. The saved views thereby underlies the same

(24)

18 CHAPTER 5. EXPERIMENTS

fluctuations as on a real UAV. Following this step, the iterative approach is performed on the same target and benchmarked against the recorded validation set.

(a) Target on a stand in the simulation. (b) Experiment set up in the simulation.

5.2.2 Simulation results

Validation set

In table 5.1 the results of the validation set is presented. The table is structured in the following way: The first column is the consecutive-number of the viewpoint evaluated. The second column shows the number of cells that are undiscovered inside the target volume. In the third column, the total number of viewpoints evaluated is listed. The fol-lowing columns are showing the number of viewpoints which fall into the folfol-lowing cat-egories:

• A(0): score = 0

• B(0-50): 0 < score <= max_score · 0.5

• C(50-75): max_score · 0.5 < score <= max_score · 0.75 • D(75-90): max_score · 0.75 < score <= max_score · 0.9 • E(>90): score > max_score · 0.9

The last column shows the maximum score that was achieved. Note that this score only has meaning in relation with scores from other view points and is arbitrary other-wise. It can be seen as an index on how much information is still in the target area.

# VP # P(c)=0.5 # total VP A(0) B(0-50) C(50-75) D(75-90) E(>90) Max score

0 1872 65856 20228 38848 5552 1116 112 11844.6

1 428 65856 25983 37730 1922 200 21 5657

2 157 65856 31068 34090 607 80 11 4567.89

3 33 65856 35004 29857 849 120 26 2908.29

4 11 65856 33477 31557 743 62 17 2829.62

(25)

CHAPTER 5. EXPERIMENTS 19

The results are displayed in Figures 5.2 - 5.6 where the viewpoint of category B are blue with transparency = 0.02, category C is shown in yellow with transparency = 0.2, D is green with transparency = 0.6 and finally the category E view poses are displayed in red with transparency = 1. Viewpoints that achieve the maximum score are displayed with a bigger red arrow. To make the graphs more perceptive, two versions are plotted next to each other, where the left one contains all the viewpoints (except if the score of the viewpoint is zero) and the right one only contains viewpoints from the category D and E (viewpoints that achieved a score ranking at the top 25 %).

(a) Validation set displaying view poses of

cate-gory B,C,D,E (b) Validation set displaying view poses of cate-_{gory D,E}

Figure 5.2: Validation viewpoint set for view point # 0

As there were 4 best viewpoints a random one was chosen for the next iteration.

(a) Displaying view poses of category B,C,D,E of the validation set.

(b) Displaying view poses of category D,E of the validation set.

(26)

(a) Displaying view poses of category B,C,D,E of the validation set

(b) Validation set displaying view poses of cate-gory D,E

(a) Displaying view poses of category B,C,D,E of

the validation set (b) Displaying view poses of category D,E of the_{validation set}

(a) Displaying view poses of category B,C,D,E of

the validation set (b) Displaying view poses of category D,E of the validation set

(27)

Iterative optimization

The same experiment is now executed using the iterative optimization approach. Table 5.2 shows the recorded performance:

# VP # P(c)=0.5 # total VP A(0) B(0-50) C(50-75) D(75-90) E(>90) Max score

0 1872 60 0 0 33 14 13 11880.5

1 428 78 0 31 16 22 9 5724.82

2 129 55 0 36 10 8 1 2166.26

3 55 35 7 17 8 1 2 597.533

4 37 36 1 14 13 5 3 269.639

Table 5.2: Result of iterative optimization

As the plots for the iterative optimization are not as cluttered as the validation set, only one plot per viewpoint is provided in the Figures 5.7 - 5.7. Also, the transparency is set to one as there are not many arrows obstructing each other.

(a) Evaluated viewpoints of the iterative opti-mization VP # 0

(b) Evaluated viewpoints of the iterative opti-mization VP # 1

(c) Evaluated viewpoints of the iterative opti-mization VP # 2

(28)

(e) Evaluated viewpoints of the iterative opti-mization VP # 4

Figure 5.7: View pose # 0-4 generated with the iterative optimization method.

The initial number of circular viewpoints corresponds to the number of different yaw-angle configurations. Table 5.3 shows the performance of the iterative approach with dif-ferent numbers for n in the circular initializations to investigate the impact that a changed nhas on the method (# total VP = total number of viewpoints).

# VP # Circular initializations (n = 24) # Circular initializations (n = 18) # P(c)=0.5 # total VP Max score # P(c)=0.5 # total VP Max score

0 1872 60 11880.5 1872 49 11808.7

1 428 78 5724.82 432 59 5178.27

2 129 55 2166.26 128 55 2585.15

3 55 35 597.533 52 54 973.419

4 37 36 269.639 43 35 495.927

0 1872 42 11565.3 1872 36 11557.4

1 481 67 5645.03 669 60 6997.45

2 53 42 1109.06 135 24 1504.01

3 43 30 321.845 62 24 478.543

4 39 29 536.682 57 18 296.939

0 1872 35 11385.8 1872 37 10643.8

1 590 27 3590.48 361 49 3973.47

2 427 21 2996.06 285 55 2365.73

3 235 27 1523.96 154 25 271.02

4 205 51 2521.32 148 25 135.633

(29)

5.3 View Planning on a Real UAV

5.3.1 Setup

To evaluate the performance in a real-world scenario, a real UAV is deployed in a test environment. The test environment is a cage with dimensions 3x3x3m and a movable podium as in the simulation. A map of the test environment can be seen in Figure 5.8.

Figure 5.8: Map of the test environment.

On top of the movable podium the following objects are placed, one at a time:

Name Radius Height Target location

Cereal r= 0.0975m h= 0.28m X= 0.0m, Y = 0.0m, Z = 0.85m Fire extinguisher r= 0.09m h= 0.5m X_{= −1.0m, Y = −1.4m, Z = 0.25m} (Not on podium) Head statue r= 175m h= 0.16m X= 0.0m, Y = 0.0m, Z = 0.92m Dinos r= 0.08m h= 0.2m X_{= 1.1m, Y = −1.0m, Z = 0.83m} Router r= 0.16m h= 0.235m X= 1.2m, Y = 0.0m, Z = 0.91m

Table 5.4: Objects and specification used with real UAV.

The UAV is placed inside the cage and receives the dimensions and the targets posi-tion. The view planning then performs the previously described steps, starting with the circular initialization with α = 45◦_{, β}_{= 10}◦ _{and τ = 30.5}◦_{. After identifying the Next Best}

(30)

iterative optimized viewpoint, the UAV takes additional snapshots of the current point-cloud and saves them as "help-point-clouds". When reaching the target view point the UAV saves the point cloud as a "main-cloud" and updates the map using this "main-cloud". The number of initial viewpoints is set to n = 12 as the previous experiment showed that 12 initial circular viewpoints are sufficient. The experiment is ended when the UAV has acquired 3 main point clouds, as the first three views are the most important ones. Table 5.5 shows the number of help- and main-point clouds and the total flight time for each object respectively.

Object # help-clouds # main-clouds Aprox. total flight time

Cereal 13 3 130sec

Fire extinguisher 5 3 155sec

Head statue 14 3 110sec

Dinos 4 3 50sec

Router 9 3 93 sec

Table 5.5: UAV flight results

5.3.2 Point Cloud Snapshots

Table 5.6 shows the resulting main point cloud snapshots for each object.

Cereal

(31)

Head statue

Dinos

Router

(32)

5.3.3 Resulting Objects

Table 5.7 shows the resulting objects using the object building method from [27]. The first column shows the object build using only the help-clouds, the second column shows the object build using only the main-clouds and the last column shows the object build using both. Note that the saved clouds where not pre-filtered as well as any other initial guessing incorporated into the modelling process.

Cereal

Fire extinguisher

(33)

Dinos

Router

(34)

Chapter 6

Result analysis

In this chapter the experimental results are analyzed. As described in the related work section of this report, previous work focuses on how to score and/or evaluate a given view. The presented method focuses on optimizing initial viewpoints, therefore we com-pare the method with respect to the generated validation set. The acquired validation set acts as a baseline for the iterative method to uncover the advantages and shortcomings of said method.

First the iterative approach is validated and the underlying assumptions verified, next the recorded performance on the real UAV is analyzed.

6.1 Iterative Approach Validation

After analyzing the results of the validation set with respect to the distribution of the information gain score, we can safely assume that the assumptions made to motivate the iterative optimization hold. We can see that there exist a continuous function encoding the information gain for each possible viewpoint. Also, the local maxima of this function correspond to good viewpoints.

When we compare the performance of the iterative optimization to the validation set we can observe that the iterative optimization matches the performance of the validation set for the first two viewpoints and only drops at the third view point (Figure 6.1a). We can also see that the iterative approach discovers a similar amount of new cells inside the target volume as the validation set (Figure 6.1b). The drop in information gain at the third view point can be linked to the fact that the first two viewpoints hold nearly all the information needed for the entire target volume. The iterative approach can then not find a continuous hill on the information gain function to climb, as the approach is limited by the initial circular viewpoints. We can also see, that while not getting the best possi-ble viewpoint of the entire set, the iterative optimization improves the information gain noticeably compared to the circular initialized viewpoints.

(35)

CHAPTER 6. RESULT ANALYSIS 29

(a) Comparison between the maximum

scores for the best view. (b) Comparison between the cells uncov-_{ered by the consecutive best views}

Figure 6.1: Comparison between iterative optimization and validation set

The number of initial circular viewpoints (n) has only a relevant effect when the num-ber reaches three or lower. Even with a smaller numnum-ber of initial viewpoints and there-fore a limitation towards the possible configuration of the UAV in yaw, the lower n has very little effect on the information gain (Figure 6.2a), or the discovery of new cells in-side the target volume (Figure 6.2b).

(a) Information gain for different n (b) # discovered cells with different n.

Figure 6.2: Comparison between iterative optimization with different n

(36)

30 CHAPTER 6. RESULT ANALYSIS

6.2 Object Modelling with a Real UAV

When employing the method on a real UAV we must deal with additional difficulties. The flight time of a real UAV is limited. Also, the computational resources are inferior to the simulation machine.

We can see that the three main-clouds hold nearly all necessary information in order to construct the desired object. That is, if a human was to align the clouds by hand. We can also see that the viewpoints chosen by the method correspond to sensible viewpoints that hold a lot of information. In the object "Head statue" we see that the second point cloud is taken from the opposite side of the first one. That is consistent with what one would assume intuitively. We can also see in the case of the object "router" that first the method takes a view frontal to the placed object and complemented this with the view from the left and right side in the second and third cloud respectively. This shows that the method can also deal with objects that are not standing freely in a room and are only accessible from certain directions. Another feature of the method compared to the pure circular initializations is that the method acquires more complete views of the object as can be seen when comparing the help-clouds to the main-clouds of the fire extinguisher. The help clouds failed to get the bottom part of the object while the main-clouds where taken further away which lead to the inclusion of the entire object.

When looking at the reconstructed objects we see that the quality of the clouds is suboptimal. This is mostly due to the unstable platform that is the real UAV. The unsta-ble movement of the real UAV results in a lot of motion blur for the captured clouds. When looking at the reconstructed objects we see that some of them, like the router or fire extinguisher, failed to correctly register. This can be lead back to the fact that the method used employs SIFT features in order to register the clouds. The main-clouds do not necessarily have a lot of overlapping features for the modelling to exploit. In fact, the presented method tries actively to achieve the opposite of overlapping features when picking view points as it optimizes for new information instead of overlapping features.

(37)

Chapter 7

Conclusion

In conclusion, this work has presented a new iterative optimized approach to find the Next Best View. It has been shown in simulation that the iterative approach leads to viewpoints that hold significant more information than randomly or circular initialized viewpoints. Also, the real-world applications have been demonstrated in a real-world experiment by deploying the method on board a UAV.

It was shown in experiments that the hill climbing method lead to more informa-tion in the resulting viewpoints. In simulainforma-tion and in a real-world scenario the method chooses viewpoints that encode much more information than a simple circular pattern, while being easily adaptive towards the size or shape of the target.

The execution of the path was handled by stitching together previously checked valid viewpoints in order to keep the object in view while traveling towards the Next Best View. This also offers the possibility to take additional snapshots of the object which improves the automated model building. While a human could use the resulting main point clouds collected at the Next Best Views in order to build a good model, the auto-mated SIFT based modelling approach from [27] needs more overlap from the clouds in order to correctly register the clouds and generate the resulting object.

This work has shown that an iterative optimization approach using a simple scoring function leads to viewpoints that hold more information about the desired object. Also, a method of reaching this NBV-points using previously checked circular view points as traverse points was demonstrated to be a safe and fast way to reach the target point, while still holding the object in the field of view the entire time. The unsatisfying re-sulting models can be attributed to the fact that not enough overlapping clouds could be collected to satisfy the chosen modelling approach.

7.1 Future Work

To extend on this project the next logical step would be to see if the modelling aspect of [27] could be modified to fit the sparser collected point clouds. One possibility is to incorporate the position recorded by the UAV as an initial guess for the position of the clouds in the final registration process.

Another improvement would be to make a modelling process that draws its needed cloud snapshots from a continuous recorded depth stream, instead of predefined snap-shots. That way the modelling approach would not depend so much on the sparsely taken clouds. One could imagine a system that oversees the relevant features on the

(38)

32 CHAPTER 7. CONCLUSION

tinuous stream and uses snapshots of the stream determined by an overlapping factor. As storage space is cheap the UAV could easily record the entire flight-time using a sim-ple compression method. Another method is to deploy a feature tracking system directly onto the UAV and use it to hold the object in view while traversing towards the NBV and taking intermediate snapshots at appropriate times.

(39)

Bibliography

[1] Parrot SA. Parrot drone bebop webpage. https://www.parrot.com/us/ drones/parrot-bebop-drone, 2017. [Online; accessed 23-February-2017].

[2] Rares Ambru¸s, Nils Bore, John Folkesson, and Patric Jensfelt. Meta-rooms: Building and maintaining long term spatial models in a dynamic world. IEEE International Conference on Intelligent Robots and Systems, pages 1854–1861, 2014. ISSN 21530866. doi: 10.1109/IROS.2014.6942806.

[3] Thomas Faulhammer, Rares Ambrus, Chris Burbridge, Michael Zillich, John Folkesson, Nick Hawes, Patric Jensfelt, and Markus Vincze. Autonomous Learn-ing of Object Models on a Mobile Robot. IEEE Robotics and Automation Letters, 2 (1):26–33, 2017. ISSN 2377-3766. doi: 10.1109/LRA.2016.2522086. URL http: //ieeexplore.ieee.org/document/7393491/.

[4] William R Scott and Gerhard Roth. View Planning for Automated Three-Dimensional Object Reconstruction and Inspection. 35(1):64–96, 2003.

[5] Y. F. Li and Z. G. Liu. Information entropy-based viewpoint planning for 3-D object reconstruction. IEEE Transactions on Robotics, 21(3):324–337, 2005. ISSN 15523098. doi: 10.1109/TRO.2004.837239.

[6] William R. Scott. Model-based view planning. Machine Vision and Applications, 20(1): 47–69, 2009. ISSN 09328092. doi: 10.1007/s00138-007-0110-2.

[7] Marshall L. Fisher and Laurence A. Wolsey. On the Greedy Heuristic for Continuous Covering and Packing Problems. SIAM Journal on Algebraic Discrete Methods, 3(4):584– 591, 1982. ISSN 0196-5212. doi: 10.1137/0603059.

[8] Paul S. Blaer and Peter K. Allen. Data acquisition and view planning for 3-D model-ing tasks. IEEE International Conference on Intelligent Robots and Systems, pages 417–422, 2007. doi: 10.1109/IROS.2007.4399581.

[9] Yu-Chi Lin, Shao-Ting Wei, Shih-An Yang, and Li-Chen Fu. Planning on search-ing occluded target object with a mobile robot manipulator. Robotics and Automation (ICRA), 2015 IEEE International Conference on, pages 3110–3115, 2015. ISSN 10504729. doi: 10.1109/ICRA.2015.7139626.

[10] Mustafa Devrim Kaba, Mustafa Gokhan Uzunbas, and Ser Nam Lim. A Rein-forcement Learning Approach to the View Planning Problem. 2016. URL http: //arxiv.org/abs/1610.06204.

(40)

34 BIBLIOGRAPHY

[11] G A Rummery and M Niranjan. On-Line Q-Learning using Connectionist Systems. (September), 1994.

[12] R.S. Sutton and A.G. Barto. Reinforcement Learning: An Introduction. IEEE Trans-actions on Neural Networks, 9(5):1054–1054, 1998. ISSN 1045-9227. doi: 10.1109/TNN. 1998.712192. URL http://ieeexplore.ieee.org/document/712192/.

[13] Richard S Sutton. Learning to Predict by the Method of Temporal Differences. Ma-chine Learning, 3(1):9–44, 1988. ISSN 08856125. doi: 10.1023/A:1018056104778. URL citeseer.ist.psu.edu/sutton88learning.html.

[14] Brendan Englot and Franz S Hover. Three-dimensional coverage planning for an underwater inspection robot. The International Journal of Robotics Research, 32(9-10): 1048–1073, 2013. ISSN 0278-3649. doi: 10.1177/0278364913490046. URL http: //ijr.sagepub.com/content/32/9-10/1048.abstract.

[15] Mark Sheinin and Yoav Y. Schechner. The Next Best Underwater View. page 32000, 2015. ISSN 10636919. doi: 10.1109/CVPR.2016.409. URL http://arxiv.org/abs/ 1512.01789.

[16] S Y Chen and Y F Li. Vision sensor planning for 3-D model acquisition. IEEE trans-actions on systems, man, and cybernetics. Part B, Cybernetics : a publication of the IEEE Systems, Man, and Cybernetics Society, 35(5):894–904, 2005. ISSN 1083-4419. doi: 10.1109/TSMCB.2005.846907.

[17] Richard Pito. A sensorbased solution to the "next best view" problem. Proceedings -International Conference on Pattern Recognition, 1(10):941–945, 1996. ISSN 10514651. doi: 10.1109/ICPR.1996.546162.

[18] Blake Fuller, Jonathan Kok, Neil Kelson, and Felipe Gonzalez. Hardware design and implementation of a MAVLink interface for an FPGA-based autonomous UAV flight control system. Australasian Conference on Robotics and Automation, ACRA, 02-04-Dece, 2014. ISSN 14482053.

[19] Korbinian Schmid, Heiko Hirschmüller, Andreas Dömel, Iris Grixa, Michael Suppa, and Gerd Hirzinger. View planning for multi-view stereo 3D Reconstruction using an autonomous multicopter. Journal of Intelligent and Robotic Systems: Theory and Appli-cations, 65(1-4):309–323, 2012. ISSN 09210296. doi: 10.1007/s10846-011-9576-2.

[20] Xin Zhong Peng, Huei Yung Lin, and Jyun Min Dai. Path planning and obstacle avoidance for vision guided quadrotor UAV navigation. IEEE International Conference on Control and Automation, ICCA, 2016-July:984–989, 2016. ISSN 19483457. doi: 10. 1109/ICCA.2016.7505408.

[21] Wei Jing, Joseph Polden, Pey Yuen Tao, Wei Lin, and Kenji Shimada. View Planning for 3D Shape Reconstruction of Buildings with Unmanned Aerial Vehicles. 2016 (November):13–15, 2016.

(41)

BIBLIOGRAPHY 35

[23] Shupeng Lai, Kangli Wang, Hailong Qin, Jin Q. Cui, and Ben M. Chen. A ro-bust online path planning approach in cluttered environments for micro rotorcraft drones. Control Theory and Technology, 14(1):83–96, 2016. ISSN 21980942. doi: 10.1007/s11768-016-6007-8.

[24] Stefan Wenhardt, Benjamin Deutsch, Joachim Hornegger, Heinrich Niemann, and Joachim Denzler. An information theoretic approach for next best view planning in 3-D reconstruction. Proceedings - International Conference on Pattern Recognition, 1: 103–106, 2006. ISSN 10514651. doi: 10.1109/ICPR.2006.253.

[25] Stefan Isler, Reza Sabzevari, Jeffrey Delmerico, and Davide Scaramuzza. An infor-mation gain formulation for active volumetric 3D reconstruction. Proceedings - IEEE International Conference on Robotics and Automation, 2016-June:3477–3484, 2016. ISSN 10504729. doi: 10.1109/ICRA.2016.7487527.

[26] Jonathan Daudelin and Mark Campbell. An Adaptable, Probabilistic, Next-Best View Algorithm for Reconstruction of Unknown 3-D Objects. 2(3):1540–1547, 2017.

[27] Michael C Welle, Ludvig Ericson, Rares Ambrus, and Patric Jensfelt. On the use of Unmanned Aerial Vehicles for Autonomous Object Modeling.

[28] Christian Potthast and Gaurav S. Sukhatme. A probabilistic framework for next best view estimation in a cluttered environment. Journal of Visual Communication and Image Representation, 25(1):148–164, 2014. ISSN 10473203. doi: 10.1016/j.jvcir.2013.07.006. [29] E. Angelopoulou, T.H. Hong, and a.Y. Wu. World model representations for mobile

robots. Proceedings of the Intelligent Vehicles ‘92 Symposium, pages 293–297, 1992. doi: 10.1109/IVS.1992.252274.

[30] Armin Hornung, Kai M. Wurm, Maren Bennewitz, Cyrill Stachniss, and Wolfram Burgard. OctoMap: An efficient probabilistic 3D mapping framework based on octrees. Autonomous Robots, 34(3):189–206, 2013. ISSN 09295593. doi: 10.1007/ s10514-012-9321-0.

[31] John Amanatides and A Woo. A Fast Voxel Traversal Algorithm for Ray Tracing. Eurographics, 87(3):3–10, 1987. ISSN 09603107. doi: 10.1.1.42.3443. URL http:// www.cse.yorku.ca/~amana/research/grid.pdf.

[32] View Frustum. What Is a View Frustum? webpage. https://msdn.microsoft. com/en-us/library/ff634570.aspx, 2017. [Online; accessed 30-October-2017]. [33] Ray caster. Raycasting? webpage, lecture. https://www.cs.princeton.edu/

courses/archive/fall00/cs426/lectures/raycast/raycast.pdf, 2017. [Online; accessed 07-November-2017].

[34] Jacobo Jimenez Lugo and Andreas Zeil. Framework for autonomous onboard nav-igation with the AR.Drone. 2013 International Conference on Unmanned Aircraft Sys-tems, ICUAS 2013 - Conference Proceedings, pages 575–583, 2013. ISSN 09210296. doi: 10.1109/ICUAS.2013.6564735.

(42)

36 BIBLIOGRAPHY

[36] Steven M.LaValle. Rapidly-ExploringRandomTrees:ANewToolforPathPlanning ,. 1998.

[37] Paul Besl and Neil McKay. A Method for Registration of 3-D Shapes, 1992. ISSN 0162-8828.

[38] Martin a Fischler and Robert C Bolles. Random Sample Consensus: A Paradigm for Model Fitting with. Communications of the ACM, 24:381–395, 1981. ISSN 00010782. doi: 10.1145/358669.358692.

[39] David G. Lowe. Method and apparatus for identifying scale invariant features in an image and use of same for locating an object in an image. 1(12):1–20, 2004.

[40] Autocad. Recap360 3D modeling online tool. https://www.autodesk.com/ products/recap/overview, 2017. [Online; accessed 07-November-2017].

[41] Skatchfab. Skatchfab 3D modeling online tool. https://sketchfab.com, 2017. [Online; accessed 07-November-2017].

(43)

Appendix A

Sustainability, Ethical and Social

con-sideration

A.1 Sustainability reflection

The presented method does not have a direct implication regarding sustainability. One need however consider indirect implications as well. An automated way to find and per-form view planning in the framework of UAVs could certainly rise the demand for those UAVs that are capable to deploy it. One of the most important measurements of sustain-ability is the ecological footprint over the lifetime of a product. Having a higher demand of UAVs would naturally impose a greater strain on the established production chains which could lead to companies cutting corners in regards of sustainability. As an increase in demand would also spark additional suppliers of UAV coming to marked the sustain-ability itself could become a marketing and selling strategy. As the modern consumer is very vigilant to those kind of things, the impact could easily be negated by the new arise in competition. Therefore the effects regarding sustainability in case of this thesis are minimal.

A.2 Ethical consideration

Autonomous UAVs rise a considerable amount of ethical considerations as those can be used to spy an perform surveillance of innocent people. The work presented in this re-port however focuses on finding a Next Best View in regards to confined spaces. While it is thinkable that the work could be misused and employed for sinister processes it is highly unlikely that it is an inspiration for said processes. The ability to find suitable views does not pose obvious ethical consideration.

A.3 Social impact

Equipping an army of drones with the presented view planning method could help them replace human workers at construction sides. The algorithm deployed on a robotic arm that generates models of objects for a library, could as well endanger the job security. The rise of automation in general has big social impact overall. Old jobs and skills are no longer needed, people need to adapt and find other fields to earn a living. What will

(44)

38 APPENDIX A. SUSTAINABILITY, ETHICAL AND SOCIAL CONSIDERATION

(45)

Appendix B

UAV platform

The following parts contains more detailed information about the UAV used in the ex-periments and are provided for the interested reader.

B.1 Hardware

The Real world experiments where carried out on the UAV shown in figure B.1.

Figure B.1: Used UAV, codename "Flyguy"

Airframe

The skeleton of the drone is made out of carbon fiber and configured for a quad-rotor configuration.

The basic frame is augmented to mount the RGBD-camera. The controller as well as the power-board and on-board computer (NUC) are mounted in the center of the UAV.

(46)

40 APPENDIX B. UAV PLATFORM

The battery is positioned over the NUC and the landing gear consist of four spacers. The four rotors are enclosed with 3D printed rotor guards that protect the UAV if it collides with obstacles.

Battery

The used battery is a Xtron 3600 mAh, 4 cell battery with C-rate of 30.

Flight controller

The flight controller used is a PX4 flight controller that works with MAVlink and Mavros. The PX4 allows frame selection as well as a good graphical interface to adjust and tune parameter. It comes with a ground control software called "QGroundControl" and offers a wide variety of configuration options like the integration of a Motion-capture system as position estimate. A guide to tune the IMU-sensor as well as instruction on how to perform the PID-controller setup and tuning are available at the PX4 Website.

Motion capture system - Motive/OptiTrack

To get an accurate position a Motion-capture system using a software called "Motive". Motive tracks small infrared reflecting balls positioned on the UAV and returns a pose estimate with an accuracy up to 1mm. This makes the pose estimation of the UAV highly accurate but confines the system to an area where the motion-capture system is installed.

Motors + Rotors

The four motors are "dys Race Edition SE2205-2550KV". Two having clockwise and two counter-clockwise threading to mount the Dalprop T5045 and T5045R ABS plastic pro-pellers respectively.

RGBD-camera Xtion Primesense

The RGB-D camera used is a Primesense RGB-D camera with the following specifica-tions:

• Field of View: 54◦ _{horizontal, 45}◦ _vertical

• Depth Image Size: VGA(640x480) • Spatial X/Y Resolution @0.5m: 1mm • Depth Z Resolution @0.5m: <1mm

• Maximum Image Throughput: (QVGA) 60fps, (VGA) 30fps • Operation Range: 0.35-3m

(47)

APPENDIX B. UAV PLATFORM 41

B.2 Software

Robotic Operation System - ROS

The Robotic Operation System - ROS is a framework for writing software that is de-ployed on robots. The system is heavily modular and therefore many contributions from one robot can be easily transfered to another robot. The UAV used had ROS Indigo in-stalled and made use of pre-existing methods, like the PCL-ROS package or the ROS im-plementation of Octomap.

MAVlink + Mavros

MAVlink stands for Micro Air Vehicle Communication Protocol and is a lightweight li-brary that is used in order to communicate with UAVs. It is released under the LGPL license and is designed as a header-only message system.

(48)

View Planning for Objects Modeling using UAVs

View Planning for Objects

Modeling using UAVs

MICHAEL C. WELLE

View Planning for Objects Modeling

using UAVs

Abstract

Sammanfattning

Contents

Chapter 1

Introduction

1.1

Research Question

1.2

Report Outline

Chapter 2

Related Work

2.1

View Planning at large

2.2

View Planning with UAVs

Chapter 3

Background

3.1

Probabilistic Representation of an Environment

3.2

3D Sensor Model

3.3

Ray Casting

3.4

Path Planning and Execution with a UAV

3.5

Object Modelling

Chapter 4

Method

4.1

Method overview

4.2

View point initialization

4.3

Evaluation of views

4.4

Optimization of best view

4.5

Execution of the planned path

Chapter 5

Experiments

5.1

Experimental setup

5.2

Evaluation of iterative optimization

5.3

View Planning on a Real UAV

Chapter 6

Result analysis

6.1

Iterative Approach Validation

6.2

Object Modelling with a Real UAV

Chapter 7

Conclusion

7.1

Future Work

Bibliography

Appendix A

Sustainability, Ethical and Social

con-sideration

A.1

Sustainability reflection

A.2

Ethical consideration

A.3

Social impact

Appendix B

UAV platform

B.1

Hardware

B.2

Software