Polygonal models from range scanned trees

Full text

(1)LiU-ITN-TEK-A--09/027--SE. Polygonal models from range scanned trees Li Qiu 2009-04-24. Department of Science and Technology Linköping University SE-601 74 Norrköping, Sweden. Institutionen för teknik och naturvetenskap Linköpings Universitet 601 74 Norrköping.

(2) LiU-ITN-TEK-A--09/027--SE. Polygonal models from range scanned trees Examensarbete utfört i medieteknik vid Tekniska Högskolan vid Linköpings universitet. Li Qiu Handledare Mark Dieckmann Examinator Mark Eric Dieckmann Norrköping 2009-04-24.

(3) Upphovsrätt Detta dokument hålls tillgängligt på Internet – eller dess framtida ersättare – under en längre tid från publiceringsdatum under förutsättning att inga extraordinära omständigheter uppstår. Tillgång till dokumentet innebär tillstånd för var och en att läsa, ladda ner, skriva ut enstaka kopior för enskilt bruk och att använda det oförändrat för ickekommersiell forskning och för undervisning. Överföring av upphovsrätten vid en senare tidpunkt kan inte upphäva detta tillstånd. All annan användning av dokumentet kräver upphovsmannens medgivande. För att garantera äktheten, säkerheten och tillgängligheten finns det lösningar av teknisk och administrativ art. Upphovsmannens ideella rätt innefattar rätt att bli nämnd som upphovsman i den omfattning som god sed kräver vid användning av dokumentet på ovan beskrivna sätt samt skydd mot att dokumentet ändras eller presenteras i sådan form eller i sådant sammanhang som är kränkande för upphovsmannens litterära eller konstnärliga anseende eller egenart. För ytterligare information om Linköping University Electronic Press se förlagets hemsida http://www.ep.liu.se/ Copyright The publishers will keep this document online on the Internet - or its possible replacement - for a considerable time from the date of publication barring exceptional circumstances. The online availability of the document implies a permanent permission for anyone to read, to download, to print out single copies for your own use and to use it unchanged for any non-commercial research and educational purpose. Subsequent transfers of copyright cannot revoke this permission. All other uses of the document are conditional on the consent of the copyright owner. The publisher has taken technical and administrative measures to assure authenticity, security and accessibility. According to intellectual property law the author has the right to be mentioned when his/her work is accessed as described above and to be protected against infringement. For additional information about the Linköping University Electronic Press and its procedures for publication and for assurance of document integrity, please refer to its WWW home page: http://www.ep.liu.se/. © Li Qiu.

(4) Abstract 3D Models of botanical trees are very important in video games, simulation, virtual reality, digital city modeling and other fields of Computer Graphics. However, since the early days of computer graphics, the modeling of trees has been challenging, because of the huge dynamical range between its smallest and largest structures and their geometrical complexity. Trees are also ubiquitous which makes it even hard to model them in a realistic way. Current techniques are limited in that they model a tree either in a rule-based way or in an approximated way. These methods emphasize appearance while sacrificing its real structure. Recent development in range scanners are making 3D acquisition feasible for large and complex objects. This report presents the semi-automatic technique developed for modeling laser-scanned trees. First, the user draws a few strokes on the depth image plane generated from the dataset. Branches are then extracted through the 2D Curve detection algorithm originally developed. Afterwards, those short branches are connected together to generate the skeleton of the tree by forming a Minimum Spanning Tree(MST). Finally, the geometry of the tree skeleton is produced using allometric rules for branch thickness and branching angles. v.

(5)

(6) Acknowledgments I wish to express sincere appreciation to Prof. Baoquan Chen, from Shenzhen Institute of Advanced Technology(SIAT), Chinese Academy of Sciences(CAS), for providing me with this thesis topic and his guidance throughout the project. I would also like to acknowledge Doctor Zhanglin Cheng for giving me the background information on Branches Reconstruction from Range Image and supplying me dataset. Furthermore, I would like to thank all the staff in Visual Computing Center(VCC) of SIAT for many exciting discussions on Point-Based Graphics. In particular, I want to thank Doctor Jun Luo for the discussions on Computational Geometry and Yotam Livny, from Ben-Gurion University of Israel, for the discussions on 2D curve detection. I would like to thank all those who gave me advice, supported through the study of the programme of Advanced Computer Graphics. They are(in no particular order): Mark E Dieckmann, Aida Vitoria, Tomas Otby, Anders Ynnerman, Haraldur Haralds, Richard Khoury, Willem Frishert, Chiyu Wang.. vii.

(7)

(8) Contents 1 Introduction 1.1 Problem Description . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 Objective . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.3 Structure of the Thesis . . . . . . . . . . . . . . . . . . . . . . . . .. 1 1 1 2. 2 Background 2.1 Rule-based Plant Modeling . . . . . 2.1.1 L-system . . . . . . . . . . . 2.1.2 Tree modeling using L-system 2.2 Sketch-based Tree Modeling . . . . . 2.3 Image-based Plant Modeling . . . . . 2.4 Conclusion . . . . . . . . . . . . . . 3. RANSAC 3.1 RANSAC . . . . . . . . . . . 3.2 Efficient RANSAC . . . . . . 3.3 2D Curve Detection . . . . . 3.3.1 Lagrange Interpolating 3.3.2 Scoring . . . . . . . .. . . . . . . . . . . . . Curve . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. 3 3 3 4 5 6 8. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. 9 9 10 11 11 12. . . . . . . . . . .. 13 13 14 14 16 16 16 17 17 17 18. 4 Modeling of Laser-Scanned Trees 4.1 System Overview . . . . . . . . . . . . . 4.2 Sketching . . . . . . . . . . . . . . . . . 4.2.1 Rendering Issues . . . . . . . . . 4.2.2 Sketching . . . . . . . . . . . . . 4.3 Connecting Branches . . . . . . . . . . . 4.3.1 Minimum Spanning Tree(MST) . 4.3.2 Connecting Branches using MST 4.4 Mesh Production . . . . . . . . . . . . . 4.4.1 Tree Allometry . . . . . . . . . . 4.4.2 Skeleton Mesh . . . . . . . . . . 5 Results. . . . . . . . . . .. . . . . . . . . . .. . . . . . . . . . .. . . . . . . . . . .. . . . . . . . . . .. . . . . . . . . . .. . . . . . . . . . .. . . . . . . . . . .. . . . . . . . . . .. . . . . . . . . . .. . . . . . . . . . .. . . . . . . . . . .. . . . . . . . . . .. . . . . . . . . . .. 19 ix.

(9) x 6. Contents Conclusions and Future Work 6.1 Conclusion . . . . . . . . . . . . . . . . 6.2 Future Work . . . . . . . . . . . . . . . 6.2.1 Image Segmentation Techniques 6.2.2 3D Ransac Branch Detection . .. Bibliography. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. 23 23 23 23 24 25.

(10) Chapter 1. Introduction This chapter gives an overview of the subject and the objective to achieve for this diploma work. It also outlines the layout of this thesis.. 1.1. Problem Description. 3D Models of botanical trees are very important in video games, simulation, virtual reality, digital city modeling and other fields of Computer Graphics. However, since the early days of computer graphics, modeling trees is challenging because of its large scale of structure and geometrical complexity. Trees are also ubiquitous which makes it even hard to model in a realistic way. Current techniques limit the modeling of tree either in a rule-based way or in an approximated way. These methods emphasize trees’ appearance while sacrificing its real structure. Recent developments in range scanners are making 3D acquisition feasible for large and complex objects like trees. This report presents the semi-automatic technique of modeling laser-scanned trees.. 1.2. Objective. The objective of the project presented in this thesis is to produce polygonal models with fine details from laser-scanned trees. The tree data acquired by 3D scanning is usually in the form of point cloud, which is a set of vertices in a three-dimensional coordinate system, measured from a large number of points on the surface of objects. Due to the lack of topology information, point clouds are generally not directly rendered in most 3D applications before converted to polygon models. Traditional techniques for converting a point cloud to a polygon mesh include Delaunay triangulation, Marching triangles [1] and the Ball-Pivoting algorithm [2]. Unlike buildings and manufactured part which have relatively simple geometry, trees are more complex and micro-structured. Moreover, the small branches at foliage region are often missing because of limited scanning resolution and occlusion from leaves and other branches. The challenge is to reproduce the models as 1.

(11) 2. Introduction. accurate as possible as real trees.. 1.3. Structure of the Thesis. The report is organized as follows: Chapter 2 provides more details on the background and describes most recent developments of tree modeling in the field of computer graphics. Chapter 3 gives a brief introduction of RANSAC and our extension with it. Chapter 4 presents our approach of building polygon model from laser-scanned trees. The results are shown and discussed in chapter 5, followed by the conclusions and future work presented in chapter 6..

(12) Chapter 2. Background This chapter gives a brief discussion of previous work and the recent advances in tree modeling. The techniques for modeling bushes and trees can be roughly classified into rule-based, sketch-based and image-based plant modeling. Rulebased system defines a set of rules and symbols to iteratively generate a string which will be interpreted as self-similar fractals at the final stage. Sketch-based tree modeling usually requires user input e.g. a few strokes to reconstruct the 3D branches. Sketch-based systems provide a more intuitive way of generating plant models. Image-based tree modeling uses images to help generate 3D models.. 2.1 2.1.1. Rule-based Plant Modeling L-system. Astrid Lindenmeyer [3] first introduced L-system as a formalism for simulating the development of multicellular organs in terms of division, growth, and individual cells. An L-system or Lindenmayer system is a formal grammar(a set of rules and symbols). L-systems can also be used to generate self-similar fractals such as iterated function systems [4]. L-system is competent for describing self-similarity and fractal-like forms due to its recursive nature [4]. By increasing the recursion level, the form slowly grows and becomes more complex. L-system is also popular in generating artificial life. L-system can be defined as a tuple G = {V, S, ω, P }. (2.1). where: • V(the alphabet): is a set of symbols containing elements that can be replaced (variables). • S: is a set of symbols containing elements that remain fixed (constants). 3.

(13) 4. Background. Figure 2.1. ’Weeds’, generated using an L-system in 3D. • ω: (start, axiom or initiator): is a string of symbols from V defining the initial state of the system. • P: is a set of production rules or productions defining the way variables can be replaced with combinations of constants and other variables. A production consists of two strings - the predecessor and the successor. There are several extensions to the basic L-systems. Parametric L-system allows the passing of arbitrary numerical information from parents to children. If a rule depends on its neighbors, it is called context-sensitive L-system; otherwise it is called context-free L-system. If there is exactly one production for each symbol, it is termed a deterministic L-system. If there are several, and each is chosen with a certain probability, then it is said to be a stochastic L-system. L-system does not actually produce any geometry but a string contains symbols. Visual output comes from the mapping of the string into visual domain. One such mapping is to interpret the strings as instructions to a turtle, for example, the symbol F moves the turtle forward and + rotates the turtle in certain degrees. With this mapping, the L-system can produce a large variety of squiggly lines. Since we want to use the L-system to model plants, we may map those symbols into cylinders and polygons to represent branch and leaves respectively. Figure 2.1 shows a few tree models generated by L-system with different levels of recursion.. 2.1.2. Tree modeling using L-system. Prusinkiewiczy et al. [5] gave a systematic description for using L-system to model plants. The paper revisited basic mechanisms that control plant development: cellular descent, endogenous interaction represented by context-sensitive L-systems. Several most recent models developed at that time were also surveyed. Parameterized L-systems were proposed in [6] [7] [8]. Xfog [9] is a combination of a rule-based approach and the graph-based object instancing paradigm. There are generally three categories of components which.

(14) 2.2 Sketch-based Tree Modeling. Figure 2.2. field [10]. 5. Computing the depth information for branches with the 2D distance. encapsulate data and algorithm for generating plant elements. One group of components creates graphical primitives like stems, twigs and leaves. The multiplying component connects other components by linking the component prototypes in a directed graph called the prototype graph, or p-graph. And the third applies parameters to the global modeling. A component instance called i-tree is built when the system traverses the p-graph.. 2.2. Sketch-based Tree Modeling. Sketch-based systems provide a intuitive way of developing plant models. Okabe et al.’s system [10] reconstruct tree branches from 2D drawn sketches in different views and using the assumption that trees spread their branches so that the distances between branches are as large as possible. The user begins to model a 3D tree by sketching simple 2D branches using a mouse or pen tablet. The system then generates 3D tree geometry after the user clicks “3D” button. The user can add or remove branches manually by using the editing interface. The user can also add more child branches to a parent branch using existing branches as example. Leaves are arranged around branches by following typical leaf arrangement patterns. Finally, branches are propagated. The task to create a 3D tree from 2D sketch is fulfilled by projecting all branches to the ground and constructing a 2D distance field. Distances are then computed from the projected branches to each pixel(see Figure 2.2). However, the projection of 3D branches onto the screen should fit the 2D sketch. This is done by restricting the search within a 3D hull, which is constructed by simply sweeping a circle along the 2D convex hull and enlarging it by a constant value. Ijiri et al.’s system [11] is an L-system-based sketch system. The system consists of two panels: a control pane and a creation pane. In the control pane, the user manipulates the control points to specify the rules and parameters. Predefined generating rules are also available in the system. After generating the rules.

(15) 6. Background. Figure 2.3. Sketch-Based Tree Modeling Using Markov Random Field [12]. and parameters, the user draw another stroke representing the central axis in the creation pane. The system then interprets the drawn stroke and applies the specified rules at the same time the recursion process occur until it reaches the depth of recursion specified by the user. Chen et al. [12] proposed a method to formulate the tree reconstruction problem as Markov random field. In their system, the user begin to model the tree with a few strokes of branches. The closest matched specimen is then selected from a predefined database which contains typical tree exemplars and their associated global parameters. Assuming that the sketch is drawn under orthographic projection, the problem of reconstructing the tree model is reduced as evaluating the depth of each drawn branch. Considering each branch segment as a node and its depth as a variable, it becomes dealing with Markov tree. Additionally, explicitly introducing additional nodes representing the relationships between neighbor nodes producing a factor graph. Solving the factor graph reconstructs the 3D shape of the drawn branches. Figure 2.3 shows the result of their system.. 2.3. Image-based Plant Modeling. Rather than requiring the user to specify the model manually, Image-based approaches use a few images as input to generate 3D models. A purely image-based modeling approach is described by Reche et al. [13]. Their system adopt a direct volumetric rendering model based on the observation that at a certain distance, each pixel in a photograph of a tree is blended projection of leaves, branches and background at that direction. First, opacity values of the tree volume are estimated from calibrated digital photographs by using alpha-mattes extracted from the photographs. Then, a billboard with view-dependent texture is assigned to each grid cell. Finally, volumetric rendering approach is deployed to represent and render the tree. Tan et al. [14] proposed another method to reconstruct the 3D model from.

(16) 2.3 Image-based Plant Modeling. 7. Figure 2.4. Tan [14]’s image based tree modeling system. multiple images which captured by a hand-held camera. The system extract point correspondences and ran structure from motion on them to recover camera parameters and 3D point cloud by using standard computer vision techniques. Once the camera poses and 3D point cloud have been obtained, the system partition the points into branches and leaves according to their color and position. The reconstruction of the visible branches starts from constructing a graph to each point cluster. The user initiate a 3D point of the primary branch and add another branch cluster to it iteratively. After recovering the visible branches, the system recover the occluded branches by replicating the reconstructed parts. Finally, leaves are added and populated. Figure 2.4 shows an overview of the system. Neubert el al. [14] presented a new image-based modeling method for 3D tree geometry from several photographs with limited user intervention. For each given input image, their system first separates the tree from background and then 2D attractor graph is computed either automatically by using Livewire approach [15] or from user drawn strokes. Secondly, back-projection is applied to the input images to compute the density volume(see Figure 2.5), a voxel-model of the tree volume with each voxel containing a density estimate of the tree’s biomass. Particles are then positioned in proportion to the density values. 2D attractor graphs are used to create direction fields which provide direction vectors for each image plane. Combining these vectors for all image planes yields vectors which direct the flow of particles. Tracing the flow of the particles produces the main skeleton of the tree. Branch thickness and branching angles are estimated using allometric rules. Multiple images are used to generate the tree models in methods mentioned above while single image is used in Tan. et al. [16]’s system. Given a single image of a tree, the user draws strokes on the image to create a tree model. At least two strokes are needed to complete the tree model: the first crown stroke around the tree crown to mark up the leaf region, the second starting at the root to mark up the main trunk. Additional strokes are optional to refine the tree model. After user draws strokes, foliage is extracted from the closed region by the first crown stroke. A Gaussian mixture model(GMM) [17] is computed for the pixel RGB values and graph-cut based extraction [18] is deployed to detect leaf pixels. To trace the visible branches, branch appearance model and non-branch appearance model are computed. With these two models, a circle along the branch stroke.

(17) 8. Background. (a). (b). Figure 2.5. Estimating the tree density: (a)voxel grid; (b)density values for one image plane [14]. is moved from bottom to up to recover the visible branches. Finally, the system converts the 2D branches and grows the tree in a similar way as that in [11].. 2.4. Conclusion. Parametric L-system are able to provide the user with a few parameters. However, they are usually restricted to produce a limited number of forms. To achieve a specific species or even a particular tree, many parameters have to be adjusted and kept which is impractical to the users. Image-based methods have the best potential for producing realistic-looking plants [12]. While they emphasis on the appearance of the tree, their approach does not recover explicit geometries of the branches and leaves. Exceptions are approaches that apply Structure From Motion(SFM), a standard method finding the three-dimensional structure by analyzing the motion of an object over time, to recover 3D point cloud from image sequences..

(18) Chapter 3. RANSAC This chapter introduces RANSAC, a shape detection method that extracts shapes by randomly drawing minimal sets from the data with noise and outlier, and its variant, Efficient RANSAC which is more useful in an interactive setting. We extended Efficient RANSAC to support 2D curve detection for our application.. 3.1. RANSAC. RANSAC, short for RANdom SAmple Consensus is an iterative method to estimate parameters of a mathematical model from observed data which contains noise and outliers. In this procedure, a basic assumption is that the data consists of inliers, i.e., data whose distribution fits some set of model parameters, and outliers, data that can not be explained by the model(see Figure 3.1): Given a set of observed data values, a parametrized model which can explain the observations and some confidence parameters, RANSAC finds the optimal model(s) by iteratively selecting a random subset of the original data. These data are hypothetical inliers and this hypothesis is tested as follows: 1. A model is fitted to the hypothetical inliers, i.e. all free parameters of the model are reconstructed from the data set. 2. All other data are then tested against the fitted model and, if a point fits well to the estimated model, also considered as a hypothetical inlier. 3. The estimated model is reasonably good if sufficiently many points have been classified as hypothetical inliers. 4. The model is reestimated from all hypothetical inliers, because it has only been estimated from the initial set of hypothetical inliers. 5. Finally, the model is evaluated by estimating the error of the inliers relative to the model. This procedure is repeated a fixed number of times T until we find the the model with lowest error. 9.

(19) 10. RANSAC. (a). (b). Figure 3.1. (a):A data set with many outliers for which a line has to be fitted; (b): Fitted line with RANSAC, outliers have no influence on the result. 3.2. Efficient RANSAC. Ransac can estimate the parameters with a high degree of accuracy even when significant amount of outliers appears in the data set. However, there is no upper bound number of iterations it takes to compute these parameters. Moreover, the datasets are often huge, which implies that the efficiency of the algorithm is of utmost importance to enable interactivity. In our implementation, we extended R. Schnabel et. al [19]’s Efficient RANSAC algorithm to support 2D Curve detection. Algorithm 1 shows an overview of Efficient RANSAC algorithm. The complexity of RANSAC is dominated by two major factors: The number of minimal sets that are drawn and the cost of evaluating the score for every candidate shape. The number of iterations can be determined by the probability that the best possible shape is indeed detected. Consider a point cloud P of size N and a shape Ψ therein consisting of n points, k the size of a minimal set required to define a shape candidate, the probability of any k points leading to an appropriate.

(20) 3.3 2D Curve Detection. 11. candidate is: n N n k P (n) = k / k ≈ N. (3.1). The probability of detecting an appropriate shape after s candidates have been drawn equals the complementary of s consecutive failures: s. P (n, s) = 1 − (1 − P (n)). (3.2). Thus, the number of candidates T required to detect shape Ψ with a probability P (n, T ) ≥ Pt : T ≥. ln (1 − pt ) ln (1 − P (n)). (3.3). From formula 3.3, we can see that the runtime complexity is directly related to the hit rate of finding good sample sets. Shapes are usually local phenomena, which means that the probability of two points close to each other belong to the same shape is higher. Exploiting this fact increases the probability of drawing minimal sets belonging to the same shape. The Efficient RANSAC algorithm uses an octree to establish spatial proximity between samples(see R. Schnabel et. al [19]). Initially, choose points for the first sample p1 among all points. Then, a cell C is randomly chosen from any level of the octree such that p1 ∈ C, and the other k − 1 samples are only chosen from C. The procedure can be expressed by a new probability Plocal (n): Plocal (n) = P (p1 ∈ Ψ) P (p2 · · · pk ∈ Ψ|p2 · · · pk ∈ C). (3.4). Assuming that there exists a C which contains half of its points supporting shape pi , the probability of finding a good C in depth d is: Plocal (n) =. 3.3. n N d2k−1. (3.5). 2D Curve Detection. Our application of RANSAC to tree modeling is to automatically extract a curve(branch) from user-sketched data.. 3.3.1. Lagrange Interpolating Curve. Since our problem is to find the curve passes through most of the discrete points in 2D space, we choose to use Lagrange interpolating curve [20] justified by several reasons. First, Lagrange curve only requires a few points as input, which can be obtained by simply sampling from given data source. Secondly, the degree of the polynomial is usually low. Given a set of k + 1 data points.

(21) 12. RANSAC. (x0 , y0 ) , · · · , (xk , yk ). (3.6). where none of them are identical, the interpolation polynomial in the Lagrange form is a linear combination L(x) :=. k X. yj lj (x). (3.7). j=0. of Lagrange basis polynomials. lj (x) :=. k Y. i=0,i6=j. x − xi (x − x0 ) (x − xj−1 ) (x − xj+1 ) (x − xk ) = ··· ··· . xj − xi (xj − x0 ) (xj − xj−1 ) (xj − xj+1 ) (xj − xk ) (3.8). 3.3.2. Scoring. The scoring function σp measures the quality of a given curve. In our case, the points on a curve may not be successive while disconnected with several parts because a branch may be occluded by other branch or leaves(see Figure 4.3). To measure the score of a candidate curve, we use only the number of points that fall within an ǫ-band around the curve, that is, ǫ close to the curve. More formally, given a curve Ψ whose fidelity is to be evaluated, σp is defined as follows: σp (Ψ) = |PΨ | (3.9) The number of points in PΨ is defined in the following step: PΨ = {p|p ∈ P ∧ d(Ψ, p) < ǫ}. (3.10). where d(ψ, p) is the euclidean distance between the point p and the curve Ψ..

(22) Chapter 4. Modeling of Laser-Scanned Trees This chapter first gives an overview of our approach of modeling tree from point clouds. Each step is then detailed in the following subsections. Our approach first require the user to draw a few strokes on the depth image plane generated from the dataset. Branches are then extracted through the 2D Curve detection algorithm. Afterwards, those short branches are connected together to generate the skeleton of the tree by forming a Minimum Spanning Tree(MST). The geometry of the tree skeleton is produced using allometric rules for branch thickness and branching angles. 4.1. System Overview. Given the point cloud of a laser-scanned tree, we start the polygon modeling process by first constructing the skeleton, which is a good representation of tree structure. The geometry of the skeleton is produced by using allometric theory. Finally, leaves are added to nearby branches. The outline below also reflects the further structure of this chapter: 1. Sketching: The user draws a few strokes to represent the branches. 2. Skeleton Generation: Based on the strokes drawn by user, the system automatically search the branch from the 3D point cloud. Those branches are resampled and connected together by forming a Minimum Spanning Tree(MST). 3. Mesh Generation: After the full skeleton is generated, the thickness of each branch is determined by using botanical rules. Once the radius has been estimated for each node, mesh is generated by connecting rings paced at each node. 13.

(23) 14. Modeling of Laser-Scanned Trees. Figure 4.1. Points scanned from a oblique branch. 4.2. Sketching. To reconstructed the skeleton of the tree, a variety of methods are surveyed. Curve extraction [21] [22] [23] [24] and curve tracing [25] [26] both involves classifying pixels as being curve points or not depending whether they lie on some curve in the image and linking curve points to form curves. The difference between curve extraction and curve tracing is that curve extraction emphasizes on the extraction while curve tracing on the identification or labeling a curve. Pattern recognition techniques such as matching-filters [27], ridge-based approaches [27], region growing [27], multi-scale approaches [27], skeleton-based approaches [27], level set methods [27], tracking-based approaches [27] et al., are popular approaches in medical segmentation. Due to limited scanning resolution and occlusion from leaves and other branches, improper classifying and linking of curve(branch) are prone to happen. Moreover, small branches and leaves in the crown are often irregular and chaotic, and when the branch is scanned from a glancing angle, very sparse points are sampled from it(see Figure 4.1). To alleviate the influence of these problems and produce a good main skeleton, the system requires user to extract the skeleton.. 4.2.1. Rendering Issues. We prefer rendering the point cloud with scanning image plane rather than doing it with standard 3D rendering pipeline since there are big holes as zooming in to look into the details of the structure of the tree for user interaction. We also color the points with its depth such that it is easy to see the occlusion of branches from each other(see Figure 4.2 and Figure 4.3)..

(24) 4.2 Sketching. 15. Figure 4.2. Rendering point cloud for user interaction. Top right: render the points as point sprites. Bottom right: render the points in scanning image plane.. Figure 4.3. Depth view of the tree.

(25) 16. Modeling of Laser-Scanned Trees. Figure 4.4. Curve detection using RANSAC. 4.2.2. Sketching. The user draws a few strokes on the image by moving the mouse cursor and holding a button to mark out branches and those occluded by other branches that are in front of them. During this process, a band of pixels are selected around the current stroke and projected to the curve drawn. Next, the 2D curve detector will find out the best curve representing the branch(see Figure 4.4).. 4.3 4.3.1. Connecting Branches Minimum Spanning Tree(MST). In the mathematical field of graph theory, given a connected, undirected graph G, a spanning tree T is a tree composed of all the vertices and some of the edges of G [28]. A single graph can have many different spanning trees. If the graph G is.

(26) 4.4 Mesh Production. 17. Figure 4.5. The minimum spanning tree of a planar graph. Each edge is labeled with its weight, which here is roughly proportional to its length.. weighted, the weight of the spanning tree is the sum of the weights of the edges in the spanning tree. A minimum spanning tree(MST) or minimum weight spanning tree is then a spanning tree with weight less than or equal to the weight of every other spanning tree(see Figure 4.5).. 4.3.2. Connecting Branches using MST. The weighted complete graph G = {V, E} is built by taking each branch as a node and connecting it to the other branches. The weight for each edge is computed using distance of two branches. The distance of two branches is defined as the minimum 3D Euclidean distance of the supported nodes on those branches penalized by the deviation of the ends:. d(p, q) =.  . d3D (p, q),.  d3D (p, q)/sin(α ∗ 0.5),. if p ∈ B1 and q ∈ B2 and (p 6= E1 or q 6= E2 ) otherwise.. (4.1) where B1 , B2 are supported node sets of two branches, E1 , E2 are two ends. The supported node is defined as the node on the curve that can be connected to the other. The weight on each edge reflects the likelihood that the two branches being connected in the real tree structure.. 4.4 4.4.1. Mesh Production Tree Allometry. There are many models formalizing the size and structure of trees. In 15th century, Leonardo da Vinci stated that “all the branches of a tree at every stage of its height.

(27) 18. Modeling of Laser-Scanned Trees. when put together are equal in thickness to the trunk” [29]. Pipe Model [30] [31] assumes that trees are composed of bundles of unit pipes and each unit pipe supplies nutrients to leaves. Thus, the sum of the cross-sectional areas of all child branches is equal to that of the parent branch. However, this model doesn’t obey mechanics. In Murray Model, the sum of the cross sectional area gets greater at each split: 2.49 2.49 rp2.49 = rc1 + rc2. (4.2). where rp is the radius of the parent branch, and rc1 and rc2 the radii of two children if a binary split. The angle between the directions of parent and each child can be described by following equation: cosθ =. rp4 + rc4 − (rp3 − rc3 )4/3 2rp2 rc2. (4.3). In our implementation, we use Murray’s law for branch splitting, and a simplified WBE model for non-splitting nodes. The same technique was used by Xu et al. [32]. 4.4.2. Skeleton Mesh. Once the radius has been estimated for each node in the skeleton, a ring is placed around each node perpendicular to its direction. At rendering time, a cyclic bark are applied to the trunk and branches. According to the density defined for different leaf location, we assign a quad with different number of leaves to them. Quad size is proportionally adjusted based on the number of leaves..

(28) Chapter 5. Results This chapter shows the results of the modeling pipeline. The process was implemented on a PC with Intel Core 2 2.33GHz and 2.0G of main memory. The constructed tree models are rendered using DirectX 9.0 on an nVidia GeForce 8800GT graphics card with 512MB video memory. The point clouds are scanned by Riegl Inc’s LMS-Z360I 3D imaging sensor. Textured are synthesized from photographs of the scanned trees. Figure 5.1 and Figure 5.2 demonstrates the most difficult case of a tree model constructed from a single point cloud (114,997 points). Even though the data from only one scan is very incomplete, our system is still capable of generating polygonal model that plausibly recreates the original tree. It only takes a few minutes to generate the tree model. Rendering of the models can be done in real time to provide timely feedback to the user.. 19.

(29) 20. Results. Figure 5.1. Mesh model rendered of the scanned tree.

(30) 21. Figure 5.2. Rendered mesh model from another view.

(31)

(32) Chapter 6. Conclusions and Future Work This report has presented a method of producing polygonal models from laserscanned trees. The whole process only takes a few minutes to create a plausible tree directly from the point cloud. User can also modify the rendered result in real time. This chapter discusses limitation of our method and mentions future work.. 6.1. Conclusion. This report has presented a semi-automatic method of producing polygonal models from laser-scanned trees based on user sketching. A 2D curve detection approach was proposed based on Efficient Ransac algorithm. Allometric theory was used to estimated the radii of tree skeleton. Our model approximates real tree structure as accurate as possible.. 6.2. Future Work. There are several strategies which can be combined or used separately to make the system more automatic and efficient. Image segmentation techniques and curve parsing techniques can be used to extract curvilinear structures or region of interest from the generated depth image. Ransac can also be used to detect the branch in 3D space directly.. 6.2.1. Image Segmentation Techniques. Curve extraction [21] [22] [23] [24] and curve tracing [25] [26] both involves classifying pixels as being curve points or not depending whether they lie on some curve in the image and linking cure points to form curves. Pattern recognition techniques such as matching-filters, ridge-based approaches, region growing, multiscale approaches, skeleton-based approaches, level set methods, tracking-based 23.

(33) 24. Conclusions and Future Work. Figure 6.1. Result from Tu [33]’s image parsing algorithm. approaches, Graph-cut approaches et al, are popular approaches in image segmentation. Tu [33] introduced an algorithm for parsing images into middle level vision representations - regions, curves, and curve groups(see Figure 6.1). Adaptions of such techniques are useful for extracting tree structure from single view.. 6.2.2. 3D Ransac Branch Detection. To make the system more automatic, detecting branches in 3D space is necessary. However, from our experiments, this process remains the difficulty for the following reasons. First, real tree branches are usually chaotic and irregular, which makes them impractical to fit curves. Secondly, defining single scoring function that works for all the branches is non-trivial, especially for those scanned from a glancing angle and those occluded by others..

(34) Bibliography [1] A. Hilton, A. J. Stoddart, J. Illingworth, and T. Windeatt. Marching triangles: range image fusion for complex object modelling. Image Processing, 1996. Proceedings., International Conference on, 1:381–384 vol.2, 1996. [2] F. Bernardini, J. Mittleman, H. Rushmeier, C. Silva, and G. Taubin. The ball-pivoting algorithm for surface reconstruction. IEEE Transactions on Visualization and Computer Graphics, 5(4):349–359, 1999. [3] Aristid Lindenmayer. Mathematical models for cellular interactions in development ii. simple and branching filaments with two-sided inputs. Journal of Theoretical Biology, 18(3):300–315, March 1968. [4] David S. Ebert, Kenton F. Musgrave, Darwyn Peachey, Ken Perlin, and Steven Worley. Texturing & Modeling: A Procedural Approach, Third Edition (The Morgan Kaufmann Series in Computer Graphics). Morgan Kaufmann, December 2002. [5] Przemyslaw Prusinkiewicz, Mark Hammel, Jim Hanan, and Radom R Mech. L-system: from the theory to visual models of plants. Machine Graphics and Vision, pages 365–392, 1996. [6] Jason Weber and Joseph Penn. Creation and rendering of realistic trees. In SIGGRAPH ’95: Proceedings of the 22nd annual conference on Computer graphics and interactive techniques, pages 119–128, New York, NY, USA, 1995. ACM. [7] Phillippe de Reffye, Claude Edelin, Jean Françon, Marc Jaeger, and Claude Puech. Plant models faithful to botanical structure and developmentr. In SIGGRAPH ’88: Proceedings of the 15th annual conference on Computer graphics and interactive techniques, pages 151–158, New York, NY, USA, 1988. ACM. [8] Peter E. Oppenheimer. Real time design and animation of fractal plants and trees. In SIGGRAPH ’86: Proceedings of the 13th annual conference on Computer graphics and interactive techniques, pages 55–64, New York, NY, USA, 1986. ACM. 25.

(35) 26. Bibliography. [9] Bernd Lintermann and Oliver Deussen. Interactive modeling of plants. IEEE Comput. Graph. Appl., 19(1):56–65, 1999. [10] Makoto Okabe, Shigeru Owada, and Takeo Igarashi. Interactive design of botanical trees using freehand sketches and example-based editing. Computer Graphics Forum, 24(3):487–496, 2005. [11] Takashi Ijiri, Shigeru Owada, and Takeo Igarashi. The sketch l-system: Global control of tree modeling using free-form strokes. Smart Graphics, pages 138– 146, 2006. [12] Xuejin Chen, Boris Neubert, Ying-Qing Xu, Oliver Deussen, and Sing Bing Kang. Sketch-based tree modeling using markov random field. ACM Trans. Graph., 27(5):109, 2008. [13] Alex Reche Martinez, Ignacio Martín, and George Drettakis. Volumetric reconstruction and interactive rendering of trees from photographs. ACM Trans. Graph., 23(3):720–727, 2004. [14] Boris Neubert, Thomas Franken, and Oliver Deussen. Approximate imagebased tree-modeling using particle flows. page 88, New York, NY, USA, 2007. ACM. [15] Artur Chodorowski, Morgan Langille Ulf Mattsson, and Ghassan Hamarneh. Color lesion boundary detection using live wire. SPIE Medical Imaging, 5747:1589–1596, 2005. [16] Ping Tan, Tian Fang, Jianxiong Xiao, Peng Zhao, and Long Quan. Single image tree modeling. In SIGGRAPH Asia ’08: ACM SIGGRAPH Asia 2008 papers, pages 1–7, New York, NY, USA, 2008. ACM. [17] H. Permuter and J. Francos. Gaussian mixture models of texture and colour for image database retrieval. In in Proc . ICASSP, pages 25–88, 2003. [18] Carsten Rother, Vladimir Kolmogorov, and Andrew Blake. "grabcut": interactive foreground extraction using iterated graph cuts. ACM Trans. Graph., 23(3):309–314, August 2004. [19] Ruwen Schnabel, Roland Wahl, and Reinhard Klein. Efficient ransac for point-cloud shape detection. Comput. Graph. Forum, 26(2):214–226, 2007. [20] Richard L. Burden and J. Douglas Faires. Numerical analysis: 4th ed. PWS Publishing Co., Boston, MA, USA, 1989. [21] J Canny. A computational approach to edge detection. IEEE Trans. Pattern Anal. Mach. Intell., 8(6):679–698, 1986. [22] C. Heipke, C. Steger, and R. Multhammer. A hierarchical approach to automatic road extraction from aerial imagery. In in Integrating Photogrammetric Techniques with Scene Analysis and Machine Vision II, pages 222–231, 1995..

(36) Bibliography. 27. [23] A. Baumgartner, C. Steger, C. Wiedemann, H. Mayer, W. Eckstein, and H. Ebner. Update of roads in gis from aerial imagery: Verification and multiresolution extraction. International Archives of Photogrammetry and Remote Sensing, 1996. [24] David Donoho. Applications of beamlets to detection and extraction of lines, curves and objects in very noisy images. In Nonlinear Signal and Image Processing (NSIP), 2001. [25] Carsten Steger. An unbiased detector of curvilinear structures. Technical Report FGBV–96–03, Forschungsgruppe Bildverstehen (FG BV), Informatik IX, Technische Universität München, July 1996. [26] Thomas W.Parks Karthik Raghupathy. Improved curve tracing in images. IEEE International Conference on Signal Processing, 2004. [27] Cemil Kirbas and Francis Quek. A review of vessel extraction techniques and algorithms. ACM Comput. Surv., 36(2):81–121, June 2004. [28] Thomas H. Cormen, Charles E. Leiserson, Ronald L. Rivest, and Clifford Stein. Introduction to Algorithms, Second Edition. The MIT Press, September 2001. [29] Leonardo da Vinci. The Notebooks of Leonardo da Vinci (Oxford World’s Classics). Oxford University Press, USA, October 1998. [30] Kichiro SHINOZAKI, Kyoji YODA, Kazuo HOZUMI, and Tatuo KIRA. A quantitative analysis of plant form-the pipe model theory : I.basic analyses. Japanese J. of Ecology, 14(3):97–105, 19640601. [31] K. Shinozaki, K. Yoda, K. Hozumi, and T. Kira. A quantitative analysis of plant form – the pipe model theory ii. further evidence of the theory and its application in forest ecology. Japanese Journal of Ecology, 14(4):133–139, August 1964. [32] Hui Xu, Nathan Gossett, and Baoquan Chen. Knowledge and heuristic-based modeling of laser-scanned trees. ACM Trans. Graph., 26(4):19, 2007. [33] Zhuowen Tu and Song-Chun Zhu. Parsing images into regions, curves, and curve groups. Int. J. Comput. Vision, 69(2):223–249, 2006..

(37)

(38)

No results found