Classification of leakage detections acquired by airborne thermography of district heating networks

Institutionen för systemteknik

Department of Electrical Engineering

Examensarbete

Classification of leakage detections acquired by

airborne thermography of district heating networks

Examensarbete utfört i Datorseende vid Tekniska högskolan vid Linköpings universitet

av Amanda Berg LiTH-ISY-EX--13/4678--SE

Linköping 2013

Department of Electrical Engineering Linköpings tekniska högskola

Linköpings universitet Linköpings universitet

Classification of leakage detections acquired by

airborne thermography of district heating networks

Examensarbete utfört i Datorseende

vid Tekniska högskolan vid Linköpings universitet

av

Amanda Berg LiTH-ISY-EX--13/4678--SE

Handledare: Erik Ringaby

isy_{, Linköpings universitet}

Jörgen Ahlberg

Termisk Systemteknik AB

Examinator: Vasileios Zografos

isy, Linköpings universitet

Avdelning, Institution Division, Department

Division of Computer Vision Department of Electrical Engineering SE-581 83 Linköping Datum Date 2013-05-27 Språk Language Svenska/Swedish Engelska/English   Rapporttyp Report category Licentiatavhandling Examensarbete C-uppsats D-uppsats Övrig rapport  

URL för elektronisk version

http://urn.kb.se/resolve?urn=urn:nbn:se:liu:diva-XXXXX ISBN

— ISRN

LiTH-ISY-EX--13/4678--SE Serietitel och serienummer Title of series, numbering

ISSN —

Titel Title

Klassificering av läckdetektioner erhållna genom flygburen termografering av fjärrvär-menätverk

Classification of leakage detections acquired by airborne thermography of district heating networks Författare Author Amanda Berg Sammanfattning Abstract

In Sweden and many other northern countries, it is common for heat to be distributed to homes and industries through district heating networks. Such networks consist of pipes buried underground carrying hot water or steam with temperatures in the range of 90-150◦C. Due to bad insulation or cracks, heat or water leakages might appear.

A system for large-scale monitoring of district heating networks through remote thermo-graphy has been developed and is in use at the company Termisk Systemteknik AB. Infrared images are captured from an aircraft and analysed, finding and indicating the areas for which the ground temperature is higher than normal. During the analysis there are, however, many other warm areas than true water or energy leakages that are marked as detections. Objects or phenomena that can cause false alarms are those who, for some reason, are warmer than their surroundings, for example, chimneys, cars and heat leakages from buildings. During the last couple of years, the system has been used in a number of cities. Therefore, there exists a fair amount of examples of different types of detections. The purpose of the present master’s thesis is to evaluate the reduction of false alarms of the existing analysis that can be achieved with the use of a learning system, i.e. a system which can learn how to recognize different types of detections.

A labelled data set for training and testing was acquired by contact with customers. Further-more, a number of features describing the intensity difference within the detection, its shape and propagation as well as proximity information were found, implemented and evaluated. Finally, four different classifiers and other methods for classification were evaluated. The method that obtained the best results consists of two steps. In the initial step, all detec-tions which lie on top of a building are removed from the data set of labelled detecdetec-tions. The second step consists of classification using a Random forest classifier. Using this two-step method, the number of false alarms is reduced by 43% while the percentage of water and energy detections correctly classified is 99%.

Nyckelord

Sammanfattning

I Sverige och många andra nordiska länder distribueras värme till hushåll och industrier genom fjärrvärmenätverk. Sådana nätverk består av nedgrävda rör i vilka vatten eller ånga med temperaturer mellan 90-150◦C flödar. På grund av dålig isolering eller sprickor i rören kan värme- eller vattenläckor uppstå. På företaget Termisk Systemteknik AB har ett system för storskalig övervakning av fjärrvärmenätverk genom fjärrtermografi utvecklats. Infraröda bilder över nät-verket tas från ett flygplan och en analys på bilderna utförs. Analysen hittar om-råden där marktemperaturen är högre än normalt. Förutom att detektera riktiga vatten- och värmeläckor så uppkommer också en hel del falsklarm. Föremål eller fenomen som kan orsaka falsklarm är sådana som av någon anledning är varmare än omgivningen, till exempel skorstenar, bilar och värmeläckage från hus. Systemet har använts i ett antal städer under de senaste åren och det finns därför en hel del exempel på olika typer av detektioner. Målet med examensarbetet är att undersöka hur stor reduktion av antalet falsklarm som kan uppnås med hjälp av ett lärande system. Det vill säga ett system som kan tränas att känna igen olika typer av detektioner.

Ett dataset bestående av märkta detektioner samlades in genom kontakt med kun-der. Dessutom definierades en rad egenskaper hos dessa baserat på fördelningen av intensitet inom detektionen, dess form och utbredning samt information från detektionens omgivning. Slutligen utvärderades fyra olika lärande system, så kal-lade klassificerare, och några andra metoder för klassificering.

Den metod som gav bäst resultat består av två olika steg. I det initiala steget plockas alla detektioner som ligger ovanpå hus bort från samlingen av märkta detektioner. Det andra steget består av klassificering av detektionerna med hjälp av en klassificerare av typen Random forest. Med denna slutgiltiga tvåstegsme-tod reduceras antalet falska detektioner med 43% medan andelen detektioner av typerna vatten och energi som klassificeras rätt ligger på 99%.

Abstract

In Sweden and many other northern countries, it is common for heat to be dis-tributed to homes and industries through district heating networks. Such net-works consist of pipes buried underground carrying hot water or steam with temperatures in the range of 90-150◦C. Due to bad insulation or cracks, heat or water leakages might appear.

A system for large-scale monitoring of district heating networks through remote thermography has been developed and is in use at the company Termisk Sys-temteknik AB. Infrared images are captured from an aircraft and analysed, find-ing and indicatfind-ing the areas for which the ground temperature is higher than nor-mal. During the analysis there are, however, many other warm areas than true water or energy leakages that are marked as detections. Objects or phenomena that can cause false alarms are those who, for some reason, are warmer than their surroundings, for example, chimneys, cars and heat leakages from buildings. During the last couple of years, the system has been used in a number of cities. Therefore, there exists a fair amount of examples of different types of detections. The purpose of the present master’s thesis is to evaluate the reduction of false alarms of the existing analysis that can be achieved with the use of a learning sys-tem, i.e. a system which can learn how to recognize different types of detections. A labelled data set for training and testing was acquired by contact with cus-tomers. Furthermore, a number of features describing the intensity difference within the detection, its shape and propagation as well as proximity information were found, implemented and evaluated. Finally, four different classifiers and other methods for classification were evaluated.

The method that obtained the best results consists of two steps. In the initial step, all detections which lie on top of a building are removed from the data set of labelled detections. The second step consists of classification using a Ran-dom forest classifier. Using this two-step method, the number of false alarms is reduced by 43% while the percentage of water and energy detections correctly classified is 99%.

Acknowledgments

There are a number of persons which have been involved during the course of this thesis to whom I would like to give thanks. First of all, I would like to thank my supervisors, Jörgen and Erik, for guiding me through this process and leading me back on track whenever needed.

Labelling of detections was made possible by Gun Bjurling at Vattenfall AB and Emma Sundin at Fortum Power and Heat AB.

Ola Friman for sharing his building segmentation Matlab code and Peter Follo for introducing me to IDL.

I would also like to give special thanks to all employees at the companies at Sen-sorum for making the coffee breaks a pleasure.

Finally, I would like to thank my family, especially Olov, for always supporting and believing in me. Without you, this would never have been possible.

Linköping, May 2013 Amanda Berg

Contents

1 Introduction 1

1.1 Background . . . 1

1.2 Problem description . . . 2

1.3 Aims and goals . . . 3

1.4 Method . . . 3

1.5 Limitations . . . 3

1.6 Outline of the report . . . 4

2 Basic concepts 5 2.1 Remote sensing using a thermal camera . . . 5

2.1.1 Thermal versus visual . . . 5

2.1.2 Camera used in the system . . . 7

2.2 Machine learning . . . 7

2.2.1 Fundamentals . . . 7

2.2.2 Classifiers . . . 8

2.2.3 Methods for classifier evaluation . . . 9

3 Description of the data set 15 3.1 Terms . . . 15 3.2 The system . . . 15 3.2.1 System overview . . . 16 3.2.2 Data collection . . . 16 3.2.3 Post-processing . . . 18 3.2.4 Analysis . . . 19 3.2.5 Visualisation . . . 20

3.3 The data set . . . 20

3.3.1 Design decisions . . . 23

3.3.2 Collection of labelled samples . . . 23

3.3.3 Class distribution . . . 24

4 Initial selection of features 27 4.1 Features based on intensity distribution . . . 27

4.1.1 Median intensity . . . 28

4.1.2 Four standardised moments . . . 28

4.1.3 Flatness . . . 29

4.2 Features based on shape and propagation . . . 29

4.2.1 Area . . . 29 4.2.2 Circumference . . . 30 4.2.3 Compactness . . . 30 4.2.4 Coverage . . . 30 4.2.5 Eccentricity . . . 30 4.2.6 Elongatedness . . . 31 4.2.7 Rectangularity . . . 32 4.2.8 Circularity . . . 32 4.2.9 Concentricity . . . 33 4.3 Proximity features . . . 33 4.3.1 Connected components . . . 33 4.3.2 Border average . . . 34

4.4 Features related to buildings . . . 34

4.4.1 On top of building . . . 34

4.4.2 Distance to building . . . 36

5 Experiments and results 39 5.1 Error measurement . . . 40

5.2 Initial pruning of data set . . . 40

5.3 Linear separability . . . 41

5.3.1 Brute force linear separability . . . 41

5.3.2 Linear Discriminant Analysis . . . 43

5.3.3 Support Vector Machines with linear kernel . . . 44

5.3.4 Summary of the evaluation of linear separability . . . 45

5.4 Classification . . . 45

5.4.1 Support Vector Machines with Radial Basis Function kernel 45 5.4.2 Random forest . . . 46

5.4.3 AdaBoost . . . 46

5.4.4 Anomaly detection . . . 47

5.4.5 Summary of the evaluation of classifiers . . . 47

5.5 Additional evaluated methods . . . 49

5.5.1 Individual classification methods . . . 49

5.5.2 Threshold invariant classification . . . 51

5.5.3 Voting . . . 52

5.6 Feature selection and evaluation . . . 53

5.6.1 Feature selection . . . 53 5.6.2 Feature evaluation . . . 55 5.7 Results . . . 59 6 Conclusion 63 6.1 Summary . . . 63 6.2 Discussion . . . 63 6.2.1 Features . . . 64

CONTENTS xi 6.2.2 Classification . . . 64 6.2.3 Feature selection . . . 65 6.2.4 Difficulties . . . 66 6.3 Future work . . . 66 6.3.1 Features . . . 67 6.3.2 Other . . . 67 A Images 69 A.1 Water . . . 71 A.2 Energy . . . 76 A.3 False . . . 78 B Feature histograms 87 B.1 Median intensity . . . 88 B.2 Mean intensity . . . 89 B.3 Standard deviation . . . 90 B.4 Skewness . . . 91 B.5 Kurtosis . . . 92 B.6 Flatness . . . 93 B.7 Area . . . 94 B.8 Circumference . . . 95 B.9 Compactness . . . 96 B.10 Coverage . . . 97 B.11 Eccentricity . . . 98 B.12 Elongatedness . . . 99 B.13 Rectangularity . . . 100 B.14 Circularity . . . 101 B.15 Concentricity . . . 102 B.16 Connected components . . . 103 B.17 Border average . . . 104 B.18 On top of building . . . 105 B.19 Distance to building . . . 106 Bibliography 107

1

Introduction

1.1

Background

In Sweden and many other northern countries, cities distribute heat to homes and industries through district heating networks. Such networks consist of pipes carrying hot water or steam, with temperatures in the range of 90-150◦

C, from a central power plant [8]. Compared to other alternatives, district heating is energy efficient and has both economic and environmental advantages [21].

Heat or water leakages due to bad insulation or cracks are common problems. The pipes degenerate with time [19] and in some cities the pipes have been used for several decades. The loss of water or energy caused by these leakages is ex-pensive and has negative impact on the environment [9]. It is therefore of great interest to the network owners to find methods to detect them. The fact that the pipes are placed underground increases the need of efficient and reliable meth-ods.

Some methods have been developed over the years, for example methods based on frequency response or change in electrical impedance for a thread installed inside the insulation of the pipe. It is also common to use liquid level switches which measures the flow of water or steam in the inlet and outlet. If the inlet and outlet flow differs, there is a leakage somewhere along the pipe. It might not be easy to localize the leakage based on these methods. They detect the presence but not the exact location. Status control of large networks is complex and time consuming, it is hard to get an overview of the status of the whole network [24]. At the company Termisk Systemteknik AB in Linköping a system for large-scale monitoring of district heating networks has been developed and is in use. The leakages are found using a thermal infrared camera mounted in an aircraft. The

resolution of the camera is enough to detect the differences in temperature that occur at ground level when a pipe has bad insulation or is leaking water. By using this system, the network owner quickly receives an overview of the whole network and maintenance can be done where it is most likely needed [24]. The system was originally developed at FOI1_{for the detection of mines in mine}

fields. In 2006 a project was started in collaboration with Göteborg Energi AB. The aim of this project was to evaluate the ability to perform large-scale moni-toring of district heating networks using thermal imagery. Thus the system was reworked to find water or heat leakages instead of mines. Today, the key compe-tence from the mine detection project lies with Termisk Systemteknik AB [24].

1.2

Problem description

A flight over a district heating network results in thousands of images. The im-ages are georeferenced and georectified, using their geographical position, and combined with the GIS2-information from the network owner of where the pipes are located. The GIS-information is used to mask the image in order to filter out the detections that are not even close to the network. Then, statistics of the ground temperature are calculated from the images and the most deviating pix-els in the high end of the distribution (i.e. the “warmest” pixpix-els) are marked as potential leakages [8].

However, this approach detects a lot of areas that are not real leakages but, for some reason, are warmer than the surroundings. For example, warm car engines or heat losses from houses. An operator usually has no problem in distinguish-ing the real from the false detections but it is almost impossible to scan tens of thousands of images manually. Therefore, developing a method for reducing the number of false detections is needed [24].

During the last couple of years, the system has been used in 17 Scandinavian cities and as a result of this there are now many examples of false and real detec-tions that can be used to design and train a statistical/learning system.

Crucial to the problem is also the asymmetric cost of incorrectly classified water leakages and incorrectly classified false alarms. The rejections of false alarms are a small cost in time for the human operator compared to the large economic costs in terms of water and/or energy losses that a missed real leakage will incur [8]. That is, it is important that a statistical/learning system applied to the data remove as few true water leakages as possible.

1_{Totalförsvarets forskningsinstitut}

1.3 Aims and goals 3

1.3

Aims and goals

The aim of this master’s thesis is to evaluate the increase in performance of the existing system that can be achieved with the use of a learning system. In this case, increase in performance means reduction of false alarms.

The reduction of false alarms should be done in a way that minimises the false positive rate while maintaining a true positive rate of 99%. The limit of 99% is a requirement from Termisk Systemteknik AB due to the asymmetric cost of false positive and false negative detections, described in the previous section.

1.4

Method

Initially, the system, algorithms and existing data were studied in order to get an understanding of the problem as well as what methods that could be applicable. In this case, understanding of data is crucial to the selection of features. A study of relevant articles and literature on different machine learning methods was also undertaken.

A collection of labelled data for training and testing was acquired by contact with customers. Two interviews were made, one with Gun Bjurling at Vattenfall AB and one with Emma Sundin at Fortum Power and Heat AB. This resulted in a number of labelled detections that could be used for training and testing. A majority of the labelled false alarms were however labelled by the student. The data processing algorithms are implemented in the programming language IDL3. Therefore, all feature extracting methods were implemented in IDL as well. Evaluation of machine learning methods and feature selection were done in Mat-lab with the use of the toolboxes PRTools4and DDTools5.

1.5

Limitations

The scope of the subject is wide and the time frame for this work is limited to 20 weeks. Therefore, all different aspects of the subject could not be studied in depth. For example, the number of evaluated classifiers is limited to four and feature selection is only performed using a naive method. Also, the number of collected labelled examples and implemented feature extraction techniques are limited due to this fact.

3_{Interactive Data Language, http://exelisvis.com} 4_{Pattern Recognition Toolbox [7]}

1.6

Outline of the report

In this chapter, some background information of the work has been presented. It is then followed by the chapter Basic concepts which contains a description of some of the basic concepts for this report. The subsequent chapters, Description of the data set and Initial choice of features, deal with the data set and chosen features while the chapter Experiments and results describes the details about what tests that have been done and what results that were obtained. Finally the conclusion, discussion of the results and future work are presented in the chapter Conclusion.

2

Basic concepts

This chapter contains some basic concepts necessary for the understanding of this report. Starting with some fundamental concepts of thermal infrared radiation, the chapter then moves on to describe machine learning, model assessment and model selection.

Further information about the subjects touched upon in this chapter can be found in [13], [3], [6] and [12].

2.1

Remote sensing using a thermal camera

For this application the use of a thermal camera for remote temperature measure-ments is central. In this chapter, some fundamental differences between working with a thermal and a visual camera are explained.

2.1.1

Thermal versus visual

Performing image processing in thermal images compared to visual images dif-fers in a number of ways. For example, feature descriptors invariant to illumi-nation and shadow effects are widely used in the visual domain. For thermal images, these effects do not exist. There are other effects that have to be taken into account instead. For example, weather effects, emissivity variations and at-mosphere propagation.

In the literature, thermal imagery is often combined with visual imagery, as in [15]. A subject of intensive research when it comes to image features in infrared images is face tracking. In [10], a summary of the progress is provided.

For visual imagery of the ground captured from an aircraft, presence of

Figure 2.1: Example of a thermal image captured from an aircraft during sunrise. All rooftops appear warmer on one side than the other.

tion from the sun, in the visual part of the electromagnetic spectrum, is crucial. For thermal imagery, this is not important. In this case, the sun can even be a source of error since its thermal radiation heats both the ground and objects on the ground. Buildings will for example appear warmer on one side than the other if the sun is not in its zenith. In figure 2.1 an example of this can be seen. This image was captured at sunrise and was the source of a number of false detections. Emissivity is the ratio of the actual emittance of an object to the emittance of a blackbody at the same temperature. Since emissivity is material dependant, it is an important property when doing temperature measurements with a thermal camera. For example, rooftops appear dark in the images since they cool off faster than the ground at night. They have a different emissivity than asphalt and trees, see example in figure 3.4. It is, however, hard to know the emissivity of every object in the image. Therefore, the emissivity is often approximated in some appropriate way. Exactly how temperatures are calculated from the raw intensity values is usually kept as a secret by the camera manufacturing company.

Something that needs to be taken into consideration when doing thermographic measurement is the interference of the atmosphere. Due to scattering by parti-cles and absorption by gases the atmosphere will attenuate radiation, making the measured apparent temperature decrease with increased distance. The level of attenuation depends on radiation wavelength, see figure 2.2. As can be seen in the figure, there are two main sections in which the atmosphere transmits a ma-jor part of the radiation. These are called the atmospheric windows and can be found between 3 and 5 µm (the mid-wave window) and 8 to 12 µm (the long-wave window) [23].

2.2 Machine learning 7

Figure 2.2: Atmospheric attenuation depends on radiation wavelength, therefore thermographic measurements are done within one of the two at-mospheric windows. The mid-wave window, 3 to 5 µm, or the long-wave window 8 to 12 µm. Here, the atmosphere transmits a major part of the radiation [2].

2.1.2

Camera used in the system

The camera used in the system is the FLIR mid-wavelength IR SC7000. It has a resolution of 640x512 pixels, a cooled detector, sensitivity up to 20 mK and a frame rate up to 3000 fps [1].

2.2

Machine learning

Machine learning is a subject of research and there is no room to describe all concepts and methods here. Therefore, only some basic concepts and methods that were used during this thesis are described in this section. In section 2.2.1, some fundamentals of machine learning are presented. It is then followed by descriptions of some common classifiers (section 2.2.2) and methods for classifier evaluation (section 2.2.3).

2.2.1

Fundamentals

Machine learning is the scientific study of systems that can learn from data. Be-low are descriptions of some of the fundamentals of machine learning.

Classification vs. regression

Classification assigns a class label to each sample. The result is discrete, i.e. the sample either belongs to a class or not. For regression, assigned is instead a con-tinuous value. Regression is suitable for problems where a value is to be pre-dicted, for example, the age of a person, while classification rather could solve the problem of deciding if the person is male or female.

Features

A feature is a description of some property of a sample. In [18] a feature is defined as "the specification of an attribute and its value". For example, a feature for a sample of the object human could be its height, shoe size or hair colour. Example of image features are features which describe the shape of the object, its colour or pixel intensities.

Supervised learning

For supervised learning, labelled samples are provided to the classifier to learn from. The classifier is unable to adapt to changes unless new labelled examples are provided, thus implying that using this method, the classifier can never be-come better than the one providing the labelled examples.

Generalisation ability

It is important for the classifier to be able to generalise when classifying sam-ples. If the decision boundary is too complex, the classifier is tuned for that specific data set and will not be able to generalise well for new samples not yet seen. This is related to the discussion of overfitting, section 2.2.3. In the same section, a method for evaluating the generalisation ability of a classifier, k -fold cross-validation, is described.

2.2.2

Classifiers

There is a whole range of different methods and combinations of methods for classification. Common for all is that they try to differentiate the samples based on the values of their features. If n is the total number of features and the features are scalars, then the features form an n-dimensional feature space in which all samples lie. The features can be multi-dimensional as well, in that case increasing the dimensionality of the space with the number of dimensions of the feature. Only brief explanations of each classifier are given due to the wide range of meth-ods. There is however a reference for each method for further reading.

Linear classifiers

First, there are the linear classifiers. These all try to optimise a hyperplane in the n-dimensional feature space which gives an optimal separation of classes. Examples of linear classifiers are

• _{Linear Discriminate Analysis (LDA) or Fisher’s Linear Discriminant [6].} Tries to minimize the variance within the classes while maximising the dis-tance between classes.

• _{Support Vector Machine [12][5].}

Finds the optimal hyperplane by maximising the margin, , between so called "Support Vectors" and the decision boundary. The Support Vectors are samples which lie close to the decision boundary.

2.2 Machine learning 9

Nonlinear classifiers

Nonlinear classifiers tries to map the feature space to a subspace in which the samples are linearly separable through a nonlinear transformation. Examples of nonlinear classifiers are

• _{Support Vector Machine with Nonlinear kernel [12].}

The nonlinear kernel maps the samples to a high-dimensional space in which they are linearly separable. In this new space, the linear Support Vector Machine algorithm is used.

• _{Decision tree [6].}

A decision tree is a tree-like graph or decision structure which, at each node of the tree, splits the samples into subsets through thresholding based on a feature. At each leaf of the tree, the samples left in that particular subset are classified.

Ensemble classifiers

The idea of ensemble classifiers is to create one strong classifier by combining several weak ones.

• _{AdaBoost [3] [12]}

Short for adaptive boosting. Boosts the performance of several weak clas-sifiers by combining them, having different weights for each sample and weak classifier. During training, the weights are updated between each new weak classifier. In each iteration, the weights for the misclassified samples are increased, causing the next weak classifier to take these samples into more account.

• _{Random forest [12] [4].}

A combination of individual decision trees where each tree is trained using a subspace of randomly selected features. The final classification result is then found by the majority vote of the different trees. The final result of the classifier is the consensus of the trees in the forest.

Anomaly detection

Anomaly detection, or one-class classifiers, are classifiers which are trained using only one class of samples, classifying all others as "outliers" or "anomalies". This method typically works well if there is a large number of samples for one class but few or none from the other.

2.2.3

Methods for classifier evaluation

In [6] the no free lunch theorem is explained. It basically implies that no sta-tistical method is better than the other if there is no prior knowledge about the problem. If a method seems to outperform another, it is only because of its match to the particular pattern recognition problem. That is, the choice of method de-pends on the problem.

x x

1 2

Figure 2.3: Illustration of overfitting. A too complex model will separate the training data (yellow rings and crosses) perfectly while previously unseen data (red cross) will be incorrectly classified. This phenomena is known as overfitting. A more simple model might perform better on test data.

The performance of the method on previously unseen data is an important part of the method evaluation. There is really no way of knowing this, but assumptions can be made, for example, by dividing the data set into a training and a validation set. Further down in this section, a method for minimisation of the generalisation error called k -fold cross-validation is explained.

Overfitting

Increasing the complexity of the classifier might result in perfect classification of the training samples. However, it is unlikely to perform well on previously unseen data, the classifier will not generalise well. This phenomena is known as overfitting and is visualised in figure 2.3. Consideration of the possibility of overfitting needs to be taken when adjusting the complexity of the classifier.

k -fold cross-validation

k -fold cross-validation is a way of evaluating the generalisation ability of a method by using k number of folds as illustrated in figure 2.4. In each iteration, the clas-sifier has to be retrained using N − N_k training samples. N is the total number of samples in the data set. This minimises the generalisation error of the method since it is set to the average generalisation error of the k folds.

The number of folds, k, depends on the objective. For k = N the estimation is approximately unbiased but has high variance. If k is lowered, the bias increases while the variance decreases. k = 10 is often recommended as a good compro-mise [16] [12] [6] and, therefore, it is that value of k that has been used for the evaluation of methods in this thesis.

2.2 Machine learning 11 Training Testing Testing Testing Testing 1 2 3 k N/k

Figure 2.4: Illustration of how data is divided into training and testing when using cross-validation. N is the total number of feature vectors in the data set and k is the number of folds.

Confusion matrix

The confusion matrix was first introduced in [18] and is a matrix showing the actual and predicted classifications. If the number of labels in the classification problem is L, then the confusion matrix is of size LxL. In figure 2.5 a general confusion matrix of size 2x2 is shown together with a more specific example. From figure 2.5 the following measurements for model assessment can be calcu-lated.

• _Accuracy

The accuracy is the proportion of the total number of predictions which are correct. This measurement can be misleading since it does not take into consideration the distribution of the labels. For example, consider a two-class two-classification problem where 90% of the data set consists of negative cases and 10% of positive cases. If all of the negative cases are classified correctly and none of the positives then the accuracy will be 90 % even though all positive cases are incorrectly classified.

Accuracy = a + d

a + b + c + d (2.1)

• _{True positive rate}

The true positive rate is the proportion of the positive cases which are cor-rectly classified, i.e., the proportion of water and energy leakages corcor-rectly classified.

T rue positive rate = a

a + b (2.2)

• _{True negative rate}

a

b

c

d

Negative

Positive

Positive

Negative

Actual

Predicted

(a) The general confusion matrix for a two-class classification problem.

a = true positives (actual leakages correctly classified

as leakages)

b = false negatives (leakages that were incorrectly classified as false alarms)

c = false positives (false alarms that were incorrectly

classified as leakages)

d = true negatives (actual false alarms correctly classified as false alarms) False alarm Leakage Leakage False alarm

Actual

Predicted

(b) A more specific example.

Figure 2.5: The confusion matrix for a classification problem with two la-bels. In (a) the general description is given, while in (b), it has been adapted to this problem.

been correctly classified, i.e., the proportion of false leakages correctly clas-sified.

T rue negative rate = d

c + d (2.3)

• _{False positive rate}

The false positive rate is the proportion of the negative cases which have been incorrectly classified as positives, i.e., the proportion of false leakages incorrectly classified.

False positive rate = c

c + d (2.4)

• _{False negative rate}

The false negative rate is the proportion of the positive cases which have been incorrectly classified as negatives, i.e., the proportion of water and energy leakages incorrectly classified.

False negative rate = b

a + b (2.5)

Receiver Operating Characteristic graph

In a Receiver Operating Characteristic (ROC) graph the true positive rate, equa-tion 2.2, is plotted on the y-axis against the false positive rate, equaequa-tion 2.4, on the x-axis. The true and false positive rate pairs are acquired by varying some

2.2 Machine learning 13 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1

False positive rate

True positive rate

Receiver Operating Characteristic graph

Figure 2.6: Example of a Receiver Operating Characteristics (ROC) graph. Ideally, the false positive rate would be zero and the true positive rate one.

parameter of the classification increasing the true positive rate at the cost of an increased false positive rate and vice versa. For example, the parameter could be the weights (importance) of the different classes. It encapsulates all informa-tion from the confusion matrix since the false negative rate is the complement of the true positive rate and the true negative rate is the complement of the false positive rate. An example of a ROC-graph is given in figure 2.6.

3

Description of the data set

Description of the data set has been divided into two main parts. Background information about the system on which the work is based upon necessary for the understanding of the data set, and then a more specific description of the data set used. But first, a clarification of some basic terms that will be used throughout the report.

3.1

Terms

An order is a physical order from a customer. It could include the whole district heating network of the customer, but it could also be only a limited part of the network.

An order consists of several flights depending on the size of the area that is to be covered. Large area implies that more flights have to be done in order to cover the whole network. The analysis of each flight is separate since the weather conditions might change between flights.

For each flight, detections are found at several percentage thresholds, 0.05%, 0.1%, 0.5%, 1%, 3% and 5%. These thresholds represent the 0.05%, 0.1%, and so on, warmest pixels within a pipe mask and flight. The concept is further de-scribed in section 3.2.4, Anomaly detection.

3.2

The system

As mentioned in section 1.1 the present master’s thesis is based on an in-house system from Termisk Systemteknik AB. Below are descriptions of the different

Data collection

Post-processing Analysis Visualisation

Figure 3.1: Overview of the system.

Figure 3.2: IR-camera mounted inside an aircraft of type Cessna.

parts, starting with a system overview.

3.2.1

System overview

The system consists of four main parts; data collection, post-processing, analysis and visualisation. In figure 3.1, an overview of the system can be seen. It was designed to be transparent to the user, relying mainly on characteristics such as area and temperature difference [8].

3.2.2

Data collection

The thermal images covering the district heating network are acquired from an aircraft. The mission is planned based on how many kilometres of district heating pipes that are to be examined. Parameters that need to be determined in advance are for example the course, height and velocity of the aircraft. The height and velocity are approximately the same for different flights, see figure 3.3. On-board is the thermal infrared camera and the computers that control the collection and store the thermal images, see figure 3.2. GPS coordinates for image acquisition are preset and at these positions the camera is triggered. This ensures that the whole area is covered and it also facilitates the post-processing. In figure 3.3 an example of a flight path for the aircraft can be seen.

3.2 The system 17

(a) The aircraft collects data at an altitude of about 700-800 m at a speed of about 180 km/h.

(b) Example of flight path (pink lines).

Figure 3.3: The flight path and positions are preset to ensure that the whole area is covered [24].

Figure 3.4: Image illustrating the effects of warm car engines and vegeta-tion when working with remote temperature measurements. The warm car engine is clearly visible on the driveway of the house, causing the algorithm to mark this as a detection if a district heating pipe passes in the vicinity of the car. Vegetation is blocking the measurement of the roofs of some of the houses as well as one of the roads.

In order to ensure the quality of the results, data collection should be done when conditions are favourable. Vegetation, warm car engines, ground heating and snow are examples that could affect the results. To minimise the number of false detections, collection of data is mainly done during the night or at dawn. At this time, the effect from sun heating is minimal and most objects have adopted a homogeneous ground temperature [8]. Also, at night, the streets are not covered with cars blocking the view. Ideally there should be neither snow nor foliage. This leaves two optimal periods for data collection, one during spring and one during autumn [24]. The effect of warm car engines and vegetation can be seen in figure 3.4 and an example of ground heating can be seen in figure 3.7.

Except for the thermal infrared camera mounted on-board the aircraft, there is a number of other sensors used as well. In each frame, the position and rotation of the aircraft is stored. There is also equipment for measuring ground temperature and weather conditions placed in the area [24].

3.2.3

Post-processing

The first step of the post-processing is to examine the quality of the collected data. This is done visually to make sure that the outcome looks as expected. Secondly, all thermal images are georectified to form a mosaic over the whole area. The radiometric information can, however, be lost during the transformation of the image and therefore all computations are done on the original thermal images [24]. The mosaic is solely used for visualisation, see section 3.2.5.

3.2 The system 19

Figure 3.5: Example of a pipe mask (right) and the corresponding georecti-fied image (left). [24].

3.2.4

Analysis

The GIS-information of where the pipes are located is provided by the network owner. This information is projected on top of the georectified images creating a pipe mask, figure 3.5, for each image. The mask is then used to limit the search for unnaturally high temperatures to areas above the pipes only [8].

Anomaly detection

Detections are treated as anomalies. Statistics of the ground temperature inside the pipe mask are calculated from all images and the most deviating pixels in the high end of the distribution (i.e. the "warmest pixels") are marked as potential leakages [8]. These pixels are found using several percentage thresholds, 0.05%, 0.1%, 0.5%, 1%, 3% and 5%. These thresholds represent the 0.05%, 0.1%, and so on, warmest pixels within the pipe mask and flight. The choice of thresholds is based on previous experience. In figure 3.6 an example of probability density functions acquired from all pixel intensities in all images and all pixel intensities inside a mask above the district heating pipes can be seen.

Rejection of false detections

When the present master’s thesis begun only two steps for rejection of false detec-tions were used. A detection had to have a certain size to be interesting. It also had to be distributed mainly within the pipe mask. Code for automatic building segmentation had been written [8] since a lot of the false alarms are connected to buildings, see section 4.4, but it was not used in the system at the time. Some additional rejection techniques had been tried but they had to be manually tuned for each city. For this reason and for the transparency of the method to the hu-man operator, these methods were all rejected and only the first two ones were used.

Figure 3.6: The probability density function for all pixel intensities (blue line) and all pixel intensities in a mask above the district heating pipes (red line). The filled tail corresponds to the 5% most deviating intensities in the high end of the distribution [24].

3.2.5

Visualisation

The software has been developed at Termisk Systemteknik AB and it consists of two different parts. FView is the tool used for easy navigation and quick overview while FView Analyse is used to make measurements in the thermal images. Both are described in more detail below.

FView

As mentioned in section 3.2.3, a mosaic is built from the georectified thermal im-ages. It is stored in a scale-space manner to allow the user to navigate and zoom in and out in the mosaic. The user sees the city through a bird’s eye perspective. Detections are marked along their borders and the level of the threshold can be chosen. The district heating network is also visualised. A snapshot from the visualisation tool can be seen in figure 3.7.

FView Analyze

The thermal images can be viewed in a separate window, called FView Analyze, and in this a number of different measurements can be done. Apart from point temperature measurements, it is also possible to see a temperature profile along a specified line drawn by the user as well as measuring the maximum, minimum and average temperature inside a box. An example of a temperature profile in FView Analyze can be seen in figure 3.8

3.3

The data set

The data set for the classification consists of labelled detections of three different classes; water, energy and false. A motivation for the choice of classes is given

3.3 The data set 21

Figure 3.7: Snapshot from the visualisation tool FView. The district heating network is marked with a blue line. Detections are marked with different colours based on the percentile threshold. In this case, red represents the most deviating pixels in the high end of the distribution, yellow the second most deviating pixels and green the third. The faint square visible in the center of the image shows which image is currently viewed in FView Ana-lyze. The reason for the immense amount of detections in this example is the presence of ground heating around the central square, i.e., a large but intentional energy leakage.

Figure 3.8: Example of a real leakage seen in FView Analyze. On top of the image, both a blue line and a box has been drawn by the user. A tempera-ture profile along the line can be seen in the lower half of the window. The maximum, minimum and average temperature inside the box can be seen in the left panel.

3.3 The data set 23

in section 3.3.1. The procedure for collection of labelled samples is presented in section 3.3.2 and their distribution in section 3.3.3.

3.3.1

Design decisions

The nature of the problem, labelled examples and distinct classes, led to the con-clusion that supervised learning and classification was the appropriate approach. During the initial study of the detections, it became clear that there were three main types of detections; water, energy and false detections. The detections which corresponds to water leakages are clearly of great interest to the customer. Energy leakages from pipes due to bad insulation are also interesting but since they generally have a quite distinct appearance (elongated along pipe, Gaussian distribution of intensity etc.) they should have their own class label. Examples of detections in the false class are ground heating, energy leakages from buildings, chimneys and warm car engines.

Ground heating is an example of a false detection where bad insulation is inten-tional. In figure A.11 both an example of ground heating and an example of a pipe with bad insulation can be seen. The intensity of the false detection is approximately equal throughout the whole detection, a property which differs from the detection of the pipe with bad insulation where the intensity rather has a Gaussian distribution. Also, the shape of the ground heating detection is rect-angular compared to that of the pipe.

In section 3.2.4 the anomaly-based approach for finding interesting areas was explained. This approach results in several layers of detections, each one corre-sponding to a different percentage threshold. Since both the feature itself and what features that are most important for discrimination might differ between these layers, a decision was made to make separate classifications for each thresh-old. During evaluation, voting between layers was evaluated further, see sec-tion 5.5.3.

3.3.2

Collection of labelled samples

The flights done during the last couple of years have resulted in thousands of thermal images from 17 different Scandinavian cities. A subset from this collec-tion was selected to form a data set for the evaluacollec-tion. Three of the most recent orders at the start of the thesis were chosen, namely the images from Uppsala, Hässelby and Southern Stockholm. This selection was based on the fact that the customers for these orders could provide ground truth, information about which detections had been investigated further and proven to be true water and energy leakages, or false alarms.

Two interviews were made, one with Gun Bjurling at Vattenfall AB (Uppsala) and one with Emma Sundin at Fortum Power and Heat AB (Hässelby and Southern Stockholm). This resulted in a number of labelled detections that could be used for training and testing. A majority of the labelled false alarms were however labelled by the student. This labelling was done carefully, only the detections

which, without a doubt, were false alarms were marked as such.

3.3.3

Class distribution

The distribution of the labelled samples for each percentage threshold and order can be seen in figure 3.9. In figure 3.10 the orders have been combined to form one single data set for each threshold.

The total number of samples differs for each order (Uppsala, Hässelby and South-ern Stockholm). For the evaluation, this should not be a problem since all features are normalised with respect to flight, see chapter 4. What could cause a problem is if a certain kind of detection is more common in one order than another, and the final method might then get adapted to a certain order if this order is over-represented. To avoid this, all evaluated machine learning methods are methods which can classify disjoint sample sets without taking sample density into con-sideration.

Since the labelled set at each percentage threshold is a subset of the labelled sets with a more permissive threshold, the total number of samples increases with an increasing threshold. This implies that a detection present at one threshold will also exist at all thresholds with a greater percentage. However, this is not always true since detections at more permissive thresholds tend to intersect and will then appear as one large detection.

As can be seen in figure 3.10, energy leakages are increasingly common as the threshold gets more permissive. This basically derives from the fact that energy leakages are seldom within the highest percentile. Also the false detections in-crease as the threshold gets more permissive. For water leakages the number of samples is however quite stable for all thresholds.

This might lead to the conclusion that all water leakages will be visible for ever more permissive thresholds. This is not true since, at some point, the detection of the water leakage will not only contain the leakage itself, but also a large por-tion of its surroundings. It might also intersect with other detecpor-tions and form large detections of several interesting areas. A property which is not wanted, the interesting areas should be marked individually including as little additional in-formation of its surroundings as possible. That is, increasing the threshold more than 5% is meaningless since already at this threshold, detections start to inter-sect.

3.3 The data set 25

Hässelby Södra Stockholm Uppsala 0 50 100 150 Number of samples Water Energy False (a) 0.05%

Hässelby Södra Stockholm Uppsala 0 50 100 150 Number of samples Water Energy False (b) 0.1%

Hässelby Södra Stockholm Uppsala 0 50 100 150 Number of samples Water Energy False (c) 0.5%

Hässelby Södra Stockholm Uppsala 0 50 100 150 Number of samples Water Energy False (d) 1%

Hässelby Södra Stockholm Uppsala 0 50 100 150 Number of samples Water Energy False (e) 3%

Hässelby Södra Stockholm Uppsala 0 50 100 150 Number of samples Water Energy False (f) 5%

Figure 3.9: Distributions of the samples for the different orders and thresh-olds.

0.05% 0.1% 0.5% 1% 3% 5% 0 50 100 150 200 250 300 350 400 450 Number of samples

Distribution of samples for the different percentage thresholds Water

Energy False

4

Initial selection of features

The existing thermal images and detections were initially studied visually with the help of the visualisation and analysis softwares FView and FView Analyze, see section 3.2.5. Based on the knowledge obtained from this study, a number of features believed to discriminate the different kinds of detections were chosen. As was explained in section 2.1.1, there are phenomena which exists in the visual domain that do not exist in thermal images, for example, illumination changes and shadows. Because of this, no consideration to these effects was taken when the features were chosen.

Features used in this system have to be normalised in order to be able to use and compare features from different flights. Since the aircraft from which the im-ages are captured flies at approximately the same altitude throughout the whole flight and also in other flights and orders, there is no need for the features to be invariant to scaling. The altitude (∼800m) is enough to imply that it takes a large change in altitude to make a difference in scale in the image plane. Nevertheless, the features have to be invariant to rotation in the image plane.

Below are descriptions and motivations for all of the initially chosen features. In Appendix A examples of water, energy and false detections cane found and in Appendix B histograms of all features can be seen.

4.1

Features based on intensity distribution

The distribution of intensity inside the detection was observed to discriminate different kinds of detections. Based on this observation, six different features were chosen. Each one is described in a subsequent section.

Figure 4.1: Example of a case where median intensity is a good feature. To the left, a blue line has been drawn across the three areas with unnaturally high temperatures. To the right, a profile along this line can be seen. Un-fortunately, the axis range of the graph do not scale well in FView Analyse for these extreme cases. Therefore, a blue box has also been drawn to the left and the maximum, minimum and average temperatures within this area have been included.

4.1.1

Median intensity

Some false detections have intensity values far above what is normal. For these, median intensity is a good feature. In figure 4.1 an example of this is shown. The detection in this example consists of three objects with unnaturally high temper-atures. Median intensity is normalised with the average intensity inside the pipe masks.

4.1.2

Four standardised moments

To describe the intensity distribution within a detection the four standardised moments mean, standard deviation, skewness and kurtosis were used.

Mean = ¯x = 1 N N −1 X i=0 xi (4.1) Standard deviation = v u t 1 N N −1 X i=0  xj−x¯ 2 (4.2) Skewness = 1 N N −1 X i=0 xj−x¯ Standard deviation !3 (4.3)

4.2 Features based on shape and propagation 29 K urtosis = 1 N N −1 X i=0 xj−x¯ Standard deviation !4 −_{3 (4.4)}

These moments all describe different properties of the intensity distribution. The mean value is the weighted average of all possible intensities, the standard de-viation describes the variation from the mean, skewness is a measurement of asymmetry and kurtosis measures the "Gaussianity" of the distribution. From the visual study of the data sets, it became clear that the intensity distribution was different for the different classes. For example, an energy leakage will have a lower standard deviation than most water leakages. There are also false alarms where the standard deviation is low. An example can be seen in figure A.9. Here, warm air has accumulated between two buildings next to each other.

4.1.3

Flatness

Flatness describes the sharpness of the maximum intensity peak. It is defined as the standard deviation of the intensity within a radius of five pixels from the maximum intensity value. Energy leakages usually have a high flatness rate com-pared to water. Water leakages tend to have a "hot spot" where the actual leakage is, with dropping temperatures as the water spreads out, resulting in a low flat-ness rate. False leakages on the other hand have values within the whole range. An example which clearly shows the difference between the flatness rate of energy and water leakages can be seen in figure A.7 and figure A.3.

The flatness is related to the feature of the second moment, which describes the standard deviation within the whole detection as opposed to the flatness which only operates in an area around the maximum intensity value. This implies that the flatness feature is approximately threshold invariant. The statement is ex-plained further in section 5.5.2.

4.2

Features based on shape and propagation

Shape and propagation of the detection were believed to be discriminating fea-tures as well. Descriptions of the chosen feafea-tures describing different aspects of shape and propagation are given below.

4.2.1

Area

Water leakages are sometimes, but not always, characterised by a large area, an example can be seen in figure A.5. But a large area can be connected to other phe-nomena as well, for example heated pavements. It is calculated by counting the number of pixels in the detection. No normalisation is needed since the altitude of the aircraft is approximately the same for different cities. A small change in altitude will not affect the pixel resolution.

(a) A compact shape

(b) A non-compact shape

Figure 4.2: Illustration of compactness (a) and non-compactness (b).

4.2.2

Circumference

A detection which is sprawled in the image plane will have a larger circumfer-ence than a more compact detection. From the visual evaluation, it seemed like true water leakages more often were sprawled than false ones, which had a more compact shape. The measurement is represented by the number of pixels of the object boundary. An outer boundary representation [25] is used, calculated by dilating the object and then removing the original object representation. No nor-malisation is needed for the same reason as for the area.

4.2.3

Compactness

Compactness, given by equation 4.5, is independent of linear transformations in this case since an outer boundary representation is used [25]. In an Euclidian space, the most compact region is a circle [25]. In figure 4.2 an example of a compact and a non-compact shape can be seen.

compactness = circumf erence

2

area (4.5)

Unnatural objects, for example, cars, chimneys and ground heating, are the sources for numerous false alarms. These detections tend to have more compact shapes than true water leakages, thus motivating the use of this feature.

4.2.4

Coverage

Coverage is a measurement of how large part of the detection that lies within the pipe mask. For example, a detection with a high coverage is elongated along the pipe. This is typical for an energy leakage. The measurement lies in the interval [0,1].

4.2.5

Eccentricity

Eccentricity is calculated by taking the ratio of the length of the longest chord a to the length of chord b which is the longest chord perpendicular to a, see figure 4.3. The direction of the longest chord is approximated by calculating the eigenvectors of the pixel coordinates of the detection. The eigenvector with the

4.2 Features based on shape and propagation 31

a b

Figure 4.3: Eccentricity is calculated by taking the ratio of the length of the longest chord a to the length of chord b which is the longest chord perpen-dicular to a.

Figure 4.4: A shape which is both elongated and curved will have a length/height ratio of the minimum bounding rectangle close to one.

largest eigenvalue is the direction of maximal variance.

Eccentricity is one way of measuring elongatedness, another one is described in the subsequent section. Energy leakages are usually quite elongated.

4.2.6

Elongatedness

The most intuitive way of measuring elongatedness of a shape might be to simply calculate the ratio of the length and width of the minimum bounding rectangle. This approach gives acceptable results but does not produce the wanted result if the shape is for example elongated and curved at the same time, see figure 4.4. Then the ratio will be close to one while it is easy to see with the eye that this is an elongated object. Another way of calculating the elongatedness is to use the maximum region thickness t and, for example, represent this with the number of erosion steps needed before the detection disappears, d. In [25] the elongatedness is evaluated as the ratio of the detection area and the squared thickness as in equation 4.6.

elongatedness = area

(t)2 (4.6)

where t = 2d and d is the number of erosions. d is multiplied with two since each erosion step reduces the maximum thickness by two.

An example of a true energy leakage can be seen in figure A.6. This is a typical energy leakage and as expected it is elongated along the pipe.

(a) Rectangularity is the ratio of the detec-tion area to the area of the minimum bounding rectangle.

(b) Circularity is the ra-tio of the detecra-tion area to the area of a circle having the same perime-ter.

Figure 4.5: Illustration of rectangularity (a) and circularity (b).

4.2.7

Rectangularity

Rectangularity is defined as the ratio of the detection area to the area of the minimum bounding rectangle, see equation 4.7. To find the absolute minimum bounding rectangle a brute force method is used. A bounding rectangle is rotated through k directions in discrete steps and in each direction the rectangularity Fk

is calculated. The direction for which Fkis maximised is the direction of the

mini-mum bounding rectangle. Due to the symmetry of a rectangle the rotation needs only to be done in the first quadrant. In figure 4.5 an illustration of rectangu-larity can be seen. Cars, chimneys and ground heating are examples of objects which have a high rectangularity. An example of ground heating outside an en-trance can be seen in figure A.11, and examples of chimneys causing false alarms is found in figure A.16.

rectangularity = max

k Fk (4.7)

4.2.8

Circularity

Circularity is the ratio of the detection area to the area of a circle having the same perimeter [11]. The area of a circle with perimeter P is calculated as P_4π2. The circularity ratio is then given by

circularity = 4πA

P2 (4.8)

where A is the detection area. An illustration of circularity is given in figure 4.5. Again, unnatural objects, which are the sources of numerous false alarms, tend to have regular forms, such as a rectangular or a circular shape. In figure A.12, a large number of circular false alarms can be seen. But, also true water

leak-4.3 Proximity features 33

a _b

c

d

Figure 4.6: Illustration of concentricity, see equation 4.9. The circle repre-sents the pixel within the detection that have the maximum intensity value.

ages are, to some extent, characterised by a circular shape. This is confirmed by the histograms in figure B.14 where water leakages and false alarms have higher circularity values than energy leakages.

4.2.9

Concentricity

Concentricity is a measurement of how central the maximum intensity value within the detection is. It is illustrated in figure 4.6 and is calculated as

concentricity = min(a b, b a) + min( c d, d c) (4.9)

Equation 4.9 is the sum of the chord ratios along each dimension. Since the image is 2-dimensional, the concentricity for the detection will lie in the interval [0,2]. If the maximum intensity value is placed in the centre of the detection in both directions, then the concentricity will be close to two. The directions of ab and cd are found by calculating the eigenvectors of the detection.

A real water leakage will, unless the pipe lies in a slope, have a maximum inten-sity which is located in the center of the detection. Meanwhile, heat leakages or leakages of warm air from buildings will have a maximum intensity close to the edge of the detection, right by the building wall. Examples of warm air leaking from buildings can be seen in figure A.8, figure A.13 and figure A.15. In all these examples it can be seen, by looking at the profile perpendicular to the building wall, that the maximum intensity value is located close to the wall.

4.3

Proximity features

When a human user tries to find information that can help discriminate between the different kinds of leakages, he or she takes proximity information into consid-eration. For example, if the detection lies close to a building, if there are several similar detections nearby, and so on. Therefore, some proximity features were included as well.

4.3.1

Connected components

Connected components is the number of detections that lie within a radius of one quarter of the image width to the detection for which the feature is currently

calculated. This is a topological feature, i.e. a feature that is unaffected by any deformation as long as no tearing or joining of the detection is done [11]. In the case of an energy leakage the whole area above the pipe is often covered with numerous detections. Furthermore, false alarms tend to, to some extent, be surrounded by other false alarms.

4.3.2

Border average

Roof-tops of buildings tend to have lower intensity than other objects in the ther-mal images since they cool off faster than their surroundings at night. Border average is the mean intensity within an area of ten pixels from the detection bor-der, not including the detection itself. If the detection then lies close to a building, the border average measurement will be lower than, for example, a detection in the middle of the road.

4.4

Features related to buildings

Many false leakages can be connected to buildings. Heat leakages from buildings, chimneys and atriums are common sources, an example of an atrium at the roof of a shopping mall can be seen in figure A.14. Chimneys and atriums are found with the help of a building mask, see section 4.4.1, while discriminating building heat leakages from water leakages at driveways is a bit more intricate. It was, however, discovered that heat leakages have their maximum intensity value close to the building wall while driveway water leakages have their maximum intensity value at the location of the leakage. To help separate between these two cases, the border average (section 4.3.2), concentricity (section 4.2.9) and distance to building (section 4.4.2) features together were hoped to be of use. An example of a water leakage at a driveway can be seen in figure A.1 and a heat leakage from a building in figure A.10.

4.4.1

On top of building

Sometimes district heating pipes pass right beneath buildings. If the building then has some sort of chimney or atrium this is marked as a detection when it obviously is not. To remove these false detections, a building mask was created. Two different approaches were tested, each one described below together with its positive and negative properties.

1. The building segmentation code from [8] based on gradient thresholding and AdaBoost was used to create a building mask.

+ No extra information needed, only the thermal images. - The code was written in Matlab, had to be rewritten in IDL.

- Had only been trained using labelled examples from one order, more exam-ples had to be labelled.

4.4 Features related to buildings 35

Figure 4.7: Example of a building mask (right) created from a freely licensed map (left) by thresholding based on colour and performing some morpho-logical operations.

2. A building mask from the freely licensed map http:\www.openstreetmap. orgwas made by thresholding based on colour. Some morphological oper-ations were then performed to minimize the error due to holes in the mask caused by text. An example of a building mask can be seen in figure 4.7. + Threshold based on colour, simple!

+ Geotiff format, i.e. for each pixel there is information about its position in world coordinates.

- There is sometimes text in the map on top of buildings. These map images are obtained through the software Globalmapper and there is no option to exclude the text. This might lead to holes in the mask!

- Since the map is open source, the accuracy depends on the accuracy of the added information from the contributors. Sometimes whole areas of build-ings are missing.

Both alternatives were tested and evaluated, and alternative two was chosen. The evaluation and reason for this choice is explained below.

In order to be able to evaluate alternative one, new samples from other orders had to be labelled and the code had to be implemented in IDL. The features were also normalised. The best training accuracy achieved was 83% and the best test accuracy 80%. In [8] the resulting training accuracy is 97% and testing accuracy 86%. The main difference between the two methods is the data set that was used for training. In figure 4.8 two examples of classified buildings in thermal im-ages are shown. The arrows indicate some problem areas where ground has been classified as buildings.

Missed buildings, false negatives, are less of a problem than other objects classi-fied as buildings, false positives. False negatives only results in incorrect classifi-cation of a false alarm, while false positives might result in a true water leakage being classified as a false alarm. This is not acceptable.

(a) (b)

Figure 4.8: Two examples of the performance of the AdaBoost building clas-sifier. The green boundaries indicate the areas that have been classified as buildings. The arrows show the problem areas. Unacceptable classification of ground as buildings which could lead to missed real leakages.

Furthermore, if the delivered thermal images for an order contains images cap-tured after or at sunrise (this actually happens sometimes) one half of the rooftops will look significantly warmer than the other. An example is provided in fig-ure 2.1. At these occasions, the building segmentation method will fail.

The actual accuracy of alternative two is hard to measure since it depends on the accuracy of the added information from the contributors. However, some exam-ples were studied and for most cases the accuracy is acceptable, see examexam-ples in figure 4.9. Only one insufficient result was found, see figure 4.10.

For alternative two, no false positive examples have been observed. There are some false negatives, but since they only result in incorrect classification of false alarms, the results are acceptable. In particular, the results outstand those of alternative one.

For the reasons explained above, alternative two was chosen to use for the cre-ation of building masks. Each pixel in the detection was then transformed into world coordinates and a percentage of how large part of the detection that lies on top of a building was calculated.

4.4.2

Distance to building

Distance to building is the distance from the maximum intensity value within the detection to the closest building.