Visualization of Traffic Information in Head-Mounted Displays for Helicopter Navigation

U

M

T

H

D

& S

Master’s Thesis

Visualization of Traffic Information in Head-Mounted Displays for Helicopter

Navigation

Johannes Maria Ernst

Author: Johannes Maria Ernst

Email: jernst@kth.se

Person Number: 871120-1254

Supervisor KTH: Prof. Dr. Ulf Ringertz Supervisor TUM: Dipl.-Ing. Franz Viertler Supervisor Airbus: Ferdinand Eisenkeil, M.Sc.

April 2015

I, Johannes Maria Ernst, confirm that the work presented in this thesis has been performed and interpreted solely by myself except where explicitly identified to the contrary. All verbatim extracts have been distinguished by quotation marks, and all sources of information have been specifically acknowledged. I confirm that this work has not been submitted elsewhere in any other form for the fulfillment of any other degree or qualification.

Garching, Monday 27

April, 2015

One of the major challenges for helicopter pilots are low level flights and landings in degraded visual environments (DVE). Without proper assistance systems, the pilots are prone to lose their situational awareness (SA) when fog, heavy precipitation, limited sunlight and stirred-up sand or snow degrades their view. In recent years, various synthetic and enhanced vision systems were de- veloped so as to assist the pilots in these demanding situations. This work enhances the existing sys- tems by proposing a concept for the visualization of traffic information in helmet-/head-mounted displays (HMD). The intuitive representation provides additional cues about the environment and decreases the pilots’ workload, especially during flights in offshore windparks or while search and rescue operations with many other vehicles operating within a small range.

The thesis, which was created at Airbus Defence & Space Friedrichshafen, analyzes the strengths and weaknesses of conventional head-down cockpit displays of traffic information (CDTIs) and of a very basic head-up traffic cueing system. Based on the results of this assessment, a prototype implementation of an integrated traffic visualization concept is developed where both display types – HDD and HMD – complement each other in their roles. All traffic information is retrieved from the on-board ADS-B system, which provides data such as the position and velocity vector of ADS-B equipped vehicles in the vicinity.

The main focus of the work is placed on the development of methods for de-cluttering the HMD and increasing the information content of the head-up symbology by coding important parame- ters visually. In order to create a de-cluttered representation, the high-dimensional traffic data is clustered with a derivation of DBSCAN so as to identify groups of traffic that can be visualized by a single symbol. Ellipsoidal shapes are applied to airborne traffic while rectangular forms high- light vehicles on ground. To avoid distraction to the pilot due to sudden symbology changes when clusters are merged or split up, a smooth transition visualization method is developed. For the graphical representation of important information on the HMD, a measure indicating the threat potential of an intruder is proposed. This parameter, which is derived from the classification ap- proach of the TCAS II, is visualized by the color of the traffic symbol. In analogy to the TCAS, it is faded from green via yellow to red when the time to a predicted conflict decreases.

The realized functions are integrated into a flight simulator and tested with synthetic ADS-B data.

Furthermore, the proposed algorithms are evaluated in terms of their effectiveness and run-time

efficiency, and finally the advantages and limitations of the approach are discussed. Now, the

application must be integrated into an operational synthetic vision system so as to conduct further

tests.

Abbreviations and Symbols III

List of Figures VI

List of Tables VIII

List of Algorithms IX

1. Introduction 1

2. Theoretical Background and Related Work 4

2.1. Automatic Dependent Surveillance-Broadcast . . . . 4

2.1.1. Introduction to Aeronautical Surveillance Systems . . . . 4

2.1.2. Functional Principle of the ADS-B Technology . . . . 5

2.1.3. Benefits of ADS-B . . . . 7

2.1.4. Implementation Details . . . . 7

2.2. The Traffic Alert and Collision Avoidance System (TCAS) . . . . 9

2.2.1. Introduction to TCAS II . . . . 9

2.2.2. The TCAS II Logic . . . . 10

2.2.3. TCAS Displays . . . . 13

2.3. Cluster Analysis . . . . 13

2.3.1. Introduction to Cluster Analysis . . . . 14

2.3.2. Brief Overview of Important Clustering Methods . . . . 15

2.3.3. The DBSCAN Algorithm . . . . 16

2.4. Head-Mounted Displays for Increasing Situational Awareness . . . . 19

3. Development of an Integrated Traffic Visualization Concept in HMDs and HDDs 22 3.1. Existing Head-Down Cockpit Displays of Traffic Information . . . . 22

3.2. Basic Traffic Visualization in Head-Mounted Displays . . . . 23

3.3. Integrated Head-Up and Head-Down Traffic Visualization . . . . 26

3.3.1. Complementary Roles of the Head-Down and Head-Up Traffic Visualization . 27 3.3.2. System Architecture . . . . 27

4. Realization of the Head-Down Traffic Display 30 5. Advanced Traffic Visualization in Head-Mounted Displays 32 5.1. De-Cluttering of the Display . . . . 32

5.1.1. Clustering to Identify Groups of Similar Traffic . . . . 34

5.1.2. Traffic Group Symbology . . . . 37

5.1.3. Smooth Merging and Splitting of Traffic Groups . . . . 40

5.2. Visual Coding of Important Information . . . . 50

5.2.1. Threat Potential Measure . . . . 51

5.2.2. Horizontal and Vertical Reference Lines . . . . 57

5.3. The Final Advanced Traffic Visualization . . . . 57

6. Evaluation and Discussion of the Proposed Visualization Concepts 60 6.1. Comparison of Basic and Advanced Head-Up Visualization . . . . 60

6.1.1. Drawing Effort . . . . 60

6.1.2. Time Consumption . . . . 63

6.1.3. Summary . . . . 66

6.2. Discussion of the Developed Traffic Visualization Concept . . . . 66

7. Summary and Conclusion 69

A. Contents of the Attached Data CD 71

Bibliography 72

Abbreviations

ACAS Airborne Collision Avoidance System

ADS-B Automatic Dependent Surveillance-Broadcast ASA Aircraft Surveillance Application

ATC Air Traffic Control

ATILa Avionics Test and Integration Laboratory ATM Air Traffic Management

ATSAW Air Traffic Situational Awareness

BIRCH Balanced Iterative Reducing and Clustering using Hier- archies

CDTI Cockpit Display of Traffic Information CPA Closest Point of Approach

DBSCAN Density Based Spatial Clustering of Applications with Noise

DLR Deutsches Zentrum für Luft- und Raumfahrt DMOD Distance MODifier

DVE Degraded Visual Environment

ES Extended Squitter

EVS Enhanced Vision System FAA Federal Aviation Administration FLIR Forward-Looking InfraRed FLTID Flight Identification

FMS Flight Management System FOV Field of View

GDBSCAN Generalized Density Based Spatial Clustering of Applica- tions with Noise

GNSS Global Navigation Satellite System GPS Global Positioning System

HDD Head-Down Display

HMD Head-/Helmet-Mounted Display

HUD Head-Up Display

IFF Identification Friend Foe

KDD Knowledge Discovery in Databases LiDAR Light Detection And Ranging

MASPS Minimum Aviation System Performance Standards MFD Multi-Function Display

MLAT Multi-LATeration

MOPS Minimum Operational Performance Standards NASA National Aeronautics and Space Administration ND Navigation Display

NextGen Next Generation Air Transportation System OpenGL Open Graphics Library

OPTICS Ordering Points to Identify the Clustering Structure PCA Principal Component Analysis

PFD Primary Flight Display PSR Primary Surveillance Radar RA Resolution Advisory

RTCA Radio Technical Commission for Aeronautics SA Situational Awareness

SAR Search and Rescue

SESAR Single European Sky ATM Research SL Sensitivity Level

SSR Secondary Surveillance Radar TA Traffic Advisory

TCAS Traffic Alert and Collision Avoidance System UAS Unmanned Aircraft System

UAT Universal Access Transceiver

VSI Vertical Speed Indicator

WAM Wide Area Multilateration

XVS eXternal Visibility System

List of symbols

N Iterations per Smooth Transition Visualization Interval

² Similarity Limit (DBSCAN) ν Smooth Transition Progress ψ,θ Relative 3D Bearing of an Intruder τ Warning Time (TCAS)

Θ Threat Potential

X

, Y

Principle Coordinates of a Cluster X , Y Window Coordinates

x, y, z Position in a World-Fixed North-East-Down Frame of

Reference

1.1. The SFERION head-up symbology . . . . 1

1.2. Examples of possible head-up traffic display applications . . . . 2

2.1. The functional principle of ADS-B . . . . 6

2.2. The ADS-B system components . . . . 8

2.3. The TCAS II installation . . . . 10

2.4. The TCAS II protected volume . . . . 12

2.5. The TCAS II threat resolution logic . . . . 12

2.6. The TCAS II traffic visualization . . . . 13

2.7. The main stages in clustering . . . . 14

2.8. Examples of important clustering methods . . . . 16

2.9. The density-based notion of clusters . . . . 17

2.10.The DBSCAN algorithm . . . . 18

2.11.The SFERION product family . . . . 20

2.12.Obstacle avoidance with the help of a LiDAR sensor . . . . 20

2.13.Symbology features of the SFERION pilot assistance system . . . . 21

3.1. Examples of different CDTIs . . . . 23

3.2. A basic traffic visualization . . . . 24

3.3. Overlapping symbols on the basic HMD . . . . 24

3.4. Altitude misinterpretation caused by the perspective view . . . . 25

3.5. Missing flight direction perception due to the perspective view . . . . 26

3.6. The system architecture of the ADSB traffic visualization software . . . . 28

4.1. The head-down traffic display . . . . 30

4.2. The head-down traffic symbol layouts . . . . 31

5.1. Traffic situation causing cluttering on the basic HMD . . . . 33

5.2. The three-dimensional bearing . . . . 36

5.3. Approaches to the visualization of a traffic group . . . . 38

5.4. The definition of a traffic group symbol . . . . 40

5.5. The smooth visualization routine . . . . 41

5.6. The step-wise visualization of smooth symbol transitions . . . . 42

5.7. Details of the smooth transition algorithm . . . . 43

5.8. Data flow within the smooth visualization functional block . . . . 45

5.9. A multiple fusion and splitting process . . . . 46

5.10.Iterative splitting of a cluster . . . . 49

5.11.The graphical representation of the threat potential . . . . 51

5.12.The frames of reference used for the threat potential calculation . . . . 52

5.13.Parameters used by the Euclidean vector algebra method . . . . 54

5.14.The horizontal and vertical reference lines . . . . 57

5.15.Screenshot of the developed symbology in the ATILa . . . . 58

5.16.The connection between HMD and HDD . . . . 58

5.17.Effects of different clustering strategies . . . . 59

6.1. The influence of radius and line width on the number of pixels . . . . 61

6.2. The number of pixels drawn by the basic and the advanced symbology . . . . 62

6.3. The number of pixels during a fusion process . . . . 63

6.4. The computation times required by the basic and the advanced visualization . . . . 64

6.5. Dependency of the computation time on the number of symbol vertices . . . . 65

2.1. Aeronautical surveillance systems . . . . 5

2.2. Parameters broadcast by ADSB equipped vehicles . . . . 8

2.3. The TCAS II parameters . . . . 11

1. The determination of the transitions to be visualized . . . . 48

2. The computation of the threat potential . . . . 55

In modern helicopter flight decks, Enhanced Vision Systems (EVS) incorporating head-tracked Head-/Helmet-Mounted Displays (HMDs) are used to improve flight safety, especially during op- erations in Degraded Visual Environments (DVEs) such as brownout landings and flights during night-time or in adverse weather conditions [11], [36]. These displays decrease the pilot’s head- down time and increase his

situational awareness by displaying important information virtually superimposed on the out-the-window view. As can be seen in Figure 1.1, this involves simple two-dimensional symbols like the speed and altitude tapes but also three-dimensional symbology.

The latter conforms with the real world behind in order to highlight terrain contours or a desired landing location.

(a) 3D conformal symbology (terrain grid with con-

(b) 3D conformal landing symbology with dog-

Figure 1.1.: The head-up symbology of the SFERION pilot assistance system by Airbus Defence and

Within the framework of the Next Generation Air Transportation System (NextGen)[21] and the Single European Sky ATM Research (SESAR)[48], a new aeronautical surveillance system called Automatic Dependent Surveillance-Broadcast (ADS-B) has been introduced. This satellite-based technology, which will be mandatory for many aircraft, allows for precise determination of an aircraft’s, ground vehicle’s or ship’s position, speed and supplemental data, which is then broadcast to be received by air traffic controllers and other vehicles in the vicinity.

1For better readability, male personal pronouns will be used throughout this text even though it applies to both genders equally.

The idea of this master’s thesis is to combine the benefits of these two existing technologies by visualizing received ADS-B data of surrounding traffic on an HMD. Such a system will assist he- licopter pilots flying in offshore windparks or taking part in search and rescue operations where many other vehicles operate within a small range. Since the pilots have to cope with high workload during such scenarios, the traffic visualization must present the required information in a way that is easy and intuitive to use and does not unnecessarily distract the pilot. To achieve this and to conform with additional requirements of head-up see-through symbologies, an integrated traffic visualization concept is developed in this thesis: a head-down and a head-up display complement each other in their roles so as to combine the advantages of both representations.

(a) A close formation of Coast Guard heli-

(b) Two helicopters operating inside an offshore

Figure 1.2.: Exemplary scenarios where a head-up traffic display increases the situational awareness.

The thesis is structured as follows. Chapter 2 introduces the ADS-B technology as well as the Traffic Alert and Collision Avoidance System (TCAS), which will be relevant to the realization of a threat potential indication within the traffic symbology. Additionally, a brief review of clustering methods is presented as such a technique is applied to the analysis and classification of the received traffic information. Finally, this part of the thesis gives an overview of related work in the field of pilot assistance systems incorporating head-mounted displays for increasing situational awareness.

This is followed by the definition of an integrated traffic visualization concept in head-mounted and head-down displays in Chapter 3. Within that, the benefits and drawbacks of different traffic vi- sualizations are analyzed so as to develop an advanced concept. The first module of this integrated system, the head-down Cockpit Display of Traffic Information (CDTI), is described in Chapter 4.

Thereafter, the main part of this work, the implementation of the advanced head-up traffic visual-

ization is presented. A clustering algorithm is adapted and used to analyze the high-dimensional

traffic data in order to identify groups of similar traffic that can be visualized by a single symbol in

the HMD. Among other techniques, this shall avoid display clutter by decreasing the number of

rendered symbols. In addition to that, the symbology presented in Chapter 5 offers supplemental

information about the cued traffic group. This includes parameters like the number and type of the

contained vehicles as well as a measure of the formation’s threat potential, based on the predicted

closest point of approach. To further decrease the pilot’s workload by avoiding distracting changes

in the visual representation of clusters, a smooth transition visualization algorithm is developed

for splitting and merging clusters.

The thesis concludes with an evaluation and discussion of the advanced traffic visualization ap-

proach in Chapter 6. To do so, a set of evaluation scenarios is used to compare the drawing effort

and the time consumption of the proposed symbology with a very basic, unclustered head-up

traffic cueing. Furthermore, potential advantages and drawbacks are named before an outlook

on future work on this topic is given in Chapter 7. This includes the integration into an opera-

tional synthetic vision system and a thorough validation of the visualization concept by means of

a simulator study.

This chapter provides the theoretical background that is needed for the reader to understand the concepts and methods developed within this thesis. It includes an explanation of the ADS- B technology, a presentation of the Traffic Alert and Collision Avoidance System (TCAS) and an introduction to the field of cluster analysis. Lastly, work related to the treated topic is presented in this chapter.

2.1. Automatic Dependent Surveillance-Broadcast

The following section presents the surveillance technology ADS-B. The traffic data provided by ADS-B forms the basis for visualization concept developed within this thesis. At first, aeronautical surveillance systems in general are introduced. Subsequently, the functional principle of the ADS-B technology is presented and a few details about the implementation are given.

2.1.1. Introduction to Aeronautical Surveillance Systems

According to ICAO Doc 9924 [25], an Aeronautical Surveillance System “provides the aircraft po- sition and other related information to Air Traffic Management (ATM) and/or airborne users”. In its simplest realization, it provides only the aircraft’s position at a certain time. More advanced designs allow the user to get information on the identification, the speed, the intent and other characteristics of the aircraft.

Depending on the technologies used, it can be distinguished between three major types of aeronau- tical surveillance systems: Independent Non-Cooperative Surveillance, Independent Cooperative Surveillance and Dependent Cooperative Surveillance [16]. Details about this classification and existing technologies are given in the subsequent paragraphs as well as in Table 2.1.

Independent Non-Cooperative Surveillance [16], [50] Independent non-cooperative surveil-

lance systems do not rely on the co-operation of the aircraft, which means that an aircraft does not

have to be equipped with any on-board surveillance equipment to be detect-able. Additionally,

the ranging is independent of other positioning systems as it directly determines the position of

the detected aircraft through its reflected radio waves. However, by using this technology it is nei-

ther possible to identify the aircraft nor is it possible to gather any additional information about

the detected aircraft. A famous example of such a system is the Primary Surveillance Radar (PSR) developed before and during World War II.

Independent Cooperative Surveillance [16], [50] In contrast to the technology described above, systems belonging to this type of surveillance systems communicate with the aircraft to be identi- fied. Therefore, it is possible to request supplemental data such as the identification or the current airspeed. To do so, each aircraft must be equipped with a radio receiver and transmitter, referred to as transponder. This device receives the interrogation signals sent by a ground station and replies to these requests by transmitting the required data. A Secondary Surveillance Radar (SSR) also first appeared during WW II as the successor of the PSR under the name Identification Friend Foe (IFF) permitting the identification of friend targets on the radar screen. This technology overcomes the flaws of the PSR since it does not consume that much energy while generating a better signal. The reason for that is that the aircraft actively sends a signal to the ground instead of just reflecting the radar signal. SSR is operated in different modes. In Mode A the aircraft only transmits a four- digit identification number (“squawk”), while in Mode C the altitude is broadcast in addition to the squawk. The most advanced Mode S allows for the transmission of even more data from the aircraft to the ground station. Such transponder information is the basis for the TCAS, which is introduced in Section 2.2. Examples of other independent cooperative surveillance technologies are Wide Area Multilateration (WAM) and Multi-LATeration (MLAT).

Dependent Cooperative Surveillance This kind of surveillance systems depend on a system that enables the on board determination of the vehicles position, for instance a GPS receiver. This information is then broadcast together with supplemental data. An example of such an approach is the ADS-B technology, which is further explained in the following.

Table 2.1.: Overview of aeronautical surveillance systems [16].

Category Technology

Independent Non-Cooperative Primary Surveillance Radar (PSR)

Multi-Static Primary Surveillance Radar (MSPSR) Cooperative Secondary Surveillance Radar (SSR) (Mode A, C, S)

Wide Area Multilateration (WAM) Multi-LATeration (MLAT)

Dependent Cooperative Automatic-Dependent Surveillance Broadcast (ADS-B)

2.1.2. Functional Principle of the ADS-B Technology

Automatic Dependent Surveillance-Broadcast (ADS-B) is a traffic surveillance technology used in

aviation. The system depends on the flight state information which is determined by the Global

Navigation Satellite System (GNSS) equipment of the aircraft or ground vehicle. This position and

velocity data is then, together with other data such as Flight Identification (FLTID) and accuracy indications, automatically and periodically broadcast by an onboard transmitter. The broadcast messages can be received by other aircraft, ground vehicles and ground stations which are con- nected to Air Traffic Control (ATC) facilities. Figure 2.1 sketches the functional principle of the ADS-B technology.

Figure 2.1.: Schematic overview of the ADS-B functional principle (adapted from [41]).

It can be distinguished between two parts: ADS-B OUT and ADS-B IN. The former implies that a vehicle is equipped with a transmitter sending flight data with a rate of at least one message per second. By contrast, ADS-B IN makes an aircraft capable of receiving messages from surrounding traffic. This information can be used to increase the pilots’ Situational Awareness (SA) by showing the received data on a CDTI.

ADS-B is a crucial part of the NextGen and SESAR programs which were launched in order to re-

organize the US and the European airspaces as well as their ATMs. This modernization is required

to be able to cope with the demands of future air traffic. According to market forecasts there

will be too many flights to be safely controlled by today’s ATM infrastructure. Thus, the Federal

Aviation Administration (FAA) [21] requires all aircraft operating in certain controlled airspace to

have ADS-B OUT installed by 2020. According to the FAA’s NextGen implementation plan [21], the

baseline installation of the ground infrastructure is completed by now and more than half of the

ATC facilities in the US were equipped with ADS-B surveillance systems until August 2014.

2.1.3. Benefits of ADS-B

The ADS-B technology offers a great many advantages over its predecessor, the radar. Owing to the usage of satellite systems, aircraft positions can be determined with improved accuracy and greater reliability. ADS-B updates more than four times faster than the best radar, which provides new data every 4.7 seconds [19]. Furthermore, the ADS-B technology overcomes the range limitations of conventional radars since the transmitter and the receiver can be separated by more than 100 nautical miles [41]. This allows air traffic controllers to monitor areas where no radar coverage exists, for instance in the Gulf of Mexico [21].

The improved navigation accuracy permits the reduction of the separation minima in controlled airspace, which implies that aircraft can operate closer to each other and choose more direct routes at the most efficient altitudes. As a consequence, the airspace capacity is increased and flights can be conducted in a more efficient way. This saves flight time as well as emissions [41].

Another plus of the ADS-B technology is the smaller size and lower cost of the required equipment.

This has two benefits. First, the construction and the maintenance of the ground surveillance stations is much easier and cheaper compared to the large and costly radar antennas [19]. Second, each aircraft can be equipped with ADS-B transmitters and receivers, which enables the direct communication between vehicles. Together with the fact that ADS-B provides a greater amount of data, this is the basis for increasing the pilot’s situational awareness by providing substantial information about surrounding traffic on his cockpit displays.

In contrast to radar systems, ADS-B is able to detect aircraft on ground as well as equipped surface vehicles. This allows for a better monitoring of the surface operations in terminal areas and reduces the risk of runway incursions by providing increased situational awareness during final approach and taxiing [19], [41].

In summary, the ADS-B technology is one of the cornerstones of the future air traffic management as it offers more precise and reliable surveillance data leading to improved safety and efficiency while the capacity of the airspace is increased significantly.

2.1.4. Implementation Details

An overview of the ADS-B system and its three submodules is given by Figure 2.2. The transmitting subsystem gathers traffic data from various sources and periodically generates ADS-B messages.

By means of this propagation medium, the information is transferred to the receiving subsystem, which processes the messages and makes the traffic data available for applications like a CDTI or a collision avoidance system.

For the data transmission the 1090 MHz Extended Squitter (ES) link is applied in both the Euro-

pean and the US airspace. This is the same frequency on which Mode S transponders reply to

interrogations. Additionally, a Universal Access Transceiver (UAT) operating on 978 MHz can be

used below FL 180 in the US [21].

Figure 2.2.: Overview of the ADS-B system components (adapted from [42]).

The Minimum Aviation System Performance Standards (MASPS) for ADS-B are defined by the Radio Technical Commission for Aeronautics (RTCA) in DO-242A [42]. Besides many other operational characteristics, this documents specifies the parameters that are transmitted via ADS-B. Table 2.2 provides an overview of these properties. The specific technical implementation of the standards described in DO-242A is defined by DO-260B, the Minimum Operational Performance Standards (MOPS) for 1090 MHz Extended Squitter ADS-B [43].

Table 2.2.: Parameters broadcast by ADS-B equipped vehicles.

Time of Applicability

ID Call Sign

Participant Address, Address Qualifier Category

Vehicle Dimensions Length and Width Codes Position Reference Point

Position Horizontal Position (Lat, Lon)

Geometric Altitude Barometric Altitude

Velocity Geometric Horizontal Velocity (North, East)

Vertical Rate Airspeed

State Vector Quality Navigation Integrity Category Position Accuracy Category Velocity Accuracy Category Source Integrity Level Heading

Air/Ground State Intent Information

Capability Class and Operational Mode

Emergency/Priority Status

All ADS-B parameters are assigned to so-called ADS-B reports. The State Vector Report comprises mainly position and velocity parameters whereas the Mode Status Report delivers the state vector quality indications, the capability class and operational mode et cetera. Besides these two basic reports various on-condition reports such as the Air Referenced Velocity Report exist. Depending on its equipage class, which is based on user operational interests, each airborne or ground vehicle must be capable of transmitting and receiving certain types of reports within specified ranges. For example, certain types of terminal surface vehicles do not have to be able to receive any ADS-B messages but must send the data contained in the State Vector Report and the Mode Status Report.

For a more detailed explanation of the technical implementation, RTCA DO-260B [43] should be consulted.

2.2. The Traffic Alert and Collision Avoidance System (TCAS)

A system that is closely related to the aeronautical surveillance systems discussed above is the Traffic Alert and Collision Avoidance System (TCAS). Its latest version, TCAS II, Version 7.1, is briefly introduced in this section as it is the basis for the threat potential visualization developed in Section 5.2.1. More thorough descriptions of the system can be found in [17], [20], [24] and [35], which are the main sources of this short summary.

2.2.1. Introduction to TCAS II

TCAS II is an Airborne Collision Avoidance System (ACAS) that works independently of airborne navigation equipment and ground-based systems, only based on the direct communication of aircraft via their transponders. In Europe, TCAS II, Version 7.0 is mandated by “all civil fixed- wing turbine-engined aircraft having a maximum take-off mass exceeding 5700 kg or a maximum approved passenger seating configuration of more than 19” [17] since 2005. Attention should be paid to the fact that TCAS is not capable of alerting to aircraft that are not equipped with an active mode S or Mode A/C transponder.

Figure 2.3 shows a schematic overview of a TCAS II installation. As can be seen, a set of antennas is installed on top and at the bottom of the fuselage to transmit and receive messages to and from the transponders and the TCAS of nearby aircraft. TCAS sends interrogations to all close transponder- equipped aircraft once per second so as to determine their range and altitude. Note that mode S transponders can be directly addressed while mode A/C transponders do not provide selective addressing. This can cause overlapping replies, called synchronous garble, when two intruders operate within a similar range from the ownship.

The central TCAS unit processes all responses and applies a special logic to the classification of

each intruder into three categories: Proximate Traffic, Traffic Advisory (TA) and Resolution Advisory

(RA). The remaining aircraft are declared Other Traffic. According to this classification, an intruder

is highlighted on the CDTI, triggers an aural annunciation (TA) or even initiates the display of

instructions on how to solve the conflict (RA). It should be noted that RAs were introduced with TCAS II; TCAS I provided TAs only.

Figure 2.3.: Schematic diagram of a TCAS II installation [17].

2.2.2. The TCAS II Logic

The logic behind TCAS can be divided into a threat detection and a threat resolution part.

Threat Detection

The TCAS II threat detection is based on the concept of the warning time τ, which is an approxima- tion of the time to the Closest Point of Approach (CPA) between ownship and intruder. The lower this measure, the higher is the threat potential of the target aircraft. TCAS II performs the threat detection in horizontal and vertical direction separately and therefore defines a range tau

τ

= sl ant r ang e

cl osi ng r at e (2.1)

for the horizontal range test and a vertical tau

τ

= ver t i c al separ at i on

ver t i c al cl osi ng r at e (2.2)

for the vertical altitude test.

A TA or an RA is issued when both the range and the altitude test are passed, which means that

τ

and τ

are below a certain limit τ

or the intruder is already closer to the ownship

than the vertical and horizontal protection thresholds (ZTHR, DMOD). The definition of the ap- plied thresholds to the threat detection is a trade-off between unnecessary alerts and adequate protection. In order to meet this challenge, the Sensitivity Level (SL) concept is introduced. From Table 2.3, it is apparent that each SL is bound to a certain ownship altitude range. The idea behind that is to dynamically adapt the chosen thresholds according to the operation altitude. The values increase with the SL, which means that advisories are triggered earlier at higher altitudes where aircraft generally fly faster and better separated. At SL 5, for instance, an RA would be issued if both τ

and τ

were lower than 25 seconds or the interrogated aircraft violated the DMOD and ZTHR limits. A TA would be caused even earlier when the two τ values go below 40 seconds.

Table 2.3.: Definition of the TCAS II Sensitivity Level (SL) and the corresponding thresholds

Ownship Altitude [ft] SL τ

[s] DMOD [NM] ZTHR [ft]

TA RA TA RA TA RA

< 1000 2 20 - 0.30 - 850 -

1000 – 2350 3 25 15 0.33 0.20 850 600

2350 – 5000 4 30 20 0.48 0.35 850 600

5000 – 10000 5 40 25 0.75 0.55 850 600 10000 – 20000 6 45 30 1.00 0.80 850 600 20000 – 42000 7 48 35 1.30 1.10 850 700

> 42000 7 48 35 1.30 1.10 1200 800

However, the simple definition of the range tau given by Equation 2.1 poses two problems. First, intruders approaching the ownship with a small closing speed may not be detected. Second, an approach of both aircraft with a high closing rate but also with a sufficient horizontal separation can be declared as dangerous without reason [20], [35].

The first problem is solved by adapting the formula for the range τ to τ

= − r

− DMOD

r ˙ r (2.3)

where r is the range, ˙ r is the range rate, i.e. the negative closure rate, and Distance MODifier (DMOD) is a horizontal distance threshold. According to [35], this definition provides more con- servative values of τ for smaller ranges and rates.

The second problem is avoided by the introduction of a horizontal miss distance filter. In simple terms, this filter checks if the predicted horizontal separation at the estimated CPA is less than the DMOD threshold. An RA is issued only if this test is passed; to TAs this check is not applied.

In summary, the threat detection concept of TCAS II can be imagined as a protected volume around

the TCAS-equipped aircraft. The dimensions of this volume are defined by the chosen SL and the

connected thresholds. Figure 2.4 illustrates the space within which an interrogated aircraft causes

a TA as a yellow area. Inside this caution area, the magenta volume visualizes the space protected

by the issuance of RAs.

(a) Top view (b) Side view Figure 2.4.: The TCAS II protected volume [17].

Threat Resolution

In addition to the traffic alerts, TCAS II provides instructions for solving encounters where the target aircraft caused an RA. To do so, a highly sophisticated logic, further explained in [17], analyzes the situation and determines proper actions for the pilots to avoid a collision. These commands, which are only given in vertical direction, are coordinated if both involved aircraft are equipped with TCAS II.

In the exemplary situation sketched in Figure 2.5, the left aircraft is instructed to descend while the right airplane is mandated to climb in order to solve the conflict. However, as the pilot of the latter does not comply with the advisory, the TCAS II threat resolution system reverts its initial command for the descending left aircraft and tells the pilot to climb now. This feature was introduced by the latest Version 7.1 so as to avoid incidents as the mid-air collision over Überlingen at Lake Constance in 2002, where two TCAS II-equipped airliners crashed due to one pilot following ATC instructions that were opposed to the coordinated RA which could not be reversed by the earlier version of TCAS [7].

Figure 2.5.: Illustration of the TCAS II threat resolution logic including the improved reversal logic intro-

2.2.3. TCAS Displays

TCAS information can be included in the existing Navigation Display (ND) and Primary Flight Display (PFD) or can be presented on a separate display as shown in Figure 2.6a. Independent of the chosen solution, each interrogated aircraft will be indicated by one of the standardized symbols depicted in Figure 2.6b and resolution advisories will always be visualized in form of desired (green) and forbidden (red) ranges on the Vertical Speed Indicator (VSI).

The display example in Figure 2.6a shows unclassified traffic at the same flight level, proximate traffic 1000 ft below, a descending TA 200 ft above as well as a climbing aircraft causing an RA 300 ft below. To avoid a collision with the latter, TCAS instructs the pilot to descend with a vertical rate between 1500 and 2000 ft/min, which is visualized by the green area on the VSI surrounding the top-down traffic view.

(a) Vertical Speed Indicator integrated with TCAS II

(b) The TCAS II symbology.

Figure 2.6.: The TCAS II traffic visualization [17].

2.3. Cluster Analysis

Cluster analysis is one of the central methods used in the field of data mining, which is a key step

within the discovery of knowledge in databases (KDD) [18]. It has been utilized in various ways by

a great many fields of science including image segmentation, artificial intelligence research, big

data analysis on data derived from different fields such as geology or biology. This vast number

of applications with different prerequisites and requirements yields numerous algorithms for the

identification of clusters in large data sets. Since a comprehensive review of the various approaches

taken by different research communities is far beyond the scope of this work, the following chapter provides a brief introduction to key ideas of cluster analysis and gives an overview of important algorithms. Thereafter, the DBSCAN algorithm, which is applied to the classification of traffic information later in this thesis, is detailed. For a more thorough description of existing clustering methods one can consider [3], [26], [27] and [55], which form the basis for this short review.

2.3.1. Introduction to Cluster Analysis

Clustering is the process of grouping a set of data points called patterns into distinct clusters based on their proximity. Consequently, the similarity between the members of the resulting subclasses is higher than between patterns not belonging to the same cluster. Figure 2.7 illustrates the main stages in clustering according to the definition of Jain et al. [27]. Xu and Wunsch [55] append cluster validation and result interpretation steps to this procedure.

Feature Selection/

Extraction

Proximity Measure Definition

Grouping

feedback loop

patterns clusters

Figure 2.7.: The main stages in clustering (Adapted from [27]).

To start with, each pattern is described by a vector in the multidimensional feature space. An exemplary feature space for the analysis of personal data can be (name, age, weight, income, job) yielding patterns like (John, 45, 90 kg, 100.000 €, consultant). As can be seen from this example, the features can have various data types. Jain et al. [27] distinguish between quantitative and qualitative parameters, where the former includes continuous, discrete and interval values and the latter comprises nominal and ordinal types.

Before the actual clustering is performed, the raw data often needs to be prepared by applying feature selection or extraction methods. This is a crucial stage because the quality of the clus- tering result is highly dependent on this preprocessing step. Both methods aim at reducing the dimensionality of the feature space in order to sharpen the picture for the subsequent grouping task. Feature selection is the process of determining the most significant subset of features to be forwarded to the clustering algorithm. In the above example one could, for instance, decide to neglect the persons name since it is irrelevant as well as one might want to select only one of the features “income” and “job” assuming that they are connected and redundant. In contrast, feature extraction is the transformation of the original feature space to a lower-dimensional representation comprising new variables. Van der Maaten et al. [51] provide a comparative review of non-linear dimensionality reduction methods which are compared to the most popular linear technique, the Principal Component Analysis (PCA).

The second step after the feature preprocessing is the definition of the proximity measure, a crite-

rion to assess how similar or dissimilar two patterns are. According to Jain and Dubes [26], this is

“the crucial problem in identifying clusters in data”. Many ways for this task have been reported for all different kinds of cluster analysis. For continuous data types, the most common metric used is the Euclidean distance. Alternatives are the Manhattan or city-block distance, the Mahalanobis distance, the general Minkowski metric [55] and even more sophisticated and problem specific metrics. A particular challenge is the selection of a proximity measure for feature spaces of mixed data types. Solutions to this problem provide Gower’s index, the Jaccard or the simple matching coefficient and many more [26]. If the chosen proximity measure is sensible to the scale of its input values, all parameters should be standardized or normalized for them to contribute with the same weight to the computed distance. This is, for example, the case when using the Euclidean distance with features expressed in different units so that their values differ in magnitude, for instance, income and age of a person.

2.3.2. Brief Overview of Important Clustering Methods

The development of new algorithms is driven by the goal of designing methods that are more ef- ficient, both in terms of time and space requirements. At the same time, they aim at producing better clustering results even for challenging data constellations, that is various cluster shapes and densities, minimal domain knowledge or feature spaces with mixed data types. Moreover, it is tried to increase the efficiency for large databases with high-dimensional feature vectors. The great number of clustering methods can be organized in many ways. The classic approach is to distinguish between two categories: hierarchical and partitional clustering algorithms. Other pos- sible distinctions include the classification into hard and fuzzy/soft clustering, where the former requires that every pattern is part of one single cluster while the latter methods imply that every data point can be a partly member of several clusters at once.

Hierarchical methods produce a hierarchy of the patterns, which is often visualized by a dendro- gram as sketched in Figure 2.8a. Jain et al. [27] state that most hierarchical methods are modifica- tions of two basic approaches. The first, the single-link method, measures the distance between two clusters based on the minimal pairwise distance of the cluster members whereas the second, the complete-link approach, utilizes the maximal distance. Both approaches have their respec- tive benefits depending on the data constellation. Subject to the approach taken for the cluster generation, it is distinguished between agglomerative and divisive hierarchical methods. As the names imply, the latter creates the tree structure from top to bottom while the former takes the inversive direction. A crucial step in these methods is the identification of a termination condition that defines the level of proximity at which the fusion or splitting of the clusters is stopped. The determination of this threshold, which is represented by the horizontal, dashed line in Figure 2.8a, is one of the biggest challenges with this type of methods.

The probably best-known partitional method is the square-error based k-means algorithm [33], which iteratively optimizes an initial, randomly chosen partition of the patterns into k clusters.

Thereby, each pattern is reassigned to the most-similar cluster center, which is then recomputed in

each iteration until convergence is reached. As depicted in Figure 2.8b, the resulting groups are rep-

resented by Voronoi cells around the cluster centers. The major drawbacks of this algorithm are its

restriction to spherical cluster shapes as well as the fact that the number of clusters must be known a priori. Moreover, the result depends on the initially chosen partitioning. In the literature, a great many variants of this simple and easy-to-implement algorithm have been proposed. Examples are k-medoids [29] and x-means [39]. Moreover, Aggarwal et al. [3] explain that the k-means approach is a special case of generative models like the Expectation Maximization (EM) algorithm [10].

x

x

A B C

D E

F G

dist anc e

simil a ri ty

A B C D E F G

(a) A dendrogram resulting from hierarchical single-

(b) k-Means

Figure 2.8.: Examples of important clustering methods.

Another well-known approach is the density-based cluster analysis where a cluster is understood as an area with significantly high data point density. An example of this group of methods is the DBSCAN algorithm [15], which was further developed and adapted to the GDBSCAN [45] and the OPTICS algorithm [5] permitting the discovery of clusters with varying density.

During the last decades, many existing clustering methods have been expanded or adapted to the requirements of new applications. For instance, the basic k-means and density-based approaches have been adopted and enhanced by the BIRCH [56] and the DenStream [8] method respectively so as to provide efficient solutions for the clustering of stream data.

2.3.3. Density Based Spatial Clustering of Applications with Noise - The DBSCAN Algorithm

The algorithm Density Based Spatial Clustering of Applications with Noise (DBSCAN) is a clustering

method proposed by Ester et al. [15] which relies on a density-based notion of clusters. It is able to

efficiently discover clusters of arbitrary shape in large, spatial databases. A plus of this method is

that it requires only a single input parameter and no pre-knowledge about the number of clusters.

Furthermore, the algorithm detects noise, that is data points which do not belong to any of the discovered classes. This section provides a short introduction to the central ideas of this algorithm, for a more thorough explanation of the approach one can consider [15], which is the basis for this part of the thesis.

(a) Exemplary set of two-dimensional data

1

2 3

4 5

6

(b) A set of core (red), border (orange) and

²-neighborhoods (

minPts

Figure 2.9.: Details about the density-based notion of clusters adopted by the DBSCAN algorithm.

Figure 2.9a illustrates that the density-based definition of clusters used by the DBSCAN algorithm conforms to the intuitive cluster understanding of most humans. The clusters are mainly recog- nized due to their higher density of points compared to the surrounding areas where the points are declared noise. The DBSCAN algorithm expresses this intuitive understanding of clusters in terms of a few mathematical definitions introduced in the following. For the sake of simplicity, the examples in Figure 2.9 are based on the two-dimensional Euclidean distance, even though the algorithm works for high-dimensional feature space and with any distance function.

For each point p of the database D an ²-neighborhood is defined as N

(p) = ©q ∈ D | dist (p, q) ≤ ²ª.

These areas are visualized by the circles in Figure 2.9b. Obviously, the ²-neighborhood of a core point (red) contains more points q than the one of a border point depicted orange. To account for this characteristic the term density-reachability is introduced.

Point q is directly density-reachable from point p, if it is inside the ²-neighborhood of p and if p is a core point, which means that the number of points inside the neighborhood is greater than the threshold minPts . In Figure 2.9b, for instance, border point 2 is directly density-reachable from core point 4 but not vice versa.

An extension to the direct density-reachability is the density-reachability implying that a chain

of directly density-reachable points connects the two involved points. An example of this relation

is border point 1 being density-reachable from the core points 4 and 5. Nevertheless, it is not

density-reachable from the other border point 2 because point 4 is not directly density-reachable from this point. To account for these cases, the notion of density-connectivity is introduced. This symmetric relation denotes that a core point exists from which both point p and q are density- reachable. As a consequence, the points 1-5 in Figure 2.9b are density-connected to every other point 1-5.

Combining the definitions introduced above, a cluster C is a subset of the database D that satisfies two conditions:

1. ∀ p, q: if p ∈ C and q density-reachable from p, then q ∈ C. (Maximality)

2. ∀ p, q ∈ C: p is density-connected to q. (Connectivity)

In short, all points of a cluster are density connected and the cluster contains all points that are density-reachable from its members. By contrast, all data points not being part of any cluster are considered noise. This means that the points 1-5 in Figure 2.9b are assigned to one cluster whereas point 6 is declared noise. To realize these definitions, Ester et al. [15] proposed the algorithm illustrated in Figure 2.10.

1: function DBSCAN(setOfPoints,², minPts) . setOfPoints is unclassified

2: clusterId ← nextId(noise) 3: for i ← 1, setOfPoints.size do 4: point ← setOfPoints.get(i) 5: if Point.clId = unclassified then

6: if EXPANDCLUSTER(setOfPoints, point, clusterId,², minPts) then 7: clusterId ← nextId(clusterId)

8: end if

9: end if

10: end for 11: end function

(a) The main loop.

1: function EXPANDCLUSTER(sOP, point, clId,², minPts)

2: seeds ← sOP.regionQuery(point, ²) 3: if seeds.size < minPts then

4: sOP.changeClusterId(point, noise) 5: return false

6: else

7: sOP.changeClusterIds(seeds, clId) 8: seeds.delete(point)

9: while seeds 6= empty do 10: currentP ← seeds.first()

11: result ← sOP.regionQuery(currentP, ²) 12: if result.size ≥ minPts then

13: for i ← 1, result.size do 14: resultP ← result.get(i)

15: if resultP.clId ∈©unclassified, noiseª then 16: if resultP.clId = unclassified then

17: seeds.append(resultP)

18: end if

19: sOP.changeClId(resultP,clId)

20: end if

21: end for

22: end if

23: seeds.delete(currentP)

24: end while

25: return true 26: end if 27: end function

(b) The function for expanding a cluster.

Figure 2.10.: Density based spatial clustering of applications with noise - the DBSCAN algorithm [15].

Each point of a cluster is density-reachable from any of the cluster’s core points. Thus, the key idea of the algorithm is to iterate through all unclassified points in the database setOfPoints and to check if the current point fulfills the core point condition, that means the number of points in its

²-neighborhood is greater than or equal to the threshold minPts . If this test succeeds, the cluster can be expanded by iteratively finding the points that are directly density-reachable from the seeds and adding these points to the seeds . This process is continued until all points in the continually growing seeds database are processed, which implies that all points that are density-reachable from the initial core point are found.

2.4. Head-Mounted Displays for Increasing Situational Awareness

Melzer [34] describes HMDs as “powerful tools that can unlock the pilot from the interior of the cockpit or the forward line of sight of the Head-Up Display (HUD).” As a result, the displays “can enable the pilots to do their job more effectively while simultaneously decreasing workload” [34].

One major goal in the development of HMD symbologies is the improvement of the pilot’s Situa- tional Awareness (SA). According to Endsley [14], SA is defined as “the perception of the elements in the environment within a volume of time and space (level 1), the comprehension of their meaning (level 2), and the projection of their status in the near future (level 3).”

A particular challenge is the development of pilot assistance systems that increase the pilot’s SA dur- ing low level flights and landings in Degraded Visual Environments (DVEs). Degraded vision due to fog, heavy precipitation, limited sunlight and stirred-up sand or snow can cause spatial disorienta- tion and lead to fatal accidents. The German Aerospace Center (DLR) as well as Airbus DS conduct research within this field so as to enhance the pilot’s performance in such situations [11], [36]. Both follow a three-dimensional, visual-conformal visualization approach since this scene-linked pre- sentation of information is a proofed concept that facilitates the mental processing and fusion of both impressions, the outside scene and the symbology [47]. It implies that, for instance, obstacle symbols on the HMD are virtually superimposed on the underlying real world scene so that they appear at the location where the pilot would see the real obstacle if the view was not degraded.

As illustrated in Figure 2.11, the SFERION product family by Airbus DS comprises four systems:

SferiAdvise, SferiAssist, SferiSense and SferiRec. Depending on the operational requirements and

other constraints, each module can be operated standalone or as an integrated SFERION sub-

system on various helicopter types [36]. While SferiAdvise provides several navigation aids and

supports the pilot in mission management tasks, SferiRec supplies equipment for data recording,

transfer and analysis [9]. Here, the focus is placed on SferiSense, which includes a set of sensors

for the recognition of the surrounding areas, as well as on the pilot assistance applications of

SferiAssist, which display a real-time synthetic view of the environment on a Head-Down Display

(HDD) enhanced with an HMD symbology.

Figure 2.11.: The SFERION product family comprising the SferiAdvise, SferiAssist, SferiSense and SferiRec

Within SFERION, a Light Detection And Ranging (LiDAR) sensor mounted on the helicopter per- forms range measurements based on the reflection of light emitted by a laser. Airbus DS developed methods for analyzing this data so as to be able to visualize terrain and obstacles on the displays.

A simplified model of the functional principle is given by Figure 2.12a.

(a) Functional priciple of edge detection with a Li-

(b) Visualization of pole (top), tree (middle) and

Figure 2.12.: Obstacle avoidance with the help of a LiDAR sensor.

The recorded LiDAR data is processed in several ways. First, the shape of the ground is identified

by means of a segmentation approach developed by Eisenkeil et al. [12]. Second, the remaining

data is classified as three different kinds of obstacles, namely wires, poles and isolated trees. The

certified detection range of a wire with 5 mm diameter is 725 m. In very challenging conditions

like heavy rain this capability is reduced to 500 m but the SferiSense system is in general even able

to “look” through dense brown-out dust [36]. Finally, data points which could not be classified as

terrain, wire, pole or tree are further processed. To do so, Eisenkeil et al. [13] propose a dynamic DBSCAN-based clustering method. The identified unclassified obstacles are visualized with sym- bols based on convex hulls that are stabilized and smoothly fused so as to create a non-distracting representation.

It should be noted that all information retrieved from the LiDAR data is fused with information stored in an on-board database. This means that the basis for the visualization of terrain and obstacles is a fusion of currently sensed and previously stored data [36]. Figure 2.12b shows the graphical representation of the three obstacle classes on the HMD. Moreover, the identified terrain is illustrated by a grid and contour lines as shown in Figure 2.13a. A special feature of the SferiSense module is the Safety Line, which can be seen in Figure 2.13b. It presents an intuitive way for the pilot to avoid collisions with obstacles and terrain since he only needs to keep the flight vector above this dynamic line in order to ensure a safe flight [4]. Besides these obstacle and terrain representations, SFERION assists the pilot with a three-dimensional, visual-conformal landing symbology and a basic two-dimensional, head-up PFD as shown in Figure 1.1. The exact layout of the displays can be adapted by the pilot according to his needs in the respective situation.

(a) Terrain grid with contour lines as well as pole

(b) The Safety Line (orange) and the flight path

Figure 2.13.: Symbology features of the SFERION pilot assistance system by Airbus DS.

As the visual-conformal representation has proved it’s worth in many applications, the question can be posed if this representation is also a suitable means for the visualization of traffic information in HMDs. Lenhart [31] presented a thorough analysis of various concepts for the visualization of air traffic in HMDs. In contrast to this thesis, Lenhart’s symbologies are restricted to the cueing of aircraft causing a TA and an RA only. Furthermore, he does not provide de-cluttering mechanisms which may be caused by the rarity of TAs/RAs in the controlled airspace and by the fact that he does not consider aircraft flying in formation. By contrast, this work does explicitly focus on scenarios where many ADS-B-equipped vehicles operate close to each other in uncontrolled airspace.

Another head-up traffic visualization is proposed by Wong et al. [54] for NASA’s eXternal Visibility

System (XVS). It displays traffic that the TCAS classified as Proximate Traffic, TA or RA with a hollow

design of the standard TCAS symbology. Moreover, the effectiveness of a range filter is verified.

Visualization Concept in Head-Mounted and Head-Down Displays

Within this chapter a concept for an integrated traffic visualization using a head-up as well as a head-down display is developed. The whole traffic symbology presented in this thesis will be based on this concept definition. The development of this implementation plan is started with a review of existing head-down CDTIs, which are the common means for displaying traffic today. Based on the results of this assessment and an analysis of a basic proposal for a head-up traffic symbology, the concept definition for the integrated traffic visualization is formulated.

3.1. Existing Head-Down Cockpit Displays of Traffic Information

Today, many solutions for the head-down visualization of traffic information exist. Various avionics companies supply a wide range of systems, from products for small General Aviation aircraft to commercial airliner equipment. Figure 3.1 shows two examples, the Avidyne EX600 Multi-Function Display (MFD) and the Air Traffic Situational Awareness (ATSAW) system by Airbus and Honeywell.

As can be seen, the layout and the features of current CDTIs vary from system to system. Often, the pilots can choose between different backgrounds including for example terrain or weather visualizations. At the same time, they may activate the display of various tactical and navigational information, such as waypoint symbols or the flight path planned by the Flight Management Sys- tem (FMS). Furthermore, as described in Section 2.1 and 2.2, these displays can be fed with data from various airborne surveillance systems and apply different versions of traffic classification systems (TCAS I/II).

Despite all these different features, most head-down CDTIs share the same basic format. As can

be seen in Figure 3.1, conventional head-down CDTIs show the situation around the ownship

as a top-down view from infinitive distance (orthogonal projection) implying that all horizontal

distances can be seen directly. Consequently, this representation allows for a good illustration of the

horizontal spacing between nearby aircraft and additionally provides the pilot with information

about the intruders’ direction of flight. However, without additional altitude information the pilot

would not be able to monitor the vertical separation. Thus, CDTIs usually display the altitude

relative to the ownship by means of two digits next to the aircraft symbol. Moreover, the altitude

trend is provided through a nearby arrow symbol. Finally, special intruder symbols allow for quick

identification of potential threats, if TCAS I or II is integrated with the display.

(a) The Avidyne EX600 MFD displaying traffic sym-

(b) The Airbus and Honeywell ATSAW navigation

Figure 3.1.: Examples of different Cockpit Displays of Traffic Information (CDTIs).

A drawback of these displays is that the pilot has to mentally construct the three-dimensional, real world scene from the traffic information provided as a two-dimensional top-down view. Fur- thermore, it requires time and work to detect the position of a certain aircraft of interest in the out-the-window view.

It must be pointed out that synthetic vision displays, which became available in recent years, are an exception to the analyzed head-down CDTIs. They introduce an interesting approach as they present terrain, traffic, obstacles and flight paths in a three-dimensional perspective view [22], [38].

However, these systems are often only an addition to the common two-dimensional navigation displays.

3.2. Basic Traffic Visualization in Head-Mounted Displays

Now, having the advantages and downsides of existing head-down CDTIs in mind, a basic, straight- forward approach to the traffic visualization in HMDs is proposed and assessed. Target aircraft are cued by displaying a symbol superimposed on the real aircraft in the outside view. Figure 3.2 sketches such a visual-conformal symbology which highlights other aircraft with a framing circle.

This perspective visualization is intuitive to use because it conforms to the way how humans see

the world through their eyes. At the same time, it overcomes the drawbacks of the head-down

presentation described in Section 3.1 since it decreases the pilot’s head-down time and facilitates

the searching process during good visibility. In DVEs, it even allows him to spot aircraft that would

be invisible otherwise. In summary, the traffic symbology assists the pilot with the task of seeing

and avoiding other aircraft.

Figure 3.2.: A basic traffic visualization on the HMD - visible and invisible traffic highlighted with circles.

Nevertheless, these advantages also implicate a great many difficulties for the development of a suitable head-up traffic visualization. Again, as seen for the head-down traffic displays in Sec- tion 3.1, one of the key challenges to be met is the illustration of a three-dimensional traffic scenario on a two-dimensional display. The problems caused by the perspective projection of the real world onto the flat HMD screen include difficulties with the perception of the altitude and lateral de- viation, the distance, the direction of flight and the speed of the target aircraft. Another major problem, namely display clutter, appears when many aircraft operate in the vicinity of the ownship.

Details of these issues are discussed in the following paragraphs.

Overlapping Symbols If more than one aircraft appears at the same relative bearing but at a different distance from the ownship, the framing circles, representing each single intruder on the HMD, will overlap and clutter the screen. Figure 3.3a depicts an exemplary traffic situation on the head-down CDTI where the proposed basic head-up presentation of traffic information results in overlapping symbols (Figure 3.3b).

(a) HDD - top view. (b) HMD - overlapping

Figure 3.3.: Illustration of a possible traffic situation causing overlapping symbols on the HMD.

Interpretation of Altitude and Lateral Deviation In contrast to the orthogonal projection em- ployed by the presented head-down CDTIs, the perspective view of the traffic on the proposed HMD, which superimposes the traffic symbols on the out-the-window scene, has the characteris- tic that all dimensions depend on the distance between observer and object. In easy words, the human eye perspective makes close objects appear larger than distant ones.

Subsequently, this makes it impossible for the pilot to directly see the altitude difference as well as the lateral deviation between the own and the target aircraft. This issue, which can lead to severe misinterpretations if one is not aware of it, is illustrated in Figure 3.4. As can be seen from the relative altitudes ∆h

in the side view (Figure 3.4b), aircraft 2 (red) operates higher than aircraft 1 (blue). However, due to the perspective projection, which is illustrated by the colored dashed lines in the side view, AC 1 seems to be lower than AC 2 on the HMD (Figure 3.4a).

Even though Figure 3.4 explicitly deals with a possible altitude misinterpretation, it can be imag- ined that the same problem applies to the lateral deviation of other aircraft. Interpreting Figure 3.4b as top-down view with ∆h

being the lateral deviation of the two intruders makes this issue obvi- ous.

(a) Projection on the

D isplay

A C 1

∆h

A C 2

∆h

(b) Side view (not drawn to scale).

Figure 3.4.:

Perception of Distance Another disadvantage of a simple head-up traffic visualization is the fact that the distance to the target aircraft can not be estimated from this perspective view. As a result, additional information is needed for the pilot to assess the current traffic situation in the vicinity of his aircraft.

Speed Interpretation As a consequence of the missing distance perception, it is also not possible

to identify how fast other aircraft approach or depart from the ownship. In addition, two aircraft

flying in the same direction with the same speed seem to fly with different speeds if their distance

to the observer is unequal. Again, the reason for this is the fact that the same length appears shorter

if it is located farther away.