Flexible User Interface - FLUSI
Jan Conrad
Jan MSI Report 06010
FLUSI
Jan Conrad
The cellular phone network has been increasing rapidly during the last years. For many people the mobile phone has become an every day gadget with a wide performance and functional range. The usage of technologies like GPRS, HSCSD, EDGE and UMTS as well as the bandwidth of networks and consequently the connectivity of the phones has also increased persistently.
Coming along with that, three technologies, which are ubiquitous or pervasive computing, mobile and wireless networks and location-based technologies, are making rapid progress.
The aim of this thesis is to offer an architecture for a location-based user interface in the intersection of the three technologies mentioned above. The system should work with a minimum of special hardware requirement. Not to overload the user with information, the user interface should be adaptable, context-aware and location-based. The context-data should remain extendible and adaptable.
First, I am grateful to Jesper Andersson. He has been a continual support. He was always available for me and inspired me with many relevant and useful ideas and critics.
Secondly I thank Marcus Edvinsson who introduced me to the XAP-System.
Last but not the least I thank Mathias Hedenborg and Maria Karlsson who made this thesis possible.
Jan Conrad
3G Third Generation
API Application Programming Interface CPU Central Processing Unit
DLL Dynamic Link Library
DOM Document Object Model
DOP Dilution Of Precision DTD Document Type Definition
EDGE Enhanced Data for GSM Evolution FLUSI Flexible User Interface GNSS Global Navigation Satellite System GPRS General Packet Radio Service GPS Global Positioning System
GSM Global System Mobile GUI Graphical User Interface
HSCSD High Speed Circuit Switched Data
HTML HyperText Markup Language
HTTP Hypertext Transfer Protocol I/O Devices Input / Output Devices
IT Information Technologies
Navstar Navigation System with Timing And Ranging NMEA National Marine Electronics Association
OO Object Oriented
OS Operating System
PC Personal Computer
POI Points Of Interest
RAM Random Accessed Memory
ROM Read Only Memory
SAX Simple API for XML
U.S. United States of America
UI User Interface
UIQ User Interface Quartz
UMTS Universal Mobile Communication System URI Uniform Resource Locator
WGS 84 World Geodetic System of 1984 WLAN Wireless Local Area Network XML eXtented Markup Language
1.1 MOTIVATION...1
1.2 RESEARCH QUESTIONS...1
1.3 METHOD...2
1.4 RESTRICTIONS...2
2 LOCATION-BASED SERVICES...3
2.1 MOBILE COMPUTING...4
2.1.1 An introduction into Symbian O.S. ...4
2.1.2 Smartphones ...9
2.1.3 Series 60 Platform ...10
2.1.4 Phone-Hardware ...11
2.1.5 User interface design on mobile phones...12
2.2 PERVASIVE /UBIQUITOUS COMPUTING...13
2.3 LOCATION-BASED SERVICES...15
2.3.1 Entities...16
2.3.2 Locations ...16
2.3.3 Location sensing techniques...16
2.3.4 Context awareness ...18
2.3.5 Filters ...18
2.3.6 Privacy and Security ...18
3 FLEXIBLE USER INTERFACE PROTOTYPE “FLUSI” ...19
3.1 SCENARIO...19
3.2 ARCHITECTURE...21
3.3 USED TECHNIQUES AND FUNCTIONALITY...23
3.3.1 The universe...23
3.3.2 Locations and filters...23
3.3.3 Distinction between universe file and location file...25
3.3.4 XML-Parser ...26
3.3.5 GPS ...26
3.3.6 GPS-Interface over Bluetooth ...29
3.3.7 Provisioning...32
3.3.8 Cell id ...34
4.3 ADAPTABILITY...38
4.4 FLEXIBEL USER INTERFACE PROTOTYPE „FLUSI“ ...39
5 CONCLUSIONS AND FUTURE WORK ...40
5.1 ENHANCED PROVISIONING...40
5.2 DATA INTEGRATION...40
5.3 VARIABLE RADIUS OF RELEVANCE...41
5.4 LOCATION-BASED SERVICES...42
5.5 PAYMENT...42
5.6 PROTECTION OF MINORS...42
5.7 POSITIONING...43
5.8 THE FUTURE OF USER EQUIPMENT...43
FIGURE 1:CLASSIFICATION OF THE TOPIC...3
FIGURE 2:SYSTEM COMPONENTS...5
FIGURE 3:DESCRIPTOR OVERVIEW...7
FIGURE 4:OVERVIEW DESCRIPTOR CLASSES...7
FIGURE 5:HEAP FAILURE...8
FIGURE 6:FUNCTIONALITY OF THE CLEANUP STACK...8
FIGURE 7:MARKET SHARES OF SMARTPHONE’S O.S. ...10
FIGURE 8:THE CURRENTLY TWO MAIN VERSIONS OF SERIES 60PLATFORM...11
FIGURE 9:THE PATH OF REASONING FROM PORTABILITY TO USER UNDERSTANDING [11]...13
FIGURE 10:WAVES OF COMPUTING [12] ...14
FIGURE 11:OFFICE RELEVANT ITEMS, FILTERED BY CELL ID...19
FIGURE 12:SELECTING POI MANUALLY...20
FIGURE 13:SELECTION OF LOCATIONS VIA GPS...21
FIGURE 14:ARCHITECTURE OF FLUSI...22
FIGURE 15:UNIVERSE FILE...23
FIGURE 16:LOCATION FILE...24
FIGURE 17:FILTER SELECTION...25
FIGURE 18:SPACE SEGMENT...26
FIGURE 19:GPS-SATELLITE...27
FIGURE 20:CONTROL SEGMENT [17] ...28
FIGURE 21:MEASUREMENT OF SIGNAL RUNTIME...28
FIGURE 22SYSON BLUETOOTH GPS-RECEIVER...29
FIGURE 23:CONNECT TO GPS-RECEIVER...30
FIGURE 24:GPSPOSITIONING OVER BLUETOOTH...30
FIGURE 25:PARSED GPS-DATA...32
FIGURE 26:SEQUENCE DIAGRAM OF A SERVER REQUEST...33
FIGURE 27:POSITIONING OVER CELL-ID...34
FIGURE 28:EXPANSION OF FLUSI PROTOTYPE...41
FIGURE 29:GALILEO PROGRAMME SCHEDULE [32]...43
TABLE 2: TECHNICAL DATA... 27 TABLE 3:NMEA-DATA... 31 TABLE 4:NOKIA'S CUSTOMER PRODUCT AND USER INTERFACE SEGMENTATION PART I[11]... 36 TABLE 5:NOKIA'S CUSTOMER PRODUCT AND USER INTERFACE SEGMENTATION PART II[11] .... 37
1 Introduction
1.1 Motivation
For many people mobiles are integrated in the everyday life beyond telephony.
They are regular smart companion. In this coherence the smart companion is also called smartphone.
This special kind of mobile phone is generally considered to be any handheld device that integrates personal information management and mobile phone capabilities in the same device. It often includes "smart" capabilities, such as PDA functions. The key feature of a smartphone is the possibility to install additional applications to the device. The applications can be developed by the manufacturer of the handheld device, by the operator or by any other 3rd party software developer.
Compared to 2004, smartphones have gained the larger part of the mobile phone market. In a couple of years, it is likely that most of the sold phones will be considered "smart" [1].
By using default applications or installing 3rd party software, smartphones can be utilised for playing movies and sounds, taking, showing and storing photos, organizing schedules, accessing the Internet, route planning etc. The range of possible applications is almost unlimited.
The number of available applications grows persistently. To unburden the user, a preselection of context-relevant application items would be useful. If it’s stated that the user is at or near to his office, he needs business-relevant items, for instance the calendar or a ToDo list. If the user is in front of a grocery the purchase list item is relevant. But if this store is closed at the actual time, this item becomes useless.
1.2 Research Questions
The aim is to offer an architecture for location-based user interfaces. The system should work with a minimum of special hardware requirements. Not to overload the user with information, the resulting user interface should be adaptable, context-aware and location-based. The context-data should remain extendible and adaptable. The following research question has been formulated:
How should the architecture be designed to offer a flexible user interface for general smartphones, which is able to adapt to the current context and environment?
1.3 Method
To understand the topic, the background will be analysed and clarified. Relevant issues are introduced and made clear with regard to the research question.
Then, the functionality and features of the prototype and the techniques used are presented.
In an evaluation, the motivation and the research question will be compared with the prototype. Eventually, it will be shown, that the prototype provides an adequate functionality to deal with the research question. At last, extensions and possibilities for further work are presented afterwards.
1.4 Restrictions
The cellular phone networks are moving targets as developments very intense.
This means that Symbian OS has to be adapted too. At the origin of this thesis, Symbian OS 8.0 is the latest available version on cell-phones. It is possible that some used libraries are not available in older or depreciated in future versions.
The user interface in this thesis is adapted and tested on Series 60 mobile phones. To use the developed prototype on other Symbian OS phones, for instance UIQ, the User Interface needs to be adapted. With regard to this, not so much effort was spent to design a startling user interface design.
2 Location-based services
Three technologies are making progress rapidly in IT: Ubiquitous or Pervasive Computing, mobile Computing and location-based technologies [2].
The software and technologies that will be developed in this project are found in the intersection of these technologies (Figure 1).
Figure 1: Classification of the topic
Challenging technical issues is part of every design process. A useful and established method in computer science is the principle of “divide and conquer”.
The key to understand the target is the understanding of the underlying issues. So they are presented first in the following chapter.
Figure 1: Classification of the topic
Mobile
Computing Ubiquitous
Computing Location-based
Computing
„Flusi“
2.1 Mobile Computing
2.1.1 An introduction into Symbian O.S.
In 1998, the software developer Psion founded the company Symbian together with the four leading cell phone manufacturers Nokia, Ericsson, Motorola and Panasonic [3]. This company developed – based on Psion’s EPOC – the Operating System Symbian OS.
Symbian OS is a robust multi-tasking operating system and is designed for the special constraints of mobile phones such as limited amount of memory and limited speed of CPU.
It has “…a number of characteristics that make it suitable for mobile computing and better against other operating systems” [4]. Some of the characteristics are:
- Stability and Security: A system crash on a mobile phone could be fatal for the user and the acceptance of the phone, so the operating system has to be very stable [3].
- Electrical Power Consumption: To save power, the operating system has to save resources [3].
- Similarity to a Real-Time Operating System: The telephone features of the device need definable and preferably short response times which are similar to Real-Time Operating Systems [3],[5].
- Integrated Multi-Mode Mobile Telephony: Symbian OS integrates the power of computing with mobile telephony, supporting complex computations, advanced data service and telephony service on a single mobile machine [4].
- Open Application Environment: Applications written in different programming languages, like C++ and Java, can be deployed onto Symbian OS [4].
- Open Standards and Interoperability: Symbian OS provides a core set of APIs that is shared by all Symbian OS phone. Thus, a program written using the core set of APIs can be used in many Symbian OS phones without the need of modifying the source and recompiling [4].
- Multi-tasking: Symbian OS fully implements multi-tasking. Its special feature active object allows program requires multi-threading to be written with only one thread with several active objects [4].
- Fully Object-oriented: Symbian OS is designed in using object-oriented techniques. It also allows object oriented programs [4].
- Flexible User Interface: The graphical user interface in Symbian OS is flexible; developers can design their own user interface easily. Nokia for example developed its own specific GUI called “Avkon” for its mobile phone [4].
The following description of the components of Symbian OS is mainly based on two sources – [6] and [7].
Symbian OS is dividable into four significant component types (Figure 2):
Kernel, application, server and engine. As every modern operating system, Symbian OS has boundaries between the different software components. These boundaries act as protection against illicit access to memory, other components or hardware. Thereby the system gains more stability. The components of the system are shown in the following Figure 2.
Figure 2: System components
The kernel manages the hardware resources of the mobile, such as the system random access memory (RAM) and hardware devices. It controls and provides the access to all the other software components of the resources that are managed by it. The kernel has hardware-supported privilege to gain access to these resources. Only the kernel has the privilege from the central processing unit (CPU) to perform certain instructions. The CPU runs other programs – the so- called user-mode programs – without privilege, so they can only access system resources through the kernel application programming interfaces (API). The boundary between kernel and user-mode programs is a privilege boundary.
An application is a program with a user interface. Each application runs in its own separate process with its own virtual address space, which guarantees that one application cannot accidentally overwrite the data of another application, because their address spaces are entirely separate. The boundary between applications is a process boundary.
A server is a program without a user interface. It manages one or more resources providing an API to its clients to gain access to its services. Each server generally runs in its own process that means that the boundary between a server and its clients is also a process boundary. For performance reasons some closely related servers may run in the same process.
Privilege boundary Process boundaries
Kernel
Server Server Application
Application
Engine
DLL boundary
An engine is the part of the applications that manipulates its data, rather than interacts directly with the user. An application engine may be a separate source module, a separate dynamic link library (DLL), or even a number of separate DLLs. The boundary between engine and application is either a module or a DLL boundary.
DLL and module boundaries that promote system integrity by modularisation and encapsulation are very cheap to cross. The privilege boundary, that promotes system integrity by hiding the kernel and devices from user-mode the code, is more expensive to cross.
In a multi tasking operation system like Symbian OS, each Thread has a stack and a heap memory. The life time of data in the stack is as long as the life time of the allocating function. The de-allocation of data in the heap occurs exclusively by the programmer.
The most important implementations of Symbian OS are described below.
Descriptors are responsible for string processing in Symbian OS. They offer a safe and consistent work with strings and binary files as well. They are more comfortable than string processing in C and provide full control over used memory.
Descriptors can be distinguished into non-modifiable and modifiable descriptors (Figure 3). Non-modifiable descriptors, marked with a capital letter ‘C’ at the end of the name of the descriptor class, can be accessed but they are not changeable. They just can be replaced completely. Modifiable descriptors can be accessed and changed. Like non-modifiable descriptors, they contain the current data length information but additionally, they need the maximum length information. This extra information is needed to allow changes within the data.
An overview about the descriptor classes is given in Figure 4.
Descriptors can be located in different memory parts. Descriptors in the source code are located in the program memory, i.e. the ROM. Other allocatable memory parts are the stack and the heap (Figure 3). The macro “_LIT” allows the declaration of strings in the source code. So, the command _LIT(KHelloROM,
“Hello”) generates the descriptor “Hello” in the program memory.
A pointer descriptor is generated by the command TPtrC helloPtr = KHelloROM.
“helloPtr” is located in the stack, but the data it points to can be located in the heap or the stack.
A buffer descriptor TBufC lives in the program stack. The command TBufC<5>
helloStack (KHelloROM) generates the string “Hello” on the buffer.
Figure 3: Descriptor overview
In heap descriptors, the data is part of the descriptor object. The descriptor object is located in the heap. To generate a string in the heap, it is necessary to generate a buffer object and copy the data to it: HBufC* helloHeap = KHelloROM().AllocLC. The result is the string “Hello” on the heap. The Pointer helloHeap points at it.
Figure 4: Overview descriptor classes
changeable
Pointer
Buffer
Heap
constant 5
Hello
TPtrC 5 8
Hello
TPtr
5 Hello TBufC<5>
5 8
TBuf<8>
Hello
5 Hello
Memory is a scarce resource on a mobile device. To prevent out-of memory problems, it is very important to free unused memory. Hence, Pointers to allocated memory must be pushed to the cleanup stack to handle memory cleanups in leave situations. For instance, it is supposed that CX is a class and x is an object reference. Subsequently, the object is used in a function x->UseL() which causes an error. Now the object reference on the stack disappears and CX on the heap memory is no longer reachable and cannot be destroyed either (Figure 5).
Figure 5: Heap Failure
This problem can be avoided by using the cleanup stack,. After creating the object reference x, a second reference is created on the cleanup stack with the command CleanupStack::PushL(x). Finally, the reference on the cleanup stack is being deleted by CleanupStack::PopAndDestroy(x). So, usually nothing happens except creating and deleting a needless reference. But in case of an error, the function has still a reference on the cleanup stack to delete the object (Figure 6).
Figure 6: Functionality of the cleanup stack
x
CX
CX* x = new (ELeave) CX;
CleanupStack::PushL(x);
x->UseL(); // leave while execution
…
CleanupStack::PopAndDestroy(x);
Pointer x is destroyed
Heap memory of CX is still accessible from Cleanup Stack
X
Stack
Heap Cleanup stack
x
CX
CX* x = new (ELeave) CX;
x->UseL(); // leave while execution
Pointer x is destroyed
Heap memory of CX is no longer reachable
X
Stack
Heap
In spite of the cleanup stack, there are still some situations where memory may leak. To use the cleanup stack, it is necessary to push a pointer to the allocated memory. But this is not possible at object creation in between the allocation performed by new and the invocation of an allocation in the C++ constructor.
Therefore, the C++ constructor should not fail, which forces a two-phase construction. The two-phase construction divides the construction process into two parts or phases. The first phase constructor is the normal construction, denoted by the object’s name. This object is pushed onto the cleanup stack. All operations that might leave are done in the second phase construction. So if an operation leaves in the second phase construction, there is still a possibility to deallocate the used memory, because it is already on the cleanup stack.
When a class is inherited, the second phase constructor of the base class has to be called explicitly. It is not called automatically like the first phase C++
constructor. To simplify the use of classes with two-phase construction, it is conventional to provide a static factory function NewL(). It performs the two- phase construction and returns a pointer to the created object.
Symbian OS is a multi-tasking operating system as mentioned above. But it does not achieve the multi-tasking capability by multi-threading alone, like many other operating systems. It utilizes an additional mechanism called active objects. The main benefit is lower system overhead. With active objects, there exists only one thread per application / server. The requests, sent by one or more active objects, will be scheduled by the active scheduler. So the request can be served in an interleaved manner. Active objects allow non pre-emptive multi- tasking within a single thread. Hence, there is no mutual exclusion code needed to use active objects. In that way, the code complexity is reduced while concurrently achieving multitasking.
2.1.2 Smartphones
The fact that applications can be installed on a cell phone like on PCs give us new opportunities since it is portable and with its means of communication possibilities. Implementing core functionality in software, not in hardware, is also a cost effective approach [8].
Programmable phones were already available in 2001, when Java MIDlets were introduced. But Java always runs on a virtual machine. So machine-oriented properties like access to the Sim-Card, address book, communication interfaces etc. is restricted. Another area which is restricted is the Bluetooth-interface which isn’t available at Java MIDP 1.0 and 2.0.
So the presentation of the first Symbian-phone in the end of 2002 was awaited eagerly. The Nokia 7650 and the Sony Ericsson P800 were the first available real smart phones [6].
There is no single feature which is characteristic for a smartphone but usually a phone is considered as “smart” if it offers the following characteristics [6]:
- Colour-display
- Applications like email, calendar, address book etc.
- Extendible through installation of other applications
Symbian O.S is not the only operating system for smartphones but Symbian’s share of the market is according to a study of the British market researchers Canalys near 61.4 percent worldwide. The operating system Windows reaches 18.3 percent and PalmSource has 10.5 percent market share (Figure 7). A list of available mobile phones using Symbian O.S. is shown in the appendix.
18% 61%
11%
10%
Symbian Microsoft PalmSource Others
Figure 7: Market Shares of Smartphone’s O.S.
2.1.3 Series 60 Platform
Series 60 is a software platform for smartphones with advanced data capabilities.
The Series 60 Platform is a source code product that manufacturers can port and integrate into their own hardware designs [9]. Currently there are three editions available (Figure 8).
Figure 8: The currently two main versions of Series 60 Platform
The Series 60 Platform 2nd Edition was initially made available in 2003 and was first implemented in the Nokia 6600 device. It has been extended since with three Feature Packs bringing new functionality without compromising the platform compatibility.
The latest edition of Series 60 Platform supports all the features and functions of the 2nd Edition and more - especially in enterprise and multimedia areas. In addition, the Series 60 Platform 3rd Edition introduces a new level of flexibility and security into the platform, enabling manufacturers more easily to create devices targeted to mass markets. By this, Series 60 Platform provides an open, secure and scalable business opportunity to mobile operators and 3rd party developers [9].
2.1.4 Phone-Hardware
Symbian devices consist of a central processing unit (CPU), memory (RAM, ROM), I/O-Devices and a power supply [6]. The hardware components are described in detail below:
- CPU: Symbian OS is a 32-bit operating system, so the used CPU are 32-bit-CPUs. Actual smartphones use an ARM respectively StrongARM [10] CPU with between 36 and 156 MHz processing power.
- Memory: Smartphones own as memory a ROM and a RAM
ROM: The ROM contains the operating system and basic applications. The size of the ROM is varying on different Smartphones; most of them have 12 or 16 MB ROM. All files in Feature Pack 3 Scalable UI, high
resolution support Feature Pack 2 WCDMA and CDMA
network support
Feature Pack 1 HTML small screen browsing and Edge network support
Series 60 Platform 2
ndEdition
Series 60 Platform 3rd Edition
All the features of the 2nd Edition + enhancements in
• Enterprise functionality
• Multimedia functionality New platform architecture
• Hardware flexibility
• Increased security
the ROM are accessible over the drive letter Z:. All files are direct addressable, hence they are executable without loading into the RAM.
RAM: The RAM acts as working memory for the kernel and applications. Additionally, the RAM provides a persistent storage medium since there’s no hard disk drive. The RAM has a size of, depending on the mobile phone, between 3 - 16 MB.
- I/O-Devices: The main I/O-Devices are the display, possibly with touchscreen-functionality and a keyfield. A Card-Slot offer additional memory and is addressed as drive with the capital letter D:. Possible I/O-Devices are also cable, Bluetooth, infrared, WLAN and access over phone-services like GPRS or UMTS.
- Power supply: The power supply is eminently important on a battery- powered device. It’s normally the embedded rechargeable battery or an external power supply.
2.1.5 User interface design on mobile phones
Are the problems with user interfaces just a fraction of those with desktop-PC UI? Why is it necessary to distinguish between desktop and mobile?
Not only business users use cell pones like in the beginning. Cell phones provide immediately access every time and everywhere to everyone. So it has to be carried everywhere. To reach portability, decreasing size is the easiest way [11].
The battery shall keep the phone alive as long as possible, so the phone- electronic must be optimised for high performance in small size and less energy demand. But the small size limits the number and the dimension of buttons [11].
Also the amount of data, which can be presented on the display, is very limited.
Desktop interfaces are characterized by direct manipulation. They provide a permanent representation of all objects of interest, the physical operations that can be performed on them and immediate visual responses to such manipulations [11]. The manufacturer offers a huge set of features, provides them all and the users select his preferences and priorities.
“The main difference in designing UI for desktop environments versus phones is about quantity, desktops can accommodate more.” [11]. Small displays with
“small” interfaces have to scale downward. They bear on parallel representation. Here, the options and items are displayed in a sequential representation. The user has to browse through options, one by one. The hardest point is the user-friendly priorization [11]. Figure 9 shows the path of reasoning used by Nokia, to find the user needs and present them in compact and portable design.
Figure 9: The path of reasoning from portability to user understanding [11]
2.2 Pervasive / Ubiquitous Computing
The history of computing is often divided into three waves [12] (Figure 10). The first wave of computing was dominated by many people serve one computer. The second wave was dominated by personal computers and the most increase wave is the assumption that one person serves many computers. In long term, computing access will be everywhere [13]. But it turned out that not necessarily the number of used computers arises. In fact, by rising portability the availability of processing capability surrounding the user is increasing. But what is ubiquitous and pervasive computing?
Portability
Small Size
Small Screen
Sequential Presentation
Priorization
User Understanding
Figure 10: Waves of computing [12]
“A few years ago I found myself on a stage at the MIT Media Lab, arguing with Nicholas Negroponte in front of 700 people. Nick was rhapsodizing about a world in which computerized "intelligent agents" will answer our every need. To illustrate Nick's idea, an actor dressed as a butler introduced speakers and entertained the audience with snide remarks. The butler was fun, up to a point, but also distracting and intrusive. Fortunately, Nick was wrong about what to expect from the third wave in computing. The defining words will not be
"intelligent" or "agent", but rather "invisible" and "calm" and "connection".”
by Mark Weiser Principal Scientist Xerox PARC
March 1996
Pervasive computing is just another term for ubiquitous computing; it is rather used by industry. The term ubiquitous computing was coined by the above cited Mark Weiser. Ubiquitous computing describes the phenomenon of interacting with artefacts and environments with processing and communication capabilities.
Computing is integrated in the environment in contradiction to having computers which are distinct objects. Ubiquitous computing is not virtual reality and not a personal intelligent agent [13]. A hard challenge is the integration of ubiquitous computing in the everyday physical world. It has as its goal to augment the real world. It should be calm, almost invisible and in the background [13] so that the user spend his time for the task, not for operating a computer.
The smartphone gives a shape to these abstract ideas [14]. It provides the often postulated cheap wireless networking [13]. It is the personified abstract handheld that are so often mentioned in visions of pervasive computing [14]. “Smart Phone
In the same way, also software is moving from desktop computer into new and unknown territories [8]. This entails many new challenges. Deployment and maintenance on ubiquitous devices is different to deploying to and maintaining desktop PCs [8]. Also a tight integration of services is needed, as demanded by the small form factor of the devices typically found in ubiquitous computing environments [15].
But ubiquitous computing is not without issues. In connection with pervasive / ubiquitous computing, five critical topics are currently discussed [16]:
- Privacy - Security
- Unmastered Complexity - Free Speech
- Intellectual Property
2.3 Location-based services
The development of technologies to communicate with mobile devices and sensors forges ahead. By the advances in wireless communication, and positioning technologies, location-based services become possible. Many new communication innovations are on the brink of breakthrough. But what are location-based services? This chapter will set the main focus on location dependent services, the term context awareness will although be reflected.
Definition:
A Location-based service is a “…service, query or process whose return or other property is dependent on the location of the client requesting the service and / or of some other thing, object or person.” (ISO TC 211)
The term location-based service denotes applications that connect geographic locations with the general notion of a service [17] to add value to a user.
According to Vodafone, location-based services are distinguishable into two types:
- Active services: The User initiates the location request (e.g. find- nearest… applications)
- Passive services: A third party locates the position of a location-based service participant (e.g. find-a-friend functionality)
2.3.1 Entities
The world consists of infinite many things. In the context of this thesis, we have to care about the resources and items of a mobile phone. To handle them, it’s necessary to map the features, applications and possibilities of the target devices to a finite set. This set is called universe and consists of representations of “real- world things”. The elements of the universe are called entities. To identify them, an entity needs a set of properties that makes it identifiable [18]. They also have to be distinct [18].
2.3.2 Locations
We consider that a location is a place or range, where an entity is positioned in the real world. Locations can be expressed in many ways using different frames, for instance as an absolute spatial location, a descriptive location or a relative location [19]. They can be distinguished in physical positions like 49° 39’ 31’’ or in symbolic locations like home or workplace [20]. Physical and symbolic locations are identified by coordinates. A coordinate is an identifier, which specifies the position of an object with respect to a given coordinate system [21].
A coordinate system adds a special context to a set of coordinates. It’s possible to classify coordinates of locations in two different formats:
- Geometric coordinates: Positions are defined in form of tuples relative to a reference coordinate system [21]. Geometric coordinates can be distinguished further in:
Absolute coordinates: These coordinate systems are valid worldwide like for example the WGS84 system or the Transversal Mercator System. It adds to every point of this planet a triple of coordinates.
Relative coordinates: Geometric coordinates can not only refer to global areas, they can refer to local areas too e.g. Cartesian coordinates in a room with indoor navigation. This coordinates can also describe a self-centred coordinate system, all other locations are described with respect to the own position.
- Symbolic coordinates: Positions are defined by abstract symbols, for instance by Cell Ids or Ids of access points the device is connected to. The Symbolic coordinates describe only the affiliation to a sector.
2.3.3 Location sensing techniques
problems. The location is a very personal and unique information. So it is a superb attribute to use in personalization [5] and a significant context parameter [22]. Almost all information is relatable to a location. To obtain the position, a positioning system is needed. According to Hightower and Borriello [20] we can choose from three major techniques:
- Triangulation: Triangulation uses the geometric properties of triangles to compute the current location. It is divisible in Lateration and Angulation.
Lateration calculates the location by measuring the distance from multiple reference positions. Angulation is similar to Lateration but instead of distances, the angles of the reference positions are used [23].
- Proximity: A proximity location sensing technique entails determining when an object is “near” a known location [23].
- Scene analysis: The scene analysis location sensing technique determines the location by observing a scene from a particular vantage point [23]. To obtain characteristic features of the scene, it will usually be simplified.
No location sensing technique takes perfect care or works well in all situations [24]. More accurate positioning requires more resources and more technical effort. So it’s important to choose the positioning technique as accurate as necessary and as easy as possible. Ericsson researched into this issue and developed at high level a scheme for the level of accuracy required (Table 1). An overview of state-of-the-art positioning techniques is in the appendix.
Table 1: Required level of accuracy [17]
Application Accuracy Application Accuracy
News Low Gaming Medium
Directions High M-Commerce Medium to High
Traffic Information Low Emergency High
Point of Interest Medium to High Sensitive Goods Transportation
High
Yellow Pages Medium to Low Child Tracking Medium to High
Car Navigation Medium to High Pet Tracking Medium to High
Personal Navigation High Electronic Toll
Collection
Medium to High Directory Assistance Medium to High Public Management
System Medium to High
Fleet Management Low Remote Workforce
Management
Low
Car Tracking Medium to High Local Advertisement Medium to High
Asset Tracking High Location-Sensitive
Billing Medium to Low
2.3.4 Context awareness
The term context embodies about everything that constitutes the user’s environment and the users themselves.
An existing example for context awareness on desktop-PCs is ICQ. The ICQ- Messenger assumes after a period of inactivity, that the user is away and switches automatically its user mode to “away”. On mobile devices there is much more context information available because the device is always on and always with the user [22].
2.3.5 Filters
The intention is the customisation of services to the context and the user profile.
Location based services process the movement and queries of their users in multidimensional databases, i.e. data warehouses [26]. The dimensions usual are:
- Location of the user - User profile
- Time
This context information also known as presence information refers to the physical and social situation, the computational devices are embedded [22].
2.3.6 Privacy and Security
Though this is not part of this paper, it should be mentioned that location-based services also evoke some perils. Location based services and the connected personal information, for instance positions, preferences and user profiles are a high privacy risk. Also the same spamming problem as in email traffic is possible. As result of these potential threats, European Parliament has incorporated these issues in its Directive on Privacy and Electronic Communications [5].
3 Flexible user interface prototype “Flusi”
3.1 Scenario
The scenarios described below are based on every day situations. The Flexible user interface prototype Flusi is used to make these situations easier. Services that do not benefit from positioning and personalisation are not described in detail. These uses-cases should help illustrating the benefits of using the mobile service.
Peter is in his office. With regard to the current cell id, the mobile recognize that it is in or near Peter’s office. Here, he needs primarily his phonebook, the calendar item, a Bluetooth connection for synchronization with his desktop PC, the voice recorder and his ToDo list. So these items are displayed. Other items like the notepad, browser or the email-client are now useless because his PC offers a more comfortable way to do these tasks. Therefore these items are hidden. Also leisure time features like camera, camcorder, SMS and MMS are not
Figure 11: Office relevant items, filtered by cell id
Scroll down
displayed. By this preselection, Peter can find relevant features easily and comfortably without tedious clicking and scrolling (Figure 11).
In the afternoon, Peter leaves his office for an external business appointment. It’s an unusual place for business, so Peter has to select the context manually (Figure 12). He browses through the points of interest and select the item “Business Trip”. On a business trip, other items are relevant than in office. Peter doesn’t need the Bluetooth connectivity for synchronization anymore. He does not have a desktop PC, so he needs his mobile to write emails and notes. Hence accordant items appear.
Figure 12: Selecting POI manually
Homebound, Peter switches on his GPS receiver to enjoy more exact functions of his location based service. Peter continues to a grocery. At home, he completed an electronic shopping list. Now, the mobile phone recognizes the proximity to the grocery and the grocery list item appears with a “beep” on the display (Figure 13). Peter is reminded and finds the shopping list application very comfortable in the list of the relevant items on the display of his mobile. Afterwards, Peter continues his walk and goes along a drugstore. He has some sanitary products on his shopping list too, but this drugstore is only open till 5 p.m. and it’s already 6 p.m. So there’s no need to inform the user about this store and the mobile doesn’t display the shopping list item near this location.
Figure 13: Selection of Locations via GPS
One week later, Peter is on holiday in London. Of course, he brings his mobile phone. By the cell-id, the mobile phone recognizes the location. The schedule and all work relevant stuff are automatically not displayed. In this way, Peter can really relax on his holidays. Instead of postcards, he sends MMS and SMS messages to his friends. These holiday-relevant items are of course displayed.
The most important things on his trip are the built-in mega pixel digital camera and camcorder functionality of his mobile. To get more exact locations Peter switches on his GPS-Receiver. When the mobile detects a dignified place for a picture, it switches the camera icon on and notices this with a “beep”. So, Peter can enjoy his trip and doesn’t miss any sight. Another important tool is the currency converter which is displayed in the UI. The clock-application appears automatically from 10 to 12 pm. This is Peter’s usual time period to go to bed and he didn’t take any additional clock with him.
3.2 Architecture
The system consists of three main parts (see Figure 14):
- The User Equipment - The Provisioning Server - The Positioning part
Item appears in adequate proximity to the affiliated location.
Figure 14: Architecture of Flusi
The User Equipment is the device, on which the Flusi-client is running. It acts as the interface between user and system and overlaps the built-in user interface. By selecting an item in Flusi, the underlying application is executed. After closing the application, Flusi becomes active again.
The Provisioning Server offers the database and points of interest. It is divided as a separate part to save phone’s resources and offer accessing possibilities for 3rd party. The same points of interest are accessible for several users.
The Positioning part includes all positioning devices usable for location-based services with a mobile phone. The GPS-Receiver is the most descriptive positioning device because it’s a separate tangible thing. Positioning over cell id doesn’t result from a single external device. Due to the fact that data from external devices are necessary, it is also counted among a self-part.
XAP
Positioning Provisioning Server
User Equipment with
„Flusi“-application User
3.3 Used Techniques and Functionality 3.3.1 The universe
The universe of the mobile phone is stored in a XML-file (Figure 15). The describing DTD-File can be found in appendix. This file contains the entity name, the entity unified resource identifier <URI> and a label for the display.
This file is device-specific and adapted to the current system. The distinction in entity name, URI and entity label is meaningful for various reasons. The URI is device-specific, for example “z:\\system\\Apps\\browser\\browser.app“ is the URI for the Browser on the used Nokia 6680. On the phone, the URI is unique.
The entity name is a device independent shortcut for that URI to ease the handling. It has to be unique on the device, but also at the entire Flusi system.
The entity label <ELabel> is responsible for a user-friendly display text. Because of the constraint of uniqueness of entity names, the <ELabel> allows a more flexible nomenclature. They don’t have to be distinct and it can be useful to use the same display text for some different entities. The XML-architecture allows increments like adding icons to the entity easily by adding additional tags to the XML-file.
Figure 15: Universe file
3.3.2 Locations and filters
The location data is saved in an XML-File (Figure 16). The describing DTD-File can be found in appendix. Several filters are applicable on locations. So, a location’s attribute “determination” destines the convenient filter. The element
<Universe>
- <Entity Name="Grocery_List">
<URI>c:\\system\\Apps\\shoppinglist\\shoppinglist.app</URI>
<ELabel>Grocery List</ELabel>
</Entity>
- <Entity Name="Contacts">
<URI>c:\\system\\Apps\\contacts\\contacts.app</URI>
<ELabel>Contact Database</ELabel>
</Entity>
- <Entity Name="Browser">
<URI>z:\\system\\Apps\\browser\\browser.app</URI>
<ELabel>Internet Browser</ELabel>
</Entity>
</Universe>
<locname> adds the location name for the user-interface. <locdata> consists of the filter specific data. An element “entities” lists all relevant items for the location.
Figure 16: Location file
On the phone, one or more filters can be switched on/off over the menu. Five filters are implemented in the prototype. They are chosen from the menu, enabled filters are displayed in the top of the display (Figure 17). The implemented filters are:
- All: This filter deactivates all other filters and all on the phone currently available entities are shown. The term “All” doesn’t appear in the XML- File, it’s a reset-function for the following determination methods.
- Cell ID: The entities are selected convenient to the current cell id. The cell id of the location is saved in the element <locdata>.
- GPS: This filter needs external hardware i.e. a GPS-Receiver. By activating that filter, the phone displays the items. The GPS-coordinates are saved in the WGS 84 format in the element <locdata>.
- Time: The elements are selected with respect to the current time. In the
<locdata> element two numbers are available, a starting hour and an end hour in 24h system.
- None: Not all entities fit to one of the filters. To get a possibility to switch these runaways on/off they are catched in this filter.
<locations>
- <location determination="None">
<locname>Living Room</locname>
<locdata>Home</locdata>
<loc entities="Browser Grocery_List Contacts Sms Mms" />
</location>
- <location determination="GPS">
<locname>Grocery</locname>
<locdata>478046 432432</locdata>
<loc entities="Grocery_List" />
</location>
- <location determination="Time">
<locname>Grocery_MainStreet</locname>
<locdata>10 18</locdata>
<locconnection>GPS</locconnection>
<loc entities="Grocery_List" />
</location>
Figure 17: Filter Selection
Is a filter switched to “on”, all locations with fitting “determination” and
<locdata> are displayed. Exceptions are the filters “All” and “None”, they don’t need any location data. If more than one filter is activated, the different filters are connected by an or. That means if a filter criteria is fulfilled for one or more locations, the entity will be displayed. Thereby, every entity is only displayed once, regardless of how many locations force it. But it can be useful to connect locations by and and not by or. The possibility to do this is given by the element
<locconnection> in the XML-file. This element can link one location to another.
This connection deals with the intersection of the location entities of both locations. These entities will only be displayed if the <locdata> of the two connected entities are fulfilled.
By selecting the menu item “Browse POI”, it’s possible to browse all accessible locations in the local location-file. On a location, one or more entities can be eligible, depending on the attributes entities in the loc-tag. This feature enables the selection of locations and circumstances manually and offers a filter independent use of Flusi.
3.3.3 Distinction between universe file and location file
Using different XML-files for universe and locations is reasonable. The universe is device-specific. By offering the possibility to install applications, the directory structure of every Smartphone is different. Applications can be located in different paths, so the URI of the universe file is not general and needs to be adapted.
Locations are not device-specific. They need only the correct name of the entity but not their positions in the file system. So this disjunction allows a device independent location database.
Selection switches filter On/Off
Active Filters
Browse Points of interest manually
3.3.4 XML-Parser
The eXtensible Markup Language is a standard for man- and machine readable documents. The data is stored in a tree structure. XML provides an easy and flexible way for data exchange between applications. But the used version of Symbian OS doesn’t offer a build-in XML-Parser. Flusi features its own XML- Parser. It is based on Java’s SAX parser, the Simple API for XML parser. It obtains a XML-file and a fitting processing instruction called handler. While parsing the file, the parser calls, for every tag, different event-handling methods of the associated handler. These methods build the data of the XML-file data.
By using different handlers, the parser is universal and applicable for various XML-file types. In the Flusi-prototype, the entity-file and the location-file are parsed with the same parser but with different handlers.
Another widely-used XML-Parser in Java is the Document Object Model-parser or DOM-parser. This parser creates a tree-structure of the XML-file in the memory. Because of the lack of resources on a mobile phone, this approach is not favourable with the current tasks.
3.3.5 GPS
To obtain a better positioning, Flusi uses a GPS-Receiver with NMEA output.
Navstar GPS or simply GPS is an outdoor location sensing technique. It gives
“every centimetre of the surface of the planet its own unique address that can be understood by anybody through the use of a universal geocoordinate system.”
[27].
Figure 18: Space Segment
GPS uses as universal geocoordinate system the WGS 84 System. To describe the
model, a so called Geoid. The WGS 84 Geoid is one of the latest and best-fit Geoids available.
GPS consists of three main segments:
- Space Segment - Control Segment - User Segment
The Space Segment and the Control Segment are operated by the U.S. Military and administrated by the U.S. Space Command and the U.S. Air Force [27].
The Space Segment (Figure 19) implies the constellation of satellites that are
currently in orbit, including operational, inoperable and backup units.
Figure 19 shows an actual GPS Satellite and table 2 gives some technical data.
Figure 19: GPS-Satellite
The Control Segment maintains the integrity of the satellites and the data they transmit. It consists of a Master Control Station at Falcon Air Force Base in Colorado, five unmanned monitor stations, located strategically around the world and three primary ground antennas, located approximately equidistant around the equator [27] (Figure 20). The Master Control Station sends periodically corrected position and clock-time data to the three ground antennas which upload them to the satellites.
Physical Characteristics:
Name: Navstar
Manufacturer: Rockwell International
Weight: ~900 kg
Size: ~5 meters wide
Lifespan 7.5 years
Orbital Parameters:
Altitude: ~20200 km
Orbital Period 12 hr
Orbital Plane 55° to equatorial plane Number of planes: 6
Orbital Spacing 60° apart Table 2: Technical Data
Table 2: Technical Data
Figure 20: Control Segment [17]
The user segment covers all military and non-military users of GPS. With respect to Flusi, these remarks concentrate on civilian use only. The satellites send at a time mark t0 on its internal clock a signal to the earth. The receiver generates that signal at t0 on its internal clock too. When the receiver picks up the signal from the satellite, the time interval ∆t of the signal-runtime is determinable by comparing the internal signal and the received signal with respect to the time t (Figure 21).
─┌─┐─┌┐┌┐─┌┐ ─┌─┐┌┐
─┘─└─┘└┘└─┘└─ ─ ─ ─ ─ ─ ─┘─└┘└
∆t
t Internal receiver
signal
Received Sattelite signal
t0
─┌─┐─┌┐┌┐─┌┐ ─┌─┐┌┐
─┘─└─┘└┘└─┘└─ ─ ─ ─ ─ ─ ─┘─└┘└
The available and necessary data, broadcasted by the satellite are:
- System time and time correction
- Ephemeris data of the broadcasting satellite - Almanac data (ephemerid data of all satellites) - System state
If the receiver receives signals from at least three satellites, it could theoretical determine its position with those ∆t. But by developing the GPS System, one goal was to make cheap receivers possible. So the receivers are not equipped with very expensive but very exact atomic clocks like the satellites. An atomic clock is accurate to within billionths of a second per month and cost something like $200.000 [27]. But the exact time can be determined too by evaluating the data from a fourth satellite, these are at least four equations with four unknowns.
So receivers can be equipped with cheap and very common quartz-clocks. The resulting equation system is not linear, so it’s complex to solve the equations and requires some iteration.
3.3.6 GPS-Interface over Bluetooth
In our experimental setup, Flusi uses a Bluetooth GPS-Receiver from SysOn (Figure 22) to receive the GPS data. It sends position data via Bluetooth to the mobile phone. Bluetooth is an industrial specification for wireless personal area networks. It is named after the Danish king Harald Blåtand who is known for his unification of previously warring tribes from Denmark, Norway and Sweden [1].
Bluetooth is designed for low power consumption, with a comparatively short range (in mobile phones about 10 m).
Figure 22 SysOn Bluetooth GPS-Receiver
To connect Flusi with the GPS-Receiver, the user has to select the menu point
“Connect to GPS” (Figure 23).
Figure 23: Connect to GPS-Receiver
If the Bluetooth device in the mobile is switched off, a dialog box opens to switch it on. Then a list with all available Bluetooth Devices appears. After selecting the GPS-Receiver, entering the receiver specific Pin-Code, the Bluetooth-Connection is established.
Figure 24: GPS Positioning over Bluetooth
After establishing a connection, the receiver streams the position information (Figure 24). The received data is in NMEA format, the most common format for
$GPGGA,132128.0,1404.9392938,N,10036.7686139,E,1,07,1.2,-72.40,M,,,,*2E
$GPGSA,M,3,06,17,23,26,08,29,09,,,,,,2.7,1.2,2.4*31
$GPGSV,2,1,07,17,58,353,47,09,53,095,48,08,46,163,46,06,36,204,43*71
$GPGSV,2,2,07,23,32,332,43,26,23,031,40,29,19,241,37*48
$GPGGA,132129.0,1404.9392161,N,10036.7685764,E,1,07,1.2,-72.31,M,,,,*20
- GGA (GPS Fix Data)
- GLL (Geographic Position – Longitude/Latitude) - GSA (GNSS DOP and Active Satellites)
- GSV (GNSS Satellites in View)
- RMC (Recommended Minimum Specific GNSS Data) - VTG (Course over Ground Speed)
- ZDA (Time and Date)
In Flusi, the RMC dataset is used. A typical RMC dataset looks like this:
$GPRMC,161229.487,A,3723.2475,N,12158.3416,W,0.13,309.62,120598,,,*10<CR><LF>
The following table contains the values and description of the dataset above:
Date Description
$ Start of a dataset
GP Information from a GPS-
Receiver
RMC Identification of RMC dataset 161229.487 UTC-Time
A A=data valid or V=data invalid 3723.2475 Latitude
N N= north or S=south
12158.3416 Longitude
W E= east or W=west
0.13 Speed Over Ground
309.62 Course Over Ground
120598 Date
*10 Checksum
<CR><LF> End of message
Table 3: NMEA-Data
If the GPS-Filter is active, Flusi parses the position every second and tests if the actual position is inside a fixed radius around the location position. If the actual position is inside, the entity will be selected.
Figure 25: Parsed GPS-Data
The accuracy of the used GPS-Receiver is, according to manufacturer information, about 10m. But in experiments, the reached accuracy was much better. The menu-option “Show GPS Data” shows the parsed NMEA-message (Figure 25). This feature is for demonstration purposes only and serves no intention in the Flusi-architecture.
3.3.7 Provisioning
The location information should be updatable. The mobile phone needs a device with a large storage space, which is always reachable from everywhere. So Flusi is integrated in the eXtended Application Provisioning (XAP) system from the MSI, Växjö University and an Apache Tomcat Web server. As database a MySQL database is used.
The data-transfer processes over the HTTP protocol. It allows the required speed and offers the lightness for distributed collaborative hypermedia information systems. It is a generic object-oriented stateless protocol.
In a HTTP protocol transaction, the following lines are set by the client in the RFC822 format headers:
- From - Accept
- Accept-Encoding - Accept-Language - User-Agent - Referer - Authorization - Charge-To
- If-Modified-Since
