Network published positional imagery

Full text

(1)2003:171 CIV. EXAMENSARBETE. Network Published Positional Imagery. LINUS ALMSTRÖM. CIVILINGENJÖRSPROGRAMMET Institutionen för Systemteknik Avdelningen för Medieteknik 2003:171 CIV • ISSN: 1402 - 1617 • ISRN: LTU - EX - - 03/171 - - SE.

(2) Network Published Positional Imagery Linus Almström 2003-06-02.

(3) Abstract This Master thesis investigates how to create a C3 system for devices connected to a common network. These devices may be low on hardware resources and may have limited bandwidth. The thesis is concentrated at techniques to provide the devices with information and how to give an interactive graphical interface for the users. A prototype system using VNC resulted in being useful for devices with low display resolution and low hardware resources. The most interesting conclusion made is to not use the same user interface and data transfer methods on different types of devices, instead one should provide the best alternative to each type of device for the best result..

(4) Sammanfattning Det här examensarbetet undersöker hur man kan skapa ett C3-system för enheter som är uppkopplade till ett gemensamt nätverk. Dessa enheter kan ha l˚ aga h˚ ardvaruresurser, samt endast tillg˚ ang till begränsad bandbredd. Examensarbetet är koncentrerat kring tekniker för att tillhandah˚ alla enheterna med information och hur man kan skapa ett interaktivt grafiskt gränssnitt för användaren. Ett prototypsystem som använde VNC resulterade i att vara användbart för enheter med l˚ ag bildskärmsupplösning och l˚ aga h˚ ardvaruresurser. Den intressantaste slutsatsen är att inte använda samma användargränssnitt och dataöverföringsmetoder för olika typer av enheter, istället bör man tillhandah˚ alla det bästa alternativet för varje typ av enhet för bästa resultat..

(5) Preface This Master thesis was carried out at Ericsson Microwave Systems AB in Lule˚ a. It was the vision to bring C3 systems out to the combat field which encouraged the work. I would like to express my thanks to my supervisor Jonas Lundgren at Ericsson Microwave Systems AB and Mikael Drugge at the Department of Computer Science and Electrical Engineering at Lule˚ a University of Technology. I would further like to thank the rest of the personnel at Ericsson Microwave Systems AB who have been nothing but kind and helpful..

(6) Master Thesis Report Prepared(also subject responsible if other). No. EMW/FH/X Linus Almstr¨ om Doc respons/Approved. Checked. EMW/FH/X Jonas Lundgren. Date. Rev. 2003-06-02. A. Contents 1 Introduction 1.1 Background . . . . . 1.2 Problem description 1.3 Goal . . . . . . . . . 1.4 Scope . . . . . . . . 1.5 Methodology . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. 2 2 2 3 3 4. 2 Related Technologies 2.1 X-Windows . . . . 2.2 VNC . . . . . . . . 2.3 Computer Games . 2.4 MPEG-4 . . . . . . 2.5 ULC . . . . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. 5 5 6 6 7 8. 3 System Description 3.1 Scenario . . . . . . . . . . . . 3.2 Technical Overview . . . . . . 3.2.1 Server . . . . . . . . . 3.2.2 Client . . . . . . . . . 3.3 System Design Considerations 3.4 Requirements . . . . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. 10 10 10 10 11 12 13. . . . .. 14 14 17 19 22. . . . . .. 4 Analysis 4.1 Client Resources . . . . . . 4.2 Bandwidth Usage . . . . . . 4.3 Implementation Complexity 4.4 Summary and Discussion .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. 5 Prototype System 24 5.1 Design Decisions . . . . . . . . . . . . . . . . . . . . . . . . . 24 5.2 Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 5.3 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . 27 6 Results. 30. 7 Conclusion 34 7.1 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 A Acronyms. 39. B Latency Test. 40. Contents. 1(41).

(7) Master Thesis Report Prepared(also subject responsible if other). No. EMW/FH/X Linus Almstr¨ om Doc respons/Approved. EMW/FH/X Jonas Lundgren. 1. Checked. Date. Rev. 2003-06-02. A. Introduction In this chapter the background, the problem, the goal, the scope and the methodology for the thesis are presented.. 1.1. Background. The technology of today has made it possible to expand the area of use for computers. Devices, such as mobile phones and hand-helds, are becoming more and more powerful by the day and are today able to not only perform simple tasks. Network communication has also improved, especially in radio based network communication where technologies such as GPRS1 and 3G2 have evolved in recent years. With this in mind Ericsson Microwave Systems AB has decided to develop the next generation C33 systems. C3 systems are tactical and strategic military systems[4], and within the Swedish military today it consists of several specially built systems. Ericsson Microwave Systems AB has the vision of developing a C3 system which aims to bring out the communication technology to the field, where every unit, or possibly even every soldier, has access to a device such as a hand-held. These devices will be connected to a common network for the possibility to be updated with the most recent information. With such a device the units could be informed of their own location, locations of other objects, friends and foes and other kinds of information. Part of the information fed to the device is the map which the objects are presented on. This thesis will explore the possibilities of presenting such information on various kinds of devices, such as PDAs4 , workstations, etc, strictly based on data from the common network.. 1.2. Problem description. The basic problem that needs to be evaluated is how to design an interactive advanced graphics system for remote clients. What is particularly interesting is to know how bandwidth can be used to ease the burden for light-weight devices and which communication protocols that can be used to transfer information within the system to create an interactive environment. The system must be able to render images on the client and provide an interface for the user to interact with. The user must be able to perform operations to update the screen or to retrieve further information on objects presented, such as getting the speed of objects, zoom in and out of the map, etc. The 1 General Packet Radio Services, a communication system for mobile devices, mostly used in mobile phones. 2 Third Generation, a communication system for wireless devices. 3 Command, Control and Communication, a military term for strategic and tactical systems. 4 Portable Digital Assistant, an electronic hand-held information device.s. 2(41).

(8) Master Thesis Report Prepared(also subject responsible if other). No. EMW/FH/X Linus Almstr¨ om Doc respons/Approved. Checked. EMW/FH/X Jonas Lundgren. Date. Rev. 2003-06-02. A. Figure 1: An overview of the system communication which shows how clients share graphical information.. server is naturally supposed to generate the data the clients will interpret and present (Figure 1). What kind of data the server provides and what interaction protocol to use is part of the problem. The client device can more or less be of any kind. It could be a laptop, a workstation or a hand-held, which means the devices can differ in hardware resources. Not only processing power can differ between the clients, but so can bandwidth, display size, etc. This means that the the system needs to be designed to efficiently use the bandwidth and to be careful of using the client’s resources. All kinds of devices should be able to use the system at the same time, thus care must be taken when designing the system.. 1.3. Goal. The aim for the thesis is to. . . - examine and evaluate related technologies. - design a solution for the given problem. - implement a prototype system for the designed solution. - propose a strategy for further development of products in the domain.. 1.4. Scope. The scope of the Master Thesis is to create a prototype of a system which has a two way communication between a server and a client, where the client shows the information asked for and exists in an interactive environment. Using the prototype as an example, a proposal for future development is required. The thesis will be concentrating on how to construct an interactive environment between the server and the client. Naturally an investigation of related technologies will be done and the knowledge gained from the research 1. Introduction. 3(41).

(9) Master Thesis Report Prepared(also subject responsible if other). No. EMW/FH/X Linus Almstr¨ om Doc respons/Approved. EMW/FH/X Jonas Lundgren. Checked. Date. Rev. 2003-06-02. A. is then used to design the prototype system. The prototype system must make it possible to render an image on the client. Since there is a requirement for the system to be interactive as well, the prototype system must also provide functionality for the client to instruct the server to retrieve the data it wants. The research of related solutions will mainly be focused on bandwidth usage, client resources and the overall complexity of the system implementation. Other various aspects could be discussed, but due to the time frame that will not be done. The number of related technologies have been limited to five, X-Windows, VNC5 , Games, MPEG-4 and ULC6 , which have been chosen carefully with respect to their relation to this problem. There will be various limitations regarding functionality of the prototype system being created, for it not to be a too huge of a task. The aim for the prototype is to create a basic implementation of the system with a small set of features.. 1.5. Methodology. The first thing to do was to find related technologies suitable for the target system. This was mainly done by browsing the Internet and brainstorming, with help from the supervisor and other colleagues at Ericsson Microwave Systems AB. To find more detailed information about the subjects the Internet was used again for searching technical documentation as well as technical articles. Lucia, which is an Internet search engine for the Lule˚ a University Library, and Libris, which is an Internet based search engines for Swedish libraries, were used to find books on related subjects to further extend the knowledge about them. Using the material found, the technologies were examined and evaluated for usage in the target system. Thereafter using the knowledge about related technologies and their solutions a system was designed. This system was after that implemented. With the results of this project and the knowledge of third party solutions, a proposal for future development was suggested.. 5 6. Virtual Network Computing, a system for remote control using a graphical interface. Ultra Light Client. 4(41). 1.5. Methodology.

(10) Master Thesis Report Prepared(also subject responsible if other). No. EMW/FH/X Linus Almstr¨ om Doc respons/Approved. Checked. EMW/FH/X Jonas Lundgren. 2. Date. Rev. 2003-06-02. A. Related Technologies This chapter will bring up a few technologies which could be interesting to introduce in the system described in the introduction. The chapter briefly covers X-Windows, VNC, computer games, MPEG-4 and ULC.. 2.1. X-Windows. X-Windows is a relatively old system for creating a graphical environment for applications under UNIX like operating systems. It is based on server-client methodology, where the X-server7 shows graphics on the screen and the Xclient is the actual application. X-Windows was created originally, among other things, to unite differences between operating system and hardware in to a common graphical system, but as well for the possibility of remote usage of the applications in a graphical environment.[18] The X-Windows protocol is based around a concept of graphical primitives, such as ‘create window’, ‘draw line’ etc, and in such a way the X-client can instruct the X-server what shall be presented on the screen. It is a twoway communication and the X-server sends messages back to the X-client when needed, mouse clicks and such. The X protocol uses a relatively small amount of bandwidth, but there are occasions when the X protocol is not optimum. In image intense applications a large amount of raw data is sent through the network, which in turn results in unacceptable delays in slow or highly used networks.[10] This was not seen as a big problem by the ‘X Consortium’, and was thus not given a high priority, despite suggestions for improvements.[10] The suggested solution to the problem is called XIE8 . XIE is an extension of the X protocol to reduce the usage of bandwidth for image intense applications. With XIE a number of different alternatives for image compression has been added to reduce the amount of data which is sent between the X-server and the X-client, for example monochrome images can be compressed to one twentieth, or less, of the original size. This means the X-client gains more control over how the X-server should act, and also a possibility for the X-server to perform certain operations supported by specific hardware to decompress the images. By using specific hardware the X-server and the X-client can lower the CPU usage when dealing with XIE. An X-server is relatively complex software since there are many primitives which an X-server needs to be able to handle. In graphic intense environments an XIE implementation is needed as well, assuming a slow 7. Warning: Do not mix up the terminology. A server according to X-Windows is the application that is run locally, contrary to what is the case for most of the other technologies. 8 X Image Extension, a concept of optimizing bandwidth usage when transferring images.. 2. Related Technologies. 5(41).

(11) Master Thesis Report Prepared(also subject responsible if other). No. EMW/FH/X Linus Almstr¨ om Doc respons/Approved. EMW/FH/X Jonas Lundgren. Checked. Date. Rev. 2003-06-02. A. network, which complicates the X-server further. Utilizing XIE the processor is used more, alternatively using specific hardware.. 2.2. VNC. VNC is a system that is used to export graphics from a server to a number of clients. The clients show the graphics the server sends and are stateless. From being stateless follows that several clients can show the exact same information from one and the same server and can thus more or less “control” each other. Another benefit with stateless clients is that they are not bound to a continues connection. A client can disconnect and then connect again to the server and continue where it left off. VNC is foremost known for remote usage of computers, where the server visualizes the desktop on the client. There is however nothing in the VNC specification that hinders an implementation of a VNC server to be used for more or less anything.9 [25] VNC uses a protocol called RFB10 . RFB is a protocol aimed at usage for “thin clients”[21], thus the main issue when designing the protocol was to create a protocol which requires the least from the client. Due to this, as mentioned above, the clients are stateless. The visualization part of the protocol consists of more or less one primitive, “put a rectangle with pixel data at given x and y position”. It may seem inefficient, but can be optimized using different encoding algorithms. The usage of different encoding algorithms makes RFB very flexible, where the choice of encoding algorithm can be decided by measuring different parameters, such as bandwidth, client resources and server resources. The update mechanism in the protocol is driven by the client asking to get updated, which makes an RFB supported system very dynamic, which can adjust to how much bandwidth there is to utilize as well as how powerful the client is. The client can control the server through sending messages such as a mouse click has been registered, a key has been pressed or other kinds of events have been registered.[21] In a VNC system the highest complexity lies at the server. How complex the client, or clients, is can be chosen, since the clients choose which type of encoding it can deal with. The server on the other hand must understand every single encoding type the clients in the system might use, which increases its complexity compared to the clients.[21]. 2.3. Computer Games. Computer games have during a period of time, and still do, used the network to be able to play more than one player in a game. Instead of sending graphics over the network or instructions of how objects should be drawn, 9. An example of this is a very simple VNC server which only shows a counter or a digital clock. <http://www.uk.research.att.com/vnc/rfbcounter.html> 10 Remote Frame Buffer, the low-level protocol used in VNC.. 6(41). 2.2. VNC.

(12) Master Thesis Report Prepared(also subject responsible if other). No. EMW/FH/X Linus Almstr¨ om Doc respons/Approved. Checked. EMW/FH/X Jonas Lundgren. Date. Rev. 2003-06-02. A. object information is sent instead, such as position, direction, speed, action, etc. Necessary graphics or other necessary data is locally stored at each computer which renders the graphics using the object information. This minimizes the amount of data sent over the network. Examples of games which implements this are FreeCraft[12] and Quake II[11]. Quake II uses a central game server. The server routes the messages from one client to all the others, if needed. FreeCraft, on the other hand, has chosen another way, which is peer-to-peer communication between the clients. The client then have to inform the others by itself, when needed. This approach renders to a higher use of bandwidth compared to the clientserver approach[12]. Although FreeCraft uses peer-to-peer methodology, the protocol it uses still manages to have a status update frequency of 6 times per second with a maximum use of 28.8 kb/s and having 8 simultaneous network players[12]. Quake II sends a frame11 to each client every tenth of a second.[11]. To reduce the usage of bandwidth Quake II only sends updates on objects within a visible range of the player. To further reduce the amount of data sent the server uses delta compression, which provides the functionality of only sending differences in frames based on the last frame the server knows the client received. The complexity between the two is concentrated differently. In FreeCraft each client needs to know of all the other clients and receives information from all of them. The server on the other hand does not do much at all. In Quake II the server is quite complex, since it has to know of all the clients, keep track of what information they need and keep track of the status of each client. The client on the other hand only has to render what the server tells it to render and send status updates to the server based on the player’s actions.. 2.4. MPEG-4. The media formats MPEG-1 and MPEG-2 were the first two generations of the open standard from MPEG12 . The MPEG-1 standard was introduced in 1992 and is used in VCD13 technology, which has been very popular in Asia. MPEG-1 is not very popular today, mainly for not being competitive in quality at any given bit rate, but the follower MPEG-2 is to date the most important of the MPEG formats. The latter is currently widely used in the industry, for example for DVD, digital cable and digital satellite systems, foremost for its enhanced quality at the same bit rate as its predecessor and various other improvements. These two are not very flexible formats and are mainly used for rectangular updates of the video being played. The creation of the successor, the MPEG-4 standard, was aimed at 11. Each frame holds the status of each object in the game Motion Picture Expert Group 13 Video Compact Disc 12. 2. Related Technologies. 7(41).

(13) Master Thesis Report Prepared(also subject responsible if other). No. EMW/FH/X Linus Almstr¨ om Doc respons/Approved. EMW/FH/X Jonas Lundgren. Checked. Date. Rev. 2003-06-02. A. eliminating the restrictions the preceding standards have.[26] MPEG-4 has stepped away from being an audio and video playback format only. Instead of just compressing video and audio, as the former MPEG standards, it defines media objects for the possibility of creating interactive content components. MPEG-4 also includes a server-client communication specification to enhance the possibilities for an interactive system. Because the MPEG-4 standard introduced the concept of using media objects in the MPEG standard, the use of still images has become useful. Still images can be used as background, for example, in media clips where it stays the same and the media objects that changes its position or its form are the only data needed to be transfered. To further lower the amount of data to be transfered, MPEG has chosen to use Wavelet compression for the still images. This methodology is very cost effective in systems where the background does not change much. Apart from the earlier MPEG standards, the MPEG-4 standard has introduced an interactive environment for the user. This means that each visual object on the client can act as a GUI14 object to further let the user get more information about that object. The information could be directly embedded in to the data stream, i.e. rich media[26], or events can be defined. When events are defined the client sends data back to inform the server that the user has performed an action of some kind. The server can then respond with an appropriate reply. The MPEG-4 standard is a huge standard. Too huge some say, because they believe it is impossible to implement all the features of the standard in to one product. Those who think so may have misunderstood an important part of MPEG-4. All of the features in the standard do not need to be implemented[26]. You can use the features you feel necessary for the specific task you aim the system to support. Even though this makes it easier to implement a MPEG-4 system, it is still complex. There are quite a few companies who provide MPEG-4 solutions[6]. Most of them are strict software solutions, while others have a more hardware oriented solution. None of them has fully implemented the MPEG-4 standard, but some of the solutions are still sophisticated. One company which deserves to be mentioned is Octaga15 , who has created a 3D based, interactive, multi-client MPEG-4 system.. 2.5. ULC. ULC is a system to ease the overall development of distributed graphics systems. At first it was developed and maintained by IBM, but is now developed and maintained by Canoo16 . ULC is a Java system for distributed 14. Graphical User Interface <http://www.octaga.com/> 16 <http://www.canoo.com> 15. 8(41). 2.5. ULC.

(14) Master Thesis Report Prepared(also subject responsible if other). No. EMW/FH/X Linus Almstr¨ om Doc respons/Approved. Checked. EMW/FH/X Jonas Lundgren. Date. Rev. 2003-06-02. A. graphics, which include a stateless client, which presents the data and forwards the user events to the server, and a server, which hold the state of the application. The basic idea of the system is that the developers only need to concentrate on developing their application using ULCs Swing17 inspired API18 , and need not to worry about distribution of the graphics. The developers need only develop a server side application, using faceless components, and each component in the API has a GUI counterpart for the actual representation of the data. This is naturally all hidden to the developers, thus the developers only need to worry about the server application.[5] For a distributed graphics system to fit in a broad variety of environments it needs to be flexible. It needs to be expandable as well as dynamic with regard to bandwidth usage and such. ULC provide a functionality of caching objects to reduce bandwidth, and it does also provide functionality for user events to be handled instantly at the client. The possibility of handling user events locally when possible also reduces bandwidth. The API has an extensive set of components ready to be used by the developers, but if those do not provide the needed functionality ULC can be extended with custom-built components. Although as soon as that is needed it quickly gets more complex, since both client level and server level development is needed. That is because both the faceless component on the server side and a GUI component to present the data on the client side have to be developed. However, this makes ULC very flexible, hence it is likely to fulfill the needs for the system being developed. What is pleasing with ULC is how easy it is to develop a distributed graphics system, due to the API and the flexibility. The use of the Java framework makes systems which uses ULC very portable, both the server and the client, and it also speeds up the overall development time.. 17 18. 2. Java framework for GUI development Application Programming Interface. Related Technologies. 9(41).

(15) Master Thesis Report Prepared(also subject responsible if other). No. EMW/FH/X Linus Almstr¨ om Doc respons/Approved. Checked. EMW/FH/X Jonas Lundgren. 3. Date. Rev. 2003-06-02. A. System Description This chapter will give a technical overview of the prototype system and the requirements for such a prototype system. It begins with a scenario which the design of the prototype system is based on.. 3.1. Scenario. Imagine a situation where every group of soldiers, or possibly even every soldier, has access to a device. This device is connected to a larger network which feeds it with information, and the group can then decide what to do based on that information. The larger network is composed by several parts, including map generators, sensors, etc, and the information is shared with everybody, thus a soldier can get information where allied troops are currently located and the like. The objects are presented differently on the device, depending on what type of object it is, for easier identification. For example would a car be identified with an icon with one shape and a boat with another shape, both of them moving on top of a map. It would also be possible to retrieve further information of the objects, such as speed, direction etc, when the user wants information of that kind. Since the system is interactive it could also be used as means of communication between troops and other involved parts. A soldier could use it to inform others if enemy troops have been spotted, as well as to call for assistance, set an alarm and other communication based events. The system could be used to define borders, which are not to be crossed by none other than allied objects, so called ‘Alarm Areas’. If others cross that border an alarm will go off and inform those parts in need of such information.. 3.2. Technical Overview. The prototype system consists of multiple clients and a server working together to be able to present as detailed information as possible. None of the clients share information directly with each other, instead client data are routed through the server. To each client the system provide object information if the client needs such information. An object can be other clients as well as various other kinds of objects, such as enemy troops, vehicles, buildings etc. 3.2.1. Server. In Figure 2 a part of a map from the server’s point of view is shown. It presents fundamental ideas of what the server must keep track of, such as the map itself, naturally, location of the clients and the areas presented on each client. The area which each client is currently monitoring is needed 10(41).

(16) Master Thesis Report Prepared(also subject responsible if other). No. EMW/FH/X Linus Almstr¨ om Doc respons/Approved. EMW/FH/X Jonas Lundgren. Checked. Date. Rev. 2003-06-02. A. Figure 2: Clients’ location and their viewing areas.. because the server has to know what kind of information the client needs, otherwise the server would send irrelevant data to every client. Knowing the area each client monitors also makes it possible for the server to send information updates on objects the client is currently viewing. For example would both Client 2 and Client 3 get updates on the black square object they share, since it is in the range of both viewing areas. Not only do Client 2 and Client 3 get updates on the object, but can also get further object information presented to them by the server if needed. Naturally this means that the server needs to be aware of all the objects presented, and not only the clients, since the image shown on the client is based on data from the server. 3.2.2. Client. The clients are more or less stateless except from the information regarding their own location. Each client is responsible for feeding the server with necessary information such as their location and what area they are currently monitoring to let the server know what kind of information the client needs. In Figure 3 an image of what can be presented on a client is shown. It is not distinctly shown in the image what each object exactly is, although different objects look different to some degree. If the user wish to get more information of an object it can be requested and the server updates what is presented with the new information. Objects within the area a client monitors will get updated by the server when needed, for example if objects moves. Clients need not to bother with filtering information since the server only feeds them with data which are 3. System Description. 11(41).

(17) Master Thesis Report Prepared(also subject responsible if other). No. EMW/FH/X Linus Almstr¨ om Doc respons/Approved. Checked. EMW/FH/X Jonas Lundgren. Date. Rev. 2003-06-02. A. Figure 3: The area Client 1 monitors. It is the same area as Client 1 monitors in Figure 2.. interesting to each particular client.. 3.3. System Design Considerations. Bandwidth usage is of interest, and the less that can be used the better. A way to restrict bandwidth usage is to have a two-way client-server communication. Only streaming data from the server to each client requires more bandwidth because the server must send data based on a predefined area size around the location of the client, which also makes the system less flexible from a client point of view. The client could not, for example, view an area outside the predefined area size, since the server only sends so much. Streaming data does not only require more bandwidth, but also makes the system less reliable. In a two-way communication the client can inform the server of its location, thus the system could use that information with radar systems, and other detecting services, to estimate the location of the client more reliably. Location estimation aside, streaming data also requires more resources out of the client, since the client has to filter the information and decide what to show. If the client wants to zoom in to an area, it has to do so by itself, and the stream must constantly send enough detailed information for the client to do so. The client can be of any type of device, a PDA, laptop, work station or the like, hence the client resources can vary from client to client. Some clients may be able to store data on disk and in memory and some may not. Those clients with more resources may then be able to cache data to utilize less bandwidth, while others may need to use as much bandwidth as possible because they lack the resources to cache as much data. The clients 12(41). 3.3. System Design Considerations.

(18) Master Thesis Report Prepared(also subject responsible if other). No. EMW/FH/X Linus Almstr¨ om Doc respons/Approved. Checked. EMW/FH/X Jonas Lundgren. Date. Rev. 2003-06-02. A. may also differ in CPU19 power, thus the system need to be designed in such a way which lets the client to be light-weight.. 3.4. Requirements. To be able to guarantee the level of functionality of the prototype system, a set of requirements has been established. These requirements set the minimum of what the prototype system should be capable of. • General system capabilities - Objects can move around. - Clients can change location. - The system must be able to differentiate between allied and enemy objects. - Updates must be time stamped. - Alarm if enemy crosses a defined border. • Server capabilities - Feeds the client with data to build the image to present. - All objects may be simulated. - Provide further information of objects when requested. - Provide data for possibility to differentiate between different kinds of objects (e.g. boat, car, tank). - Must be able to handle a minimum of 100 objects at the same time. • Client capabilities - Can view other areas besides the one where the client is located. - The client must work on several platforms. - It must be possible to zoom in and out. - Can retrieve further information of objects. - Has functionality to present different kinds of objects differently. - Must be able to present a minimum of 20 objects at the same time.. 19. 3. Central Processing Unit. System Description. 13(41).

(19) Master Thesis Report Prepared(also subject responsible if other). No. EMW/FH/X Linus Almstr¨ om Doc respons/Approved. Checked. EMW/FH/X Jonas Lundgren. 4. Date. Rev. 2003-06-02. A. Analysis of Related Technologies This chapter includes an analysis of the previously mentioned technologies with regard to usage of client resources, bandwidth usage and complexity of the system implementation. The chapter ends with a summary and a discussion of the results.. To get a better overview of the relationship between the technologies regarding their capabilities and in what way they affect the prototype system, they have been compared using a rating system. Each section begins with a table of the ratings and continues with a more detailed description for each technology. The rating grades the differences between the technologies. The scale ranges from 0 to 100, where 0 is the lowest rating and 100 is the highest possible rating. The scale is in this case arbitrary and serves only as an aid to compare the technologies. The higher the score, the more suitable for the target system.. 4.1. Client Resources. In the analysis of client resources, the number of elements studied has been limited to the client’s CPU, RAM20 and ROM21 usage. Table 1 gives the rating for each technology to present an overview of the result of the analysis with regard to client resource usage. Table 1: Rating for usage of client resources. Technology Games ULC X VNC MPEG-4. Rating 10 40 70 90 70. Games The traffic which FreeCraft and Quake II generates does not require much from the client CPU. However, the protocols are based on static graphic components on the client. These two games require the necessary data to render the graphics to be stored locally on each client when the game starts. Doing so on the prototype system described earlier would be virtually 20 21. Random Access Memory Read-Only Memory. 14(41).

(20) Master Thesis Report Prepared(also subject responsible if other). No. EMW/FH/X Linus Almstr¨ om Doc respons/Approved. EMW/FH/X Jonas Lundgren. Checked. Date. Rev. 2003-06-02. A. impossible due to the demands of low usage of client resources. The client would need to store the entire geographical map, which could be very large, hence the low rating.. ULC ULC itself is not especially demanding of resources. The UI engine22 is less than 300 kB in size[5], but a big disadvantage is that ULC currently requires J2SE23 . J2SE does not fit well in an embedded environment, mainly because its huge memory footprint of circa 8 MB RAM for a simple application[3], but also for the requirements of better CPU processing power due to the interpretation nature of Java compared to native software. A serious threat to overall performance in embedded environments is J2SEs automatic garbage collector. The garbage collecting thread cannot be interrupted until it is finished, and presents a risk of missing deadlines for important tasks in systems which require real-time demands. This makes J2SE unpredictable and unusable for real-time systems.[15] To address these problems with ULC, Canoo is looking into creating a ULC version for J2ME24 , according to Bruno Schäffer Chief Technology Officer at Canoo, which hopefully will make ULC more attractive for embedded environments.. X-Windows X-Windows is known to be a resource hungry environment. A typical Xserver running a minimal desktop needs circa 4.5 MB RAM and 16 MB in file size. There are however possibilities to reduce the resource usage of XWindows at the expense of flexibility. Such a system could be reduced to use as little as 900 kB of RAM and be as small as 600 kB in file size.[1] An example of such a X-Windows environment is Micro-X, developed by Metro Link Inc.25 X-Windows’ CPU usage is fairly low. A test with a video stream, in a 100 Mb/s network, displayed on the client consumed an average of 15-20% of the CPU. The test case used computers with a PII 450 MHz CPU.[27] With the use of XIE the memory footprint numbers would be higher, and X-Windows would also require more processing power. 22. The standard client software ULC provides with its standard set of components. Java 2 Standard Edition 24 Java 2 Micro Edition, a Java environment for embedded systems. 25 <http://www.metrolink.com> 23. 4. Analysis. 15(41).

(21) Master Thesis Report Prepared(also subject responsible if other). No. EMW/FH/X Linus Almstr¨ om Doc respons/Approved. EMW/FH/X Jonas Lundgren. Checked. Date. Rev. 2003-06-02. A. VNC VNC can be customized to be very lightweight for the client. The VNC client for OSs26 on workstations ranges from 70 kB to 150 kB.[23] There is however a project in the works for creating a very lightweight VNC client called microVNC27 , which currently has produced a working prototype. The prototype of microVNC has been created for an Atmel Atmega161 microcontroller with 16 kB program memory and 1 kB SRAM28 . It has some issues with frame-rate when larger parts of the frame is updated, but in general it has a pleasant frame-rate when small areas are updated.[2] However, this is one of the examples of VNC clients running on hardware with low resources. On high-bandwidth networks in an environment where large parts of the display is updated often, VNC has a high CPU usage. Compared to X-Windows in the same video stream test-case, VNCs CPU usage is higher with a magnitude of 2-4.[27] However, due to the polling nature of VNC the client can adjust to the environment at the cost of display latency or quality. The server can decide what is important to the client, once the client requests more data. This makes VNC very flexible in relation to the client resources.. MPEG-4 Evaluating MPEG-4 is a tricky business. MPEG-4 is a huge standard, which also lets products have subspaces of it implemented. Due to this, there exists a few products which differentiate in features and resource requirements. Octaga’s player, which supports interactive content, requires 64 MB RAM on a PII 450 MHz.[17] Envivio29 has a product called EnvivioTV, which also supports interactive content, and it requires 32 MB RAM on a PII 350 MHz or equivalent.[9] These two products demands quite a bit of the client’s resources, but there are players which do not. A study by ARM Ltd. shows that a video stream30 requires only 60 MHz and 21.5 kB RAM on an ARM926 CPU and the software reside in 107 kB ROM. When adding streamed sound it requires 80 MHz and 42.5 kB RAM and needs 130 kB ROM.[16] This implementation does however not support object based interactive content, it only handles streamed video. Object based interactive content support is more complex than video streaming and would most likely require higher utilization of hardware. How much it would actually require is difficult to say, if not impossible, without an actual implementation. 26. Operating Systems <http://www.embedded-creations.com/projects/microVNCoverview.htm> 28 Static Random Access Memory 29 <http://www.envivio.com> 30 15 frames per second with a size of 176x144 pixels for each frame 27. 16(41). 4.1. Client Resources.

(22) Master Thesis Report Prepared(also subject responsible if other). No. EMW/FH/X Linus Almstr¨ om Doc respons/Approved. Checked. EMW/FH/X Jonas Lundgren. 4.2. Date. Rev. 2003-06-02. A. Bandwidth Usage. The analysis of bandwidth usage examines how much network traffic the technology generates and how that traffic affects the system. Table 2 gives the rating for each technology to present an overview of the result of the analysis with regard to bandwidth usage. Table 2: Rating for usage of bandwidth. Technology Games ULC X VNC MPEG-4. Rating 90 90 10 30 75. Games FreeCraft is designed in such a way that it requires only a bandwidth of 28.8 kb/s. Yet, it updates each client 6 times a second, but it has a restriction of maximum 8 clients. Each client send update messages to all the others and receives update messages from all the others in the same time. It may seem efficient, but one have to remember the 8 clients restriction. The client-to-client type of communication requires higher bandwidth for the clients compared to server based routing of data,[12] and since each client continuously updates all the other clients as well as getting update messages from all of them, the bandwidth requirement has a linear relation to the number of clients. Contrary to FreeCraft Quake II has a server based communication. Each client sends events to the server and the server collects those events for forwarding to the rest of the clients. Every tenth of a second the server gathers the information from every entity of the game and send an update to each client. To reduce bandwidth usage only data relevant to the client is sent. The server only sends information of objects which the client can see and to further reduce network traffic delta compression is applied on each update message, that is, only what is changed from the last update message the client received is sent.[11] For Quake II there is no definite bandwidth requirement set. The bandwidth requirement for the server varies with the number of clients and the requirement for the client varies with the number of visible objects and how much the scene visibly changes. 4. Analysis. 17(41).

(23) Master Thesis Report Prepared(also subject responsible if other). No. EMW/FH/X Linus Almstr¨ om Doc respons/Approved. Checked. EMW/FH/X Jonas Lundgren. Date. Rev. 2003-06-02. A. ULC ULC is optimized for low bandwidth. It tries to minimize the network traffic with a few simple methods. The client only requests data from the server when it needs to display it, or if it needs to update the user interface. To further reduce network traffic the client can also cache data for later use, and to make the client-display smoother an option to pre-fetch data is also available. Another method ULC uses to decrease bandwidth usage is to handle some events locally when possible, i.e. the client will not send the event to the server. For example it can let the client validate input fields or trigger events to enable or disable GUI objects locally, thus ULC provide means to lower server requests.[5] Depending on the implementation of the system, a ULC solution may consume a very small amount of bandwidth. At an average it could be as low as 8 kb/s.[23] X-Windows X-Windows has a few techniques to lower bandwidth usage. The major one being the usage of identifiers as abstractions for windows, fonts etc. Whenever a request is done on an abstraction the identifier is sent instead of an entire structure, thus reducing network traffic.[18] Various extensions to X-Windows can help to make the usage of the network more efficient. For example can XIE, as stated earlier on page 5, compress images to become much smaller. Although X-Windows uses techniques to reduce the bandwidth usage, it still uses a considerable amount of bandwidth compared to other technologies. In a video streaming benchmark in a 100 Mb/s network, X-Windows transferred circa 70 MB of data for a 352x240 sized video with 24 frames per second during 34.75 s. The source of the video was a 5.1 MB MPEG movie. In networks with bandwidth constraints the data transfered is lower, but the quality of the video shown became worse. In a 128 kb/s bandwidth network the amount of data transfered for the same video stream was roughly only 5 MB, although it had an immense effect on the quality.[27] VNC The simplicity of the protocol VNC uses, the RFB protocol mentioned earlier, has benefits, but with regard to bandwidth usage it may seem to not be optimal. It is simple in the way that the only update instruction from the server is rectangular updates of the screen with raw pixel data and the idea of communicating with raw pixel data leads to thoughts of VNC utilizing more bandwidth than necessary. VNC offers a way to reduce bandwidth usage with encoding the raw pixel data. Custom encoding types can be added as well.[21] 18(41). 4.2. Bandwidth Usage.

(24) Master Thesis Report Prepared(also subject responsible if other). No. EMW/FH/X Linus Almstr¨ om Doc respons/Approved. Checked. EMW/FH/X Jonas Lundgren. Date. Rev. 2003-06-02. A. Since a VNC client requests to be updated it is adaptable to bandwidth resources too and not only client resources. When using VNC in an environment with low bandwidth resources either the output quality or the latency is affected. It is up to the server to choose what is most important, latency or quality. [21] In the same performance test as for X-Windows above, VNC performed poorly in high bandwidth networks while better in low bandwidth networks. For the same video stream as for X-Windows VNC transfered circa 10 MB of data in the 100 Mb/s network. This may seem to be a good result compared to X-Windows, but even though VNC only used about 3% of the bandwidth the quality of the output was not even half as good as for X-Windows. In 10 Mb/s networks, and slower networks, VNC outperforms X-Windows. VNC transfers significantly less data, but have either as good or better output quality than X-Windows.[27] As for ULC the system based on VNC decides how much bandwidth is used. In the same test when ULC 8 kb/s, VNC used in average about 30 kb/s bandwidth.[23] MPEG-4 MPEG-4 has been created to meet other requirements than MPEG-1 and MPEG-2. MPEG-1 was optimized for non-interlaced video at bit-rates of 1.2-1.5 Mb/s. MPEG-2 optimized for interlaced video at bit-rates 4-9 Mb/s.[19] MPEG-4, however, deals with video bit-rates ranging from 5 kb/s to 20 Mb/s.[22] The need of such low bit-rates is for possible integration of MPEG-4 products in mobile applications, and MPEG-4 provides much better quality in lower bit-rates.[26] It cannot be told often enough that MPEG-4 is complex and its complexity does also affect how MPEG-4 handles network communication. Due to the object-based nature of MPEG-4 and the two-way communication, the MPEG-4 standard defines functionality for rate control of the stream.[7] This means that the server can adjust the quality of the object streams to the capacity of the network and the client. Each object stream can have its own priority and the server can then decide what to do based on the priorities and the current situation. Those object streams with lower priorities, could be dropped at any given moment if needed, for example, instead of uniformly degrading the quality.[8][24]. 4.3. Implementation Complexity. In the analysis of implementation complexity, the implementation of the whole system is covered, i.e. both the implementation of the clients and the server. The analysis examines both the individual complexity of the client and server implementation, as well as what the causes are for extending the 4. Analysis. 19(41).

(25) Master Thesis Report Prepared(also subject responsible if other). No. EMW/FH/X Linus Almstr¨ om Doc respons/Approved. Checked. EMW/FH/X Jonas Lundgren. Date. Rev. 2003-06-02. A. technology for custom use. Table 3 gives the rating for each technology to present an overview of the result of the analysis with regard to implementation complexity. Table 3: Rating for implementation complexity of the system. Technology Games ULC X VNC MPEG-4. Rating 25 90 30 85 40. Games Neither of the software for FreeCraft or Quake II can be used except perhaps the algorithms they use. Looking at the solutions, on the other hand, the ideas used could be very useful in a new system, but none of the two solves every technical problem for the prototype system. This means that for possibly using any of the technology they use, one have to implement a completely new server as well as new clients. It will most likely be complex to do so, since one have to start from square one by designing a protocol too. ULC Due to the design of ULC, only server based development is needed in general. To only develop the server based functionality, and get the client interface for free, creates an environment for easy developing of remote applications. Furthermore does the possibility to extend the client capabilities and receive input from client hardware, not only user related events, make it possible to easily integrate the ULC technology with a wide range of products.[5] X-Windows Since the X protocol are based on drawing primitives, all the development for showing the graphics would be on the server side. An ordinary X-server could exist on the client and render the image presented on the client based on the instructions from the server. However, there is an obstacle to overcome. The fact that, in this case, the X-server needs to connect to the X-client31 and not vice versa as is the 31. remember that in X-Windows terminology the client is the X-server. 20(41). 4.3 Implementation Complexity.

(26) Master Thesis Report Prepared(also subject responsible if other). No. EMW/FH/X Linus Almstr¨ om Doc respons/Approved. EMW/FH/X Jonas Lundgren. Checked. Date. Rev. 2003-06-02. A. normal case. This adds the need to either modify current X-server solutions running on the client to connect to the server or extra software running on the client instructing the server where to connect. Either way it makes the system more complex. There is also a possibility for other modifications of the X-server to better fit in to the system. The X environment provides functionality to add extension to the X-server (for example for input devices, GPS32 etc.). There exists various extensions already, such as XIE, but unfortunately they are not trivial to implement comparing to, for example, extending ULC. VNC Developing a system using VNC is not overly complex, since only server based development is needed. Unlike ULC, however, the transport of the graphic is primitive, thus there is no API for widgets or graphic components. The design of the server implementation has to consider what part of the client screen needs to be updated and what has the highest priority, quality of the output or latency. Since the protocol is simple, it is pretty easy to extend the protocol for custom use. However, with custom protocol extensions an ordinary VNC client will not be able to use the extensions, thus a new VNC client need to be implemented. Implementing a client is however simple due to the nature of the protocol.[20] Naturally the server needs to support the additions to the protocol too. An example of a project which have extended the RFB protocol is tightVNC33 . MPEG-4 Comparing to MPEG-4’s former video streaming relatives (MPEG-1 and MPEG-2) it is more complex, both on the client side and on the server side.[13][8] Due to the new features and functionality which MPEG-4 offers, the client has become more complex. In addition to decoding the object streams and displaying the objects, it also has to compose the scenes. This additional complexity added to the client makes its performance dependent on the complexity of the content. For reliable content presentation, intelligent resource management, such as memory usage, is required.[8] The object-based methodology of MPEG-4 introduces additional complexity to the server part as well. The object-streams need to be scheduled efficiently when they get multiplexed to the stream. It is not always an easy task to perform, because quite a few issues need to be considered. The scheduler has to make sure the object arrives at the client before its decoding time 32 33. 4. Global Positioning System <http://www.tightvnc.com>. Analysis. 21(41).

(27) Master Thesis Report Prepared(also subject responsible if other). No. EMW/FH/X Linus Almstr¨ om Doc respons/Approved. Checked. EMW/FH/X Jonas Lundgren. Date. Rev. 2003-06-02. A. and therefore has to consider client-buffer underflow, client-buffer overflow, network fluctuations, object-stream priorities etc. to name a few.[8][14] At the moment there seems to be no available products which provide the functionality needed for the prototype system. Implementing both serverside functionality as well as client-side functionality is a complex operation. Although the MPEG-4 standard is created partly for integration with the mobile industry[19], it seems to mainly be for simple video streaming. More advanced interactive content with client-server communication could likely be too advanced to incorporate in small devices with low hardware resources.. 4.4. Summary and Discussion. Table 4 gives a good overview of the technologies’ rating for each category. The Total column in Table 4 gives an idea of which of the technologies that is most suitable for the target system. Table 4: Summary of the ratings for the different technologies and a column showing the total rating. Technology Games ULC X VNC MPEG-4. Resources 10 40 70 90 70. Bandwidth 90 90 10 30 75. Complexity 25 90 30 85 40. Total 125 220 110 205 185. In the analysis none of the technologies covered have proved to be the ultimate choice. In Table 4 one can see that ULC and VNC finishes on top. Both of them have their weaknesses, but for ULC the future looks a bit more promising. ULCs drawback is the use of Java, although SUN is trying to make Java more attractive for embedded environments, and hand-held devices are getting more powerful by the day. At the same time Canoo is already looking in to the possibility for a ULC version for J2ME, which is better suited for embedded environments. VNCs weakness is the bandwidth usage. Although technologies have emerged in recent years to improve the bandwidth for radio based communication, it is still important to try to keep the bandwidth usage at a minimum. MPEG-4 is a technology with interesting features, but the standard is clearly complex. The complexity is MPEG-4’s weakest point. It is a huge standard and includes a lot of good things but also things that are of no interest for the proposed system. Because of the amount of features the standard embraces, it is designed to be flexible, i.e. not everything has to be supported in an MPEG-4 product. This is good and bad, since if you 22(41). 4.4. Summary and Discussion.

(28) Master Thesis Report Prepared(also subject responsible if other). No. EMW/FH/X Linus Almstr¨ om Doc respons/Approved. EMW/FH/X Jonas Lundgren. Checked. Date. Rev. 2003-06-02. A. implement a MPEG-4 server you have to pick your clients with care and it might happen that you don’t find a suitable client for a specific platform. Creating your own client for all the platforms you intend to support might be a too time consuming task. X-Windows has a few weaknesses and bandwidth usage is the most significant one. X-Windows has shown during the analysis to not offer much for this particular purpose. It is good at what it does, i.e. as a remote desktop on high bandwidth networks, but is not suitable for this system. The ideas from the way Quake II and FreeCraft solves the network communication can be used when designing a completely new protocol for the system. It is very interesting to investigate them further if there is interest in creating a system which uses bandwidth efficiently.. 4. Analysis. 23(41).

(29) Master Thesis Report Prepared(also subject responsible if other). No. EMW/FH/X Linus Almstr¨ om Doc respons/Approved. EMW/FH/X Jonas Lundgren. 5. Checked. Date. Rev. 2003-06-02. A. Prototype System This chapter describes the design decisions, the architecture and the implementation of the prototype system.. 5.1. Design Decisions. The first decision regarding the prototype system was if one of the technologies could be used or if there were reasons to start from the beginning with designing a protocol to use. Starting from the beginning had indeed been interesting, but considering the time frame it would have been too difficult, especially to create a system with clients for several platforms. The only reasonable thing to do was to implement a prototype based on one of the technologies discussed earlier. VNC was decided to be used mainly because it is certain that you can find a VNC client for more or less any platform. The clients are lightweight and the behavior of each client is well defined because of the simplicity of the protocol. It was also interesting to investigate just how much bandwidth VNC requires in this kind of environment, and what the effects are if we have a low-bandwidth network. The reason for not developing a prototype based on ULC is because of the uncertainty regarding J2SE and embedded environments. Regarding MPEG-4 it was considered too complex, but also for MPEG-4 it is uncertain whether MPEG-4 players for embedded environments support the features required for this kind of system. Although VNC is suitable for this kind of system it contains something that could in our case be considered a flaw. The RFB specification lets the server decide the size of the VNC screen to be shown on the client, and there is no way of knowing the preferred screen size of the client. Once the VNC screen size is set, it is fixed to that value. Using one size for all clients is not recommended, since if we choose a size which is convenient for a stationary computer it would be inconvenient to use the PDA and vice versa. The solution chosen for the prototype is to have several connection points, where each has a different predefined frame size, thus PDAs connects to, for example, the first connection point, laptops to the second, etc. Figure 4 visualizes the example, but does also show that there could be as many connection points as desired. Slow communication links brings us to another design decision. How should the server act when the communication link is too slow? It could either degrade the quality, which would in this case mean that the map is not updated accordingly and will be less accurate, or the rate at which the client is updated could be lowered. The latter was chosen for the prototype, since it is important that the data is accurate. If the information on the screen is not accurate it could easily confuse the user. Designing the user interface is always something that needs to be done 24(41).

(30) Master Thesis Report Prepared(also subject responsible if other). No. EMW/FH/X Linus Almstr¨ om Doc respons/Approved. EMW/FH/X Jonas Lundgren. Checked. Date. Rev. 2003-06-02. A. Figure 4: Shows what different connection points do.. with care and designing an interface for several platforms even more so. For this prototype the interface were designed for the lowest common denominator, i.e. in this case a PDA, with other platforms in mind. A PDA generally does not have functionality to conveniently simulate a second mouse button being pressed. Such a simple thing as using pop-up menus when clicking on an object might be annoying, because it is difficult to decide when to show the menu. If there is a predefined delay, it is difficult, if not impossible, to find a delay which suites everyone, and if there is no delay it is likely to be annoying that the menu always shows when you click on an object. There is however another reason to not use a menu in this case, namely because of bandwidth usage. Since VNC clients are stateless, one will have to re-send the data to redraw the background if needed. Such a thing would be necessary when using menus. First when showing the menu, all the necessary data have to be sent to show the menu, and when the menu is closed all the data to redraw the screen again need to be sent. It will use a fair amount of bandwidth each time, depending on the size of the menu. There is, however, yet another reason to not use a menu. It gets slightly more difficult to implement since one would have to make sure none of the objects in the area of the menu gets redrawn, because they would otherwise get redrawn on the menu. The other option is to use a status-bar. A status-bar does not suffer from any of the above mentioned obstacles for a menu. It does however have another unwanted side effect, which is that the space available for the map would be less. That one effect is less important than the drawbacks for a menu system, though. However, using the status-bar does not exclude the usage of a menu. It could of course have a menu button as well, but will suffer from the same consequences as mentioned earlier. The status-bar could have a limited amount of information about a selected object and a few buttons performing zooming operations and the like. The fact that space for information is limited in a status-bar makes it only valuable to some degree, but for the prototype it is enough. When designing the use of alarm-zones, there is simply one thing that needs to be considered. Each alarm-zone need to be an area surrounded with borders of a closed shape. The shape can be arbitrary as long as there is no 5. Prototype System. 25(41).

(31) Master Thesis Report Prepared(also subject responsible if other). No. EMW/FH/X Linus Almstr¨ om Doc respons/Approved. Checked. EMW/FH/X Jonas Lundgren. Date. Rev. 2003-06-02. A. Figure 5: The left shows an open shape, which will not work as an alarm zone, and the right shows a closed shape as an alarm zone.. possibility for objects to cross the borders or enter the alarm-zone without being detected, because if that was possible it would defeat the purpose of alarm-zones. Having a mere line as border to guard an area is not enough, since objects can move around the line thus entering the area without we knowing it (see Figure 5).. 5.2. Architecture. The architecture of the prototype is object oriented. A few classes defined for the prototype inherits some of its functionality from general interfaces, to be able to treat different classes the same through these interfaces. Despite that the prototype is only to support the RFB protocol, it does not hinder a more flexible solution. Each protocol implements a general protocol interface. The protocol interface creates a possibility for easier additions of other protocols, thus the prototype could be used for testing different protocols. As for the protocol interface, the same goes for objects presented on the screen. Each type of object implements a general object interface, thus the prototype treats every object the same, whether it is an object with predefined movement, randomized movement or movements received from external sources, etc.. Figure 6: General interface.. Figure 6 is valid for both the protocol interface and for the object interface. They work theoretically the same way, but provides different function26(41). 5.2. Architecture.

(32) Master Thesis Report Prepared(also subject responsible if other). No. EMW/FH/X Linus Almstr¨ om Doc respons/Approved. EMW/FH/X Jonas Lundgren. Checked. Date. Rev. 2003-06-02. A. ality to the implementations and are treated differently internally. The prototype was designed with multi-threading in mind. There are two main reasons for a multi-threaded server. Firstly you do not want clients with different conditions to affect one another with respect to performance and responsiveness and the author finds it less complex to guarantee that in a multi-threaded environment. Secondly the server will perform better on SMP34 systems. Each client that connects to the server runs in its own thread and that thread does all of the network communication, thus if a connection to a client is slower and gets blocked longer than the other threads, they will not be affected.. 5.3. Implementation. The server was implemented in C/C++. The development of the prototype has all-in-all been fairly straight forward. The prototype has a minimum of implemented features. It has one protocol interface implementation, i.e. RFB, and one object interface implementation. The object interface implementation is an object that reads its detailed information and its movements from a configuration file, thus each object needs such a file. Even the client objects are simulated, because no system to retrieve client locations have been implemented. The current implementation of the layer with the rest of the server is currently implemented assuming an RFB protocol, thus adding a protocol implementation may or may not be awkward, depending on the protocol. In retrospective, greater care should have been taken when designing the rest of the components, for them to not be designed with respect to RFB. Implementing the RFB protocol was more work than expected. However, the implementation cannot be considered to be complex. The difficulties with using RFB is only that the protocol is too simple. You get nothing for free. Even the most basic GUI widgets, for example a button, need to be implemented. Drawing text on the client would in a real implementation need a font rendering engine, but in the prototype implementation raw bitmaps of characters were used to accomplish that. Except for the assumed use of RFB in some components of the prototype, not much have been assumed or hard-coded. Most of the behavior of the server is defined in configuration files, such as the map, number and movements of objects, etc. This is very helpful when testing the prototype, for example for benchmarking purposes. The background map is not rendered from map information, but is rather an image of a map. The objects move around on the map and can be selected. When a selected object moves out of the visible area, the background map will be updated to follow the object. The map area will be redrawn when 34. 5. Symmetric Multi-Processing. Prototype System. 27(41).

(33) Master Thesis Report Prepared(also subject responsible if other). No. EMW/FH/X Linus Almstr¨ om Doc respons/Approved. EMW/FH/X Jonas Lundgren. Checked. Date. Rev. 2003-06-02. A. Figure 7: Shows two views from what two clients shows at the same time.. zoomed and the objects within the area will be redrawn. The status-bar has only a few features implemented. It will show basic information of a selected object, location, type and such, and has buttons to perform actions. The buttons are zoom in, zoom out, follow toggling and a fourth button associated with alarm-zone activities (see Figure 7). To toggle follow means that the selected object will be highlighted and followed when moved around and information about the object is shown in the status-bar. The fourth button informs the user that an object that is not supposed to be in an alarm zone is moving inside an alarm-zone. It changes color when such an activity is taking place and selects the target object when invoked. The selected object is then followed, until de-selected. Figure 7 visually shows the functionality in the prototype described in chapter 3.1. The left and right images represents the views of two clients at the same time. The two clients shares three objects in their views: the diamond shaped object in the top left in the right view, the triangle shaped object in the middle of the left view and the second client (marked as a diamond in the left view and as a cross in the right view). The object marked with a cross in each view represents that the object is its own client object. Figure 7 relates very well to Figure 2 on page 11 and is a practical example of the same behavior. The implementation of alarm zones have been limited to only support rectangular alarm zones. The number of zones are defined in configuration files, as well as their size and location. When an object moves within such an area that is not supposed to be there, an alarm is set, i.e. detections of 28(41). 5.3 Implementation.

(34) Master Thesis Report Prepared(also subject responsible if other). No. EMW/FH/X Linus Almstr¨ om Doc respons/Approved. EMW/FH/X Jonas Lundgren. Checked. Date. Rev. 2003-06-02. A. objects crossing borders are not done, movements within areas are.. 5. Prototype System. 29(41).

(35) Master Thesis Report Prepared(also subject responsible if other). No. EMW/FH/X Linus Almstr¨ om Doc respons/Approved. EMW/FH/X Jonas Lundgren. 6. Checked. Date. Rev. 2003-06-02. A. Results This chapter highlights the results of the prototype system.. Overall the prototype system works quite well, apart from that it will suffer from latencies on low bandwidth networks, and fulfills the requirements given in chapter 3.4. Below follows results with respect to the topics in chapter 4. Client Resources The prototype has not included any client-wise implementation, so there is not much that needs to be said that has not already been told in Section 4.1. In a situation with 600 objects presented on the same screen, with a size of 900x756 and with a color depth of 32 bits, on computer with a PII 500 processor running Red Hat Linux, the viewer used up to 6% of the CPU. Each of the objects were updated once a second. The client used 1536 kB of memory in that particular situation. Bandwidth Usage VNC itself does not require a specific bandwidth, but latency for the graphics and input is affected if it does not have enough bandwidth. To test how bandwidth limitations affects the prototype, a bandwidth analyzer application was created. The application behaves like a proxy which can restrict the bandwidth between the client and the server. Figure 8 and 9 shows a summary of the results, for further details of the test see Appendix B. In both of the figures the map latency charts measures the latency for the map to be displayed on the client screen, the map and objects latency measures the additional objects and the map to be displayed. The objects latency is the difference between the two preceding charts, to get a measure of the latency to display only the objects. Figure 8 shows that for a 900x756 resolution the prototype system needs a lot of bandwidth, because the latency results stabilizes at a pretty high bandwidth. Naturally the latency is lower for the 8 bit color depth than for the 32 bit color depth and stabilizes much earlier. However, the latencies, after the 32 and 8 bit color depth results have stabilized, differs very little or hardly differs at all. Figure 9 shows that a 227x255 resolution does not require a lot of bandwidth, because the latencies stabilizes pretty early. Regarding 32 bit color depth versus 8 bit color depth, the same as for 900x756 resolution goes for this resolution too. Comparing Figure 8 and 9 is not completely fair. Only the charts for map latencies can be compared, because in the charts which involves objects are based on two different amounts of objects. The 900x756 resolution were 30(41).

(36) Master Thesis Report Prepared(also subject responsible if other). No. EMW/FH/X Linus Almstr¨ om Doc respons/Approved. EMW/FH/X Jonas Lundgren. Checked. Date. Rev. 2003-06-02. A. Figure 8: The latency graphs for a screen with the dimensions of 900x756. The gray columns are average test results for 32 bit color depth, and the black for 8 bit color depth.. Figure 9: The latency graphs for a screen with the dimensions of 227x255. The gray columns are average test results for 32 bit color depth, and the black for 8 bit color depth.. 6. Results. 31(41).

(37) Master Thesis Report Prepared(also subject responsible if other). No. EMW/FH/X Linus Almstr¨ om Doc respons/Approved. EMW/FH/X Jonas Lundgren. Checked. Date. Rev. 2003-06-02. A. using 600 objects in the test and the 227x255 resolution were using 240 objects. Although, Figure 9 shows that the latencies are significantly lower for that resolution than for the resolution in Figure 8. The tests were not performed in an environment dedicated to these tests and lots of factors could thus have influenced the tests, such as sharing bandwidth and sharing hardware resources. Another factor to consider is that the bandwidth analyzer application may contain bugs which can influence the tests in a negative way. One such occasion when the bandwidth analyzer can have influenced the tests is for the tests with 4 kb/s bandwidth limitations, which seems to deviate from the other tests. Implementation Complexity As expected the server was not overly complex to implement, but it was however more work than expected, as briefly explained previously. For further details on the subject see the Section 4.3, since the explanation given in that section applies to the prototype as well. The prototype confirms the expectations of the simplicity to support more or less every platform. Going from the test environment, where the server and client ran on a computer with Solaris, to a client running on Redhat Linux and then Windows XP and finally to a client running on an iPAQ with Windows CE was very smooth.. Additional Results The prototype system was also tested on an iPAQ running Windows CE. The iPAQ was connected to the network through wireless LAN35 . The test ran four different scenarios, which differed in the number of moving objects. It was tested with 25, 50, 100 and 200 objects, where each object moved once a second. In each scenario all objects were updated at the same time. The result of the test is that during the update of the objects all of the processing power of the iPAQ were used. The response time for normal user operations such as opening a menu, running another application, etc, increased significantly and normally did not respond until the update were completely finished. For 25 objects the response time for normal operations were not affected that much, it was noticeably higher for 50 objects and was starting to be in the way. For 100 and 200 objects the iPAQ became unusable, because of long response times for every user operation. It was unexpected that the iPAQ unit got as affected as it did, but other factors apart from processing power could be of interest. Foremost could the prototype system possibly not use VNC technology efficiently and other less likely factors which could be of influence could be the VNC client implementation, the underlying OS, the wireless LAN, etc. 35. Local Area Network. 32(41).

(38) Master Thesis Report Prepared(also subject responsible if other). No. EMW/FH/X Linus Almstr¨ om Doc respons/Approved. EMW/FH/X Jonas Lundgren. Checked. Date. Rev. 2003-06-02. A. In a situation where the client barely had a connection to the access point in the wireless LAN the latency got worse. At times the VNC application on the client could completely freeze and also the rest of the client, but eventually the VNC application closed itself and the system got back to normal. The prototype was meant to be generic - although that was not part of the plan from the start, but was more something that came to be a natural thing to do - and was designed with that in mind. However, the design allowed for a few mistakes and later the implementation enhanced those mistakes to let the prototype not to be as generic as it was meant to be. The server got implemented with paying too much respect to the RFB protocol, since that was supposed to be the only supported protocol, but big parts of the server can still be used if it is decided to support yet another protocol.. 6. Results. 33(41).

No results found