IT 12 015
Examensarbete 30 hp May 2012
Delay-tolerant data delivery in a distributed sensor network testbed
Wenbin Wu
Teknisk- naturvetenskaplig fakultet UTH-enheten
Besöksadress:
Ångströmlaboratoriet Lägerhyddsvägen 1 Hus 4, Plan 0
Postadress:
Box 536 751 21 Uppsala
Telefon:
018 – 471 30 03
Telefax:
018 – 471 30 00
Hemsida:
http://www.teknat.uu.se/student
Abstract
Delay-tolerant data delivery in a distributed sensor network testbed
Wenbin Wu
Sensei-UU testbed is an inexpensive, expandable, relocatable testbed, which is designed by CoRe group in Uppsala University. It provides wireless sensor network (WSN) experiments with repeatable mobility. However, end nodes are always deployed in extreme environments where connectivity is not able to be guaranteed and Sensei-UU testbed lacks the ability to deal with this case. Reliable data delivery, therefore, is an important feature of delay tolerant network and it will extend the usage of Sensei-UU testbed. This thesis work investigates Sensei-UU testbed and improves it to support delay-tolerant data delivery.
The thesis report starts with an introduction of related background concepts regarding wireless sensor network, Sensei-UU testbed and delay tolerant network.
Then a detailed technical solution of delivering delay-tolerant data in Sensei-UU testbed is presented in terms of system analysis, design and implementation.
Handledare: Lars-Åke Larzon
TABLE OF CONTENTS
List of Figures . . . vii
List of Tables . . . . ix
Chapter 1 Introduction . . . . 1
1.1 Motivation . . . . 2
1.2 Requirements and Goals . . . . 2
1.3 Report structure . . . . 3
Chapter 2 Background . . . . 4
2.1 Delay tolerant network . . . . 4
2.2 Sensie-UU Testbed . . . . 5
2.2.1 Testbed . . . . 5
2.2.2 Drawbacks in Sensei-UU testbed . . . . 6
Chapter 3 System Analysis . . . . 7
3.1 Sensor node, Vclient and Sensor Host . . . . 7
3.2 Site manager . . . . 8
3.3 Monitor . . . 10
3.4 Overall . . . 10
Chapter 4 System Design . . . 12
4.1 Cellphone . . . 12
4.2 Messages . . . 13
4.3 Database . . . 15
4.3.1 Database selection . . . 15
4.3.2 Database design . . . 16
4.4 Site manager . . . 16
4.4.1 Site manager in sensor host and computer . . . 17
4.4.2 Site manager in cellphone . . . 19
Chapter 5 System Implementation . . . 21
5.1 Overview . . . 21
5.2 DTNMessage . . . 21
5.3 DTNDatabase . . . 22
5.4 DTNDevice . . . 22
5.5 DTNMobileSiteManager . . . 22
5.6 DTNSiteManager . . . 23
5.7 DTNConnection . . . 23
5.7.1 Establish connection . . . 23
5.7.2 Strategy of gaining data from database . . . 24
5.7.3 Threads in DTNConnection . . . 25
Chapter 6 Evaluation . . . 26
6.1 Experiment setup . . . 26
6.2 Experiment . . . 27
6.3 Storage and memory use . . . 27
6.4 Overhead . . . 28
6.4.1 Message overhead . . . 28
6.4.2 Session overhead . . . 29
6.5 Measurement of sending time . . . 29
Chapter 7 Future Work . . . 31
7.1 Protocol . . . 31
7.2 Security and compression . . . 31
7.3 Multiple connections . . . 32
7.4 Opportunistic networks . . . 32
Chapter 8 Conclusions . . . 33
References . . . 34
Appendix . . . 35
Appendix A Code implementation . . . 36
A.1 DTNDevice . . . 36
A.1.1 Constructor . . . 36
A.1.2 Thread . . . 37
A.1.3 Broadcast and its handler . . . 39
A.1.4 Handler for synchronization . . . 39
A.2 DTNSiteManager . . . 39
A.2.1 Handler for Vclient port . . . 39
A.2.2 Handler for monitor port . . . 39
A.2.3 Handler for monitors and Vclient sockets . . . 40
A.2.4 Auto-connect . . . 40
A.3 DTNDatabase . . . 40
LIST OF FIGURES
Figure 2.1 Demonstration of store-and-forward technique . . . . 5
Figure 2.2 General architecture of Sensei-UU testbed . . . . 6
Figure 3.1 Activity graph of site manager . . . . 9
Figure 4.1 Cellphone in Sensei-UU testbed . . . 13
Figure 4.2 Overview of improved Sensei-UU testbed . . . 17
Figure 4.3 Structure of site manager . . . 18
Figure 4.4 Structure of mobile site manager . . . 19
Figure 5.1 Class graph . . . 21
Figure 5.2 Preparation before synchronization . . . 24
Figure 6.1 Test network . . . 26
Figure 6.2 Measurement of storage . . . 28
Figure 6.3 Measurement of sending time . . . 30
LIST OF TABLES
Table 4.1 Message types in the testbed . . . 14 Table 4.2 Database tables . . . 16
Chapter 1
Introduction
Wireless sensor network (WSN) has been identified as one of the most promising technologies for the 21st century. WSN is used for many different applications in various environments and a testbed is used to simulate and emulate for evaluation of WSN applications. Sensei-UU testbed [1] is one of many testbed designed to be easily relocatable among different physical environ- ments and not tightly dependent on specific sensor hardware or OS [2]. Sensei-UU testbed is developed within the CoRe group, Uppsala University. Users are able to evaluate wireless sensor network applications in their own environments due to the relocatability of the testbed.
Moreover, Sensei-UU testbed supports a wide range of sensor node platforms, which enables users to evaluate different kinds of applications easily.
Delay tolerant network (DTN) is a concept that was first discussed in 2003 [3]. It makes use of store-and-forward techniques within the network to compensate the unstable connectivity.
The core concept of delay tolerant network is the Internet-independent middle-ware where the protocols used are applicable to different environments. Reliable message delivery is an impor- tant feature of the delay tolerant network, which can be used by applications in a connection unstable network.
Sensei-UU testbed can be only used in a stable environment where connections are available all the time. Otherwise, data will be lost. By applying store-and-forward techniques, Sensei- UU testbed will be suitable for worse environment. The aim of the study is to analyze the structure of Sensei-UU testbed and improve it in order to make it deliver delay-tolerant data.
This chapter has a brief introduction on the topic, motivation, requirements, and goals of this thesis.
1.1 Motivation
Most researchers conduct experiments to collect data for analysis. Because of the technical advances in computer science, the process of collecting data is accelerated. Besides, new tech- nique increases the amount of collected data as well as its accuracy. The vast majority of data is collected automatically and stored digitally[4].
In spite of the rapid growth of new technique, there is still space for more improvements.
One example comes from environmental research. Because of the particular nature in environ- mental research, researchers need to set up all kinds of equipment, such as sensors and cameras, in the wild. All equipment writes collected data on loggers and waits for researchers coming and collecting these loggers. However, the task to gather these loggers is not convenient, even very difficult in extreme environment. The challenges are complex in this scenario. It is dangerous for researchers to get to some place because of weather conditions or geographical conditions.
Additionally, there is always no infrastructure like networks and power stations in that area.
Both make the task hard to proceed. In order to meet these challenges, WSN is often used to set up connections. Moreover, DTN could be used to guarantee the connections between WSNs and the Internet.
Sensei-UU testbed relies on the availability of transmission. If the connection is lost, the network will not be able to work as normal, or even lead to a system crash. Sometimes there may be no end-to-end potential path between nodes in a network. DTN is able to work properly in a no connectivity assurance network. Considering the scenario above, Sensei-UU can benefit from DTN which grants the ability of working in a disconnected network. Therefore, Sensei-UU testbed can be used in more complex environments.
1.2 Requirements and Goals
The idea of this thesis comes from Professor Lars-˚Ake Larzon. The aim of this project is to deliver delay-tolerant data based on existing Sensei-UU testbed in order to expand the usage the testbed. Eventually, Sensei-UU testbed will be easy to deploy, and has a wider range of uses.
New testbed should satisfy the following requirements:
• All nodes should be able to deliver delay-tolerant data.
• Programs running in wireless nodes should be as simple as they could be because of energy consuming.
• User should be able to expand testbed easily to handle different kinds of data in the future.
1.3 Report structure
This thesis is organized as follows:
• Chapter 1 gives the introduction about the general description of motivation and require- ments of the thesis.
• Chapter 2 introduces a set of background knowledge of delay tolerant networks and Sensei- UU testbed.
• Chapter 3 analyzes the structure of Sensei-UU testbed.
• Chapter 4 discusses the design of new testbed.
• Chapter 5 describes implementation decisions.
• Chapter 6 gives the evaluation results.
• Chapter 7 looks into the future and describes possible future work.
• Chapter 8 concludes the thesis.
Chapter 2
Background
2.1 Delay tolerant network
There is no guarantee of available connectivity between nodes in delay tolerant network (DTN).
DTN is designed to operate effectively over extreme distances such as those encountered in space communications or on an interplanetary scale [5]. Sometimes, end-to-end connectivity between two endpoints cannot be provided and nodes in WSN may have long time of disconnections.
In this case, the latency to send a message is measured by hours or even days rather than mil- liseconds. DTN can also be used in a sensor network where sensor nodes are able to transmit data infrequently because they have limited power. Both situations can make use of DTN that sends data when connectivity is available.
As an ‘overlay’ architecture, DTN is intended to operate above the existing protocol stacks in various network architectures and provide a store-and-forward gateway function between them when a node physically touches two or more dissimilar networks [3].
Figure 2.1 shows a brief explain of store-and-forward technique. If node A wants to send one message to node D, it can send this message to node B first. Node B stores this message and tries to forward it to node C. Node C does the same work as node B. At last, node D will get this message from node C.
There are important distinctions between delay tolerant network and disruption tolerant network though both refer to some of the same technologies. Hardware is required to store large amounts of data in a delay-tolerant network. Such hardware must be able to survive extended power loss and system restarts. And it must also be accessible immediately at any time [5]. Hard disks and high-volume flash memory are ideal devices for this purpose.
Figure 2.1: Demonstration of store-and-forward technique
2.2 Sensie-UU Testbed
2.2.1 Testbed
Sensei-UU testbed is a flexible, portable testbed, which can operate using own hardware, emu- lab nodes, planet-lab nodes or a combination of them all. The design objectives are to provide wireless sensor network (WSN) experiments with repeatable mobility and to be able to use the same testbed at different locations, including the target location. The testbed is inexpensive, expandable, and relocatable and it is possible to reproduce it by other researchers. [6]
Sensei-UU follows the general architecture as depicted in Figure 2.2. Sensor hosts are used to observe and control the operations of sensor nodes. They are typically small Linux ma- chines. There are programs, which are called Vclients running in sensor hosts. Sensor nodes are normally attached to sensor hosts via their USB interfaces. Vclients receive data from sen- sor nodes and send to site manager. Site manager is the gateway to testbed and sensor nodes.
It communicates with sensor hosts over a control channel. Information about positions of the sensor nodes are tracked by sensor hosts and reported to site manager. All sensor nodes and their sensor hosts are designed as being mobile, even though most of them do not move during an experiment. In the testbed, only one site manager exists.
In addition, Sensei-UU is designed for heterogeneous sensor nodes, expandability and for relocation to diverse environments. Other important designs objectives are low cost and re- producibility of the testbed at other institutions. To achieve the relocation and expandability features, the interconnecting network is based on IEEE 802.11b/g. It gives flexibility to deploy nodes anywhere and an unlimited number of sensors as long as there is WLAN coverage and capacity. Avoiding a wired network simplifies relocation of the testbed - setting up the testbed in a new location mainly involves ensuring that all nodes know their current position. Sensei-
Figure 2.2: General architecture of Sensei-UU testbed
UU testbed lets every sensor host keep track of its own position and report its coordinates to the site manager. Hence Sensei-UU is not tied to one specific external position system which makes it easy to use different positioning techniques, including GPS.
2.2.2 Drawbacks in Sensei-UU testbed
Apparently, Sensei-UU testbed can only be used in a convenient environment; otherwise, it cannot work properly and lose data. Based on the concept of DTN, it makes use of store-and- forward techniques within the network to compensate the unstable connectivity. In order to make Sensei-UU support DTN, one storage solution must be made. In every node of Sensei-UU testbed, data should be able to be kept in a place, so that it would not be lost in case of the unavailability of connections. At the same time, Sensei-UU testbed must be capable of dealing with connectivity problems. Besides, once one node is not available for a long time, there should be a way to get data from that node. In a word, the structure of connection part needs to be redesigned.
Chapter 3
System Analysis
The previous chapter describes background of this thesis project. And the architecture seen in figure 2.2 from previous chapter shows the general structure of Sensei-UU testbed. The archi- tecture of Sensei-UU testbed is flexible overall. Sensors are attached to distributed machines which are called sensor hosts. Data from sensor nodes can be sent to site manager by different methods. For example, if one sensor host, where one sensor node is connecting, is geographically near to a site manager, they can exchange data by using Ethernet. But if they are far from each other, WIFI or other long distance transmission method can be adopted. This follows dis- tributed design. For this reason Sensei-UU testbed has almost none geographically limitation.
But it causes data lost when connection is unavailable. So a major work to do in this thesis to handle unstable connectivity. Now we will go into more detailed about the composition of current testbed and elaborate on the problems we meet in different parts of Sensei-UU testbed.
3.1 Sensor node, Vclient and Sensor Host
Sensor nodes gather sensory information, perform some processing work and send it to desig- nated address. In this thesis, I use sensor nodes which can get location information. Sensor nodes are attached to sensor hosts where sensor software and configuration data are stored.
Therefore, Sensei-UU testbed can easily relocate part of itself. The software is called Vclient.
Figure 2.2 shows the structure of Sensei-UU testbed. Square corner rectangles are hardware while rounded rectangles are software running in hardware. Vclient is the software running in sensor host. From the figure, there could be several Vclients running in one sensor host. One is for sensor host itself while the others are for WSN nodes attached to this sensor hosts.
The ultimate destination of all data is the site manager running in the computer, which is
the center of whole testbed, hence testbed user is able to observe the status of all sensor nodes.
Before this, data is sent to Vclient. After Vclient retrieves data from sensor nodes, it forwards the data to one pre-configured site manager. However, it uses UDP to send data. Therefore, there is no guarantee that the data can be transferred to its destination. All data will be lost once site manager is closed or network is broken. So, Vclient is not applicable to delay tolerant network and the problem is that Vclient lacks of capability of local storage.
One obvious solution to this problem is to modify the source code of Vclient. But Vclient is written in C and quite stable. So it is not wise to change Vclient considering my change may break Vclient. To make things easier, a program like site manager can be placed in sensor host where this Vclient is running. This program should have ability of receiving data from Vclients and storing data in local database. The analysis of site manage can be found in the next section. Because this program and Vclients are running in the same sensor host, there is no connection problem. And another goodness is that part of the code in site manager, which is in charge of receiving data from Vclients and communicating with monitors, can be reused.
3.2 Site manager
All Vclients running in sensor hosts and monitors in the network connect to site manager. The functionality of site manager is extremely simple. Site manager receives data from Vclients and monitors, and then handles these data according to the data type. For example, a PING message from a Vclient includes position information of a sensor node. Whenever site manager receives this data from Vclient, it will forward it to all connected monitors. Hence, site manager knows all the information of connected Vclients and monitors.
Consequently, the implementation of site manager is straightforward and concise. Figure 3.1 demonstrates the activity graph of the site manager in Sensei-UU testbed. There is an infinite loop in site manager. In every loop, site manager waits for connections from others.
Function select allows site manager to monitor multiple sockets at the same time and block the program until there are data incomings from one or more sockets. When one or more sockets are available, site manager uses one for-loop to identify the sockets, for example, whether the socket is for a Vclient or a monitor. Once a socket is identified, corresponding handler will be called and executed. Then site manager continues to wait in select call after handling all sockets.
Site manager works perfectly in original testbed but it is not applicable to delay tolerant network. One problem of original site manger is the same as Vclient has. It does not have capability of local storage. In other words, site manager does not support database storage.
Figure 3.1: Activity graph of site manager
All data received is only be forwarded to monitors, so these data cannot be used in the future.
And from the analysis in the previous section, site manage should be able to run not only in computer but also in sensor host in new testbed. From this point of view, site manage also need a database to keep data.
Configuration of original site manager is fixed in the beginning. Once it starts, its config- uration cannot be changed. In DTN, connection is not stable. Once connection is lost, site manager could find a different site manager and continue transferring data. No need to restart.
As a result, new site manager should be able to broadcast to find a new partner. If there is response after broadcasting, two site managers exchange their own identities and synchronize data.
If both problems above could be solved, then site manager will work properly no matter where it runs and no data will be lost. Obviously, site manager needs to be redesigned.
3.3 Monitor
In Sensei-UU testbed, monitor is a well-configured Vendetta. It can run in the same machine as where site manager is running or in a separate machine. In the configuration file, a proxy field should be set so that monitor can start regularly.
In proxy field, there are three attributes: address, port down, port up. Attribute address indicates the IP address of site manager. Attribute port down and port up are the ports that a monitor uses for downloading and uploading data.
When starting, monitor tries to connect to site manager that is configured in the config- uration file by using download port. Once it connects to the site manager, monitor will wait for data from site manager. Monitor only processes data that follows a fixed pattern. Because site manager only forwards data from Vclients to monitors, the pattern used in monitors is the same as the pattern in Vclients.
If monitor starts properly, all the WSN sensor nodes can be seen in the screen. If a user wants to send commands to a WSN sensor node, he/she should select the specified node in the table first and click one command button. In Sensei-UU testbed, these commands are written in scripts in the same file as proxy uses. These scripts use SSH to connect to sensor host where the WSN sensor node is attached and run commands then. Apparently, this is not prefect for delay tolerant network since SSH tends to fail when network is not available. A solution for this could be using command messages. When sending a command, monitor creates a command message and inserts it into the local database. Then this command message will be sent to destination when the connection is available.
Monitor is used to present GUI on the screen. For this reason, researchers always use it in a convenient environment like in a lab. We could suppose network condition works well and we do not need to worry about it. Besides, its usage is to demonstrate data on screen. Since we store data in database, it is still acceptable. Therefore, the only changes in monitor would be to create some new command messages to control site managers.
3.4 Overall
Delay tolerant network does not have connectivity guarantee, even there may be no end-to-end potential path between nodes in the network. For example, for such scenario that there is no path between two nodes, how can a user exchange data between these two nodes? In original
testbed, it is impossible to synchronize data if there is no connection.
One mobile device like cellphone therefore should be introduced to the network. A cellphone is used when user wants to obtain data from or send commands to a sensor host, which is not reachable in any type of connections (wire or wireless). A user or a helicopter can carry a cellphone to the place where the sensor host is located. This cellphone then tries to connect to the sensor host. When connection is established between the cellphone and the sensor host, synchronization will start. Cellphone will get all the data from sensor nodes which are attached to the sensor host after synchronization, while sensor host receives commands from user and execute them if possible. Cellphone will play an important role in the network.
Chapter 4
System Design
In previous chapter, we analyze the architecture and find drawbacks of Sensei-UU testbed.
And we also pose possible solutions to these drawbacks. In this chapter, we summarize these solutions mentioned above and raise an overall design to improve Sensei-UU testbed.
4.1 Cellphone
Figure 4.1 shows why cellphone is introduced into Sensei-UU testbed.
The motivation of introducing cellphone into Sensei-UU testbed is to get data from nodes that are geographical distant or hard to connect to. As mentioned in previous chapter, in some situation, nodes in the network may be far away each other. In this case, wireless signal will be so weak that nodes cannot communicate with common way, such as Bluetooth. In some extremely conditions, nodes are impossible to send data to each other. With cellphones, re- searchers can retrieve data from those nodes. Otherwise, this data would hardly be obtained.
In figure 4.1, two site managers cannot connect to each other for some reason. One of them is running in a sensor host which means there is always new data produced. If we use original testbed, data cannot be sent out and continues to lose until the connection is fixed. So in this scenario, one cellphone can be used as a carrier. Whenever this cellphone is near to one sensor host, it can connect to this sensor host and begin to exchange data. After that, if it reaches the control center or another sensor host where one site manager is running, they will synchronize as normal. Thereby, the ‘lost’ data is back to our network. This is a typical and complete process of how one cellphone work in new testbed.
Figure 4.1: Cellphone in Sensei-UU testbed
Cellphones are important in new Sensei-UU testbed. But they are different from sensor hosts in original Sensei-UU testbed. And site managers running in cellphones are not the same as those in sensor hosts. Cellphones have no responsibility to receive data from sensor nodes, therefore, there is no need to support site managers running in cellphones to capture sensor data. Details about site manager in cellphone will be presented in section 4.4.2.
4.2 Messages
Before talking about the design of site manager, data in the network is considered. Data trans- ferred in the network, which is called message in this report, has varieties of types.
First of all, site manager in sensor host receive data from sensor node. Since we do not change configuration of sensor node, this raw data received follows pattern like ‘TIME IP:PORT PING {DATA}’. In this message, TIME shows when this message is sent, IP and port indicate where this message is sent. PING is a plain text flag that means this is a ping message. And DATA contains all information received from sensor. In this thesis project, sensor gets location information. If other types of sensor nodes are adopted, new information will be revealed in
Table 4.1: Message types in the testbed
Message type TYPE field
Ping PING
Command CMD
Destination DST
Command response CMD RES
Acknowledge ACK
DATA part.
Then, testbed user can send commands out to specific sensor nodes by using monitor. These command data uses pattern like ‘TIME DST TTL CMD {CMD DATA}’. Just as TIME field above, TIME here indicates when this message is sent. DST field points out which sensor node is the destination of this command. TTL field illustrates value of Time To Live (TTL).
This value determines how long this message can live. The default value in Sensei-UU is 2 days. After 2 days, no matter where the message is, it will be ignored and will not be sent any others. CMD field shows this message is a command message and CMD DATA field includes command user want to run in specific sensor node. In general, sensor host which connects to the specific sensor node keeps many scripts, such as start, reboot and stop sensor node scripts. In CMD DATA field, it must contains a completed and legal command that can run in sensor host.
Both types of data above will be converted into common messages transferred in the network and inserted into database. A common message follows pattern ‘HASH TTL TIME IP PORT SRC DST TYPE DATA’. Apparently, it is more complicated than PING and CMD messages.
TTL, TIME, PORT and DATA fields have the same meanings as mentioned above. HASH field is used to test and verify if a message is valid. How to get this hash value will be discussed in next chapter. SRC and DST indicate the source and destination of a message. Unlike IP and PORT fields, SRC and DST use unique identities of sensor node. To make our system concise, we use one common message pattern in DTN. However, messages still have different types. So, TYPE is another important field in a message. For different messages, they have corresponding different types.
Table 4.1 shows what messages we have in our system.
We have talked ping and command messages before. Destination message is created and sent back when one message reaches its destination. Command response message contains re- sult of one command. Acknowledge message is the most common message. It is introduced to make sure one message is sent to the other end when synchronizing.
In general, PING and CMD messages are recognized and converted into a common message first. Then it is inserted into database. All data in database contains all fields of a common message.
From object-oriented view, it is better to present message as an object. So after getting data from database, data will be converted into a message instance first. This process will be presented in next chapter.
Overall, all messages in the network follow specific patterns thus our testbed knows what their meanings are. And all these different messages are stored in the same format in database.
4.3 Database
A delay tolerant network can be deployed in the environment where it lacks of continuous net- work connectivity. This means it cannot rely on the infrastructure, such as Ethernet network.
Therefore, data collected from sensor node must be kept in local database.
4.3.1 Database selection
In this thesis project, memory usage and energy consumption are main factors to consider.
Generally, both router and cellphone are memory constrained devices. And cellphone is also a energy constrained device.
There are large numbers of database products in the market. They have different features and performance. MySQL, Oracle and SQLServer are famous database management systems, but they do not fit in our network because they are complicated in data structure and main- tenance, heavy in running. Routers and cellphone are deficient in space and effectiveness to handle these massive database management systems.
SQLite on the other hand, is an in-process library that implements a self-contained, server- less, zero-configuration, transactional SQL database engine [7]. It is quite light (with all features
enabled, the library size can be less than 300KiB, depending on compiler optimization settings) and does not require a separate process running in the background. Besides, Python has built- in support for the sqlite3 since version 2.5. So, SQLite is chosen as the default database for Sensei-UU testbed in this study. Moreover, an interface will be provided to operate the database that helps simplify the procedures of changing another database.
4.3.2 Database design
Table 4.2: Database tables
column name type meaning
id integer unique id
hash text hash value
sent integer 0 if not sent
ack integer 0 if no ack received
ttl integer TTL value
time integer when the message is created
ip text IP address
port text port
src text source node name
dst text destination node name
type text message type
data text data body
We have seen how many message types in the network in section 4.2, then we should consider how to keep them in database. Because only data is stored in database, only one table will be created. In this table, there are several columns. Table 4.2 shows all column names, column types and their meanings.
4.4 Site manager
Figure 4.2 gives us an overview of improved Sensei-UU testbed. In original Sensei-UU testbed, site managers are programs running in control center which is normally a computer. This is
Figure 4.2: Overview of improved Sensei-UU testbed
obviously not suitable for DTN. In figure 4.2 and 4.1, site managers are running in not only in computers but also in routers and cellphones.
4.4.1 Site manager in sensor host and computer
Apparently, as mentioned in chapter 3, there should be a site manager running in sensor host that is in charge of keeping data in the local database and managing connections to Vclients and monitors.
Other than the requirements above, it is better to allow user to run command in sensor hosts also. There is only one site manager in original Sensei-UU testbed. All sensor hosts are con- nected to this site manager. In this case, site manager is the center of the network. It is perfect when the connectivity can be ensured, because everything can be transferred to its destination.
However, in a delay-tolerant network, there may be a long geographical distance between two nodes and no connectivity is guaranteed. A typical scenario is that if the connection between one sensor host and site manager is down, all data from the sensors in that sensor host will be lost. Another scenario is that a person walks near a sensor host and wants to reboot it. With the central controlled Sensei-UU, he has to go back to where the site manager is located and run
the reboot command. Still, it is the case that the command cannot be executed immediately because of transmission delay.
Therefore, the site manager in sensor host has all functions as site manager in original Sensei-UU testbed, including receiving messages from Vclients and communicating with moni- tors. It is also required that a node in the network should be able to detect others nearby. So the site manager should be able to send and response broadcast messages. Besides, the new site manager has capability of storage. And it can handle no connectivity situation. The most important part in this project is to write these site managers.
Figure 4.3: Structure of site manager
So, we have a structure for site manager as figure 4.3 shows. It contains four servers and a database. The following sections will introduce them in detail.
Data server
Data server is in charge of managing connections between Vclients and site managers, receiving data from sensors and inserting data into database.
Broadcast server
Broadcast is responsible for sending and receiving broadcast messages. Site manager broadcasts in order to find other devices nearby. When receiving broadcast message, one connection is established between sender and receiver. Then they synchronize data in synchronization server.
Monitor server
Monitor server listens requests from monitors. Whenever one monitor tries to connect to one site manager, the site manager will handle this request. If connection successes, all instant data will send to the monitor from site manager, and of course the site manager will receive commands from the monitor.
Synchronization server
Functionalities of both data server and monitor server above are more or less implemented based on site manager in original Sensei-UU testbed, but synchronization server is new and important. Assume one cellphone connects to a site manager, and tries to exchange data with the site manager, situation becomes complex. In section 5.7, detail operations of how to establish the connection and how to exchange data will be showed.
4.4.2 Site manager in cellphone
Figure 4.4: Structure of mobile site manager
From figure 4.3, one site manager contains several servers. As section 4.1 says, however, cellphones are used to exchange and carry data, so there is no need to support site managers in cellphones to receive data and commands. In this case, comparing to site managers running in sensor hosts and computers, program in cellphone only has synchronization server and broadcast server. In a way, program in the cellphone is a simplified site manager.
Chapter 5
System Implementation
5.1 Overview
Figure 5.1: Class graph
This graph illustrates all the classes implemented in improved testbed. Based on the design in chapter 4, six major classes are implemented. The following sections will present them in details.
5.2 DTNMessage
DTNMessage is an abstract representation of messages in the network. It contains all the columns in database. One hash function is implemented in DTNMessage. Every time site manager receives a raw message, DTNMessage will compute hash value for this message. After
this procedure, hash values we get become a part of this message and go along with it since then.
5.3 DTNDatabase
DTNDatabase is an interface of SQLite operations. It provides basic functions to create table, insert message, update message. All functions are implemented by using sqlite3 API in Python.
If user wants to change another database, it is extremely easy to rewrite DTNDatabase without touching other parts of the testbed.
The most complex job in DTNDatabase is to access to database from multi-threads at the same time. Python 2.5, which is used in test environment, does not support for SQLite access from multi-threads. Even this feature is implemented in the following version of Python, it is still wise to implement it in a clear way. The method used in DTNDatabase is in appendix A.3.
5.4 DTNDevice
In chapter 4, program running in cellphone has been analyzed. It’s only a simplified site man- ager. However, it is meaningless if DTNSiteManager inherit from DTNMobileSiteManager.
Therefore, we introduce DTNDevice.
DTNDevice is an abstract class. It is the base class of both DTNMobileSiteManager and DTNSiteManager. It represents one physical device in the network. It is able to store data in local database, send broadcast message and synchronize with others.
Every device in Sensei-UU testbed has a unique identifier. This identifier is used to dis- criminate different devices in the network. DTNDevice implements vast majority of logical operations. And it supports to listen ports for broadcast and synchronization. Moreover, some Python technical are applied to make the code concise and easy to extend. More detail can be found in appendix A.1.
5.5 DTNMobileSiteManager
DTNMobileSiteManager inherits from DTNDevice. It presents a cellphone in Sensei-UU testbed.
It does not have new functions implemented. It only overrides default value of the unique iden- tifier in DTNDevice.
5.6 DTNSiteManager
DTNSiteManager inherits from DTNDevice. According to section 4.4.1, a site manager running in sensor host or computer should have the capability of receiving data from sensor nodes and communicating with monitors. In other words, it has data server and monitor server as well as synchronization server and broadcast server.
DTNSiteManager extends and rewrites some code of DTNDevice. The most important variable in both DTNSiteManager and DTNDevice is called ’f map’. This variable contains a map in which a socket variable is the key while its corresponding event handler is the value.
Whenever one socket receives message, its corresponding handler will be called. In appendix A.2, key point code is presented.
5.7 DTNConnection
DTNConnection is an abstract representation of the connection between two devices. Once two devices are able to communicate and exchange connection information, an instance of DTNConnection is created in both devices. There are three threads running in DTNConnection, and DTNConnection itself is a thread too.
5.7.1 Establish connection
In the case of a site manager knows information about another site manager which includes IP and port, this site manager can try to make connection. However, there should be some information exchanging during this process. The most important information is what data this site manager is interested in.
Considering Figure 5.2, one mobile site manager named MOBILE wants to synchronize with VINCENT. First, it must know IP and port of VINCENT. This figure shows how to use broadcast to find nearby working site manager. We can also skip this part if we know IP and port. Then MOBILE sends its id which is MOBILE and its interesting data. Here it is interested in anything. Then VINCENT returns its name, an opening port number and its interesting data. VINCENT is only interested in data sent to itself. After this step, another socket is created by using the new port. So we have two sockets, one is for sending and one is for receiving data.
Figure 5.2: Preparation before synchronization
5.7.2 Strategy of gaining data from database
Data should be retrieved before being sent. But not all data in database should be sent to the other end, because some data may have already received by the other end and some data should not be sent to the other end. Therefore, a strategy must be applied when obtaining data from database. The following part will explain the detail of the strategy used in delay tolerant network.
Several factors are considered when retrieving data from database. The reason for having this strategy is to avoid wasting bandwidth and shorten transmission time.
• Destination field is not the device itself. Because it means data has already reached its destination and there is no need to transfer it anymore if destination field is the device itself.
• Source field is not the device in the other end. Because data should not be sent to its source.
• Ack field does not equal 1. Message has already been sent to its destination if ack field equals 1.
• Data has not expired. Expired data can be ignored. This can be checked by TTL field.
• Device in the other end should be interested in this data.
5.7.3 Threads in DTNConnection
In one site manager, sending and receiving data happen at the same time. It is better to have two threads for sending and receiving separately.
Sending thread
In sending thread, data are retrieved from database and added to a message queue. Then an infinite loop starts. In every loop, a message is got out from the queue, and it will be sent by the sending socket. Apparently, this queue records messages that are going to be sent. When all messages are sent and the transition is complete, a terminating message will be sent to the other end and a flag variable, which is monitored in daemon thread to observe the status of sending thread, will be set as well.
Receiving thread
In receiving thread, program waits for data from the other end. Whenever data arrives, a message instance can be created. Different operations are carried out on account of the message type.
• If the destination of message is the device itself, DST ACK is created, and then inserted into both database and sending queue. If message type is CMD, a CMD RES message is also created after running the command.
• If message type is DST ACK, which means one message has reached its destination, ack field of this message will be updated into 1. Set ack field of this DST ACK message one, and insert it into database.
• If terminating message is received, one flag is set.
• If message type is ACK, just ignore it. Otherwise, create an ACK message and inserted into sending queue.
Daemon thread
A daemon thread monitors the status of both threads above periodically. When both flags in sending and receiving threads are set, it means synchronization is done. Then one class scope flag is set and all loops in sending thread, receiving thread and daemon thread will be broken.
Chapter 6
Evaluation
6.1 Experiment setup
Figure 6.1: Test network
In the test network, I have two routers as sensor hosts, two sensor nodes, one cellphone and one computer. Graph 6.1 presents what the test network looks like.
Two sensor nodes are attached to the same sensor host called Vincent. According to the
illustration in previous chapter, there are two Vclients and one site manager running in Vincent.
The site manager in Vincent has a default destination which is Trapper. The computer is used to listen to data in the network, so one monitor is running in this computer and connects to Trapper which means so long as Trapper receive any messages, it will send to the computer.
All devices are connected through WIFI except from computer to Trapper.
6.2 Experiment
The aim of all experiments is to send data from Vincent to Trapper. The following three ex- periments cover all the situations that will happen in the testbed.
First, if network is stable, there is no need to use cellphone to carry data between two sensor hosts. Vincent is able to send data directly to Trapper. This process is the basic of the testbed and it works perfectly.
Secondly, suppose two sensor hosts cannot connect to each other, data must be transferred through cellphone. When the cellphone comes close to Vincent, it is able to find Vincent in- formation by sending broadcast message. Once cellphone and Vincent exchange their identity information, the regular synchronization process begins. The process between Trapper and the cellphone is the same, only that the monitor can also get data and present it on screen. Besides, the process is bidirectional. Command messages can also transfer reversely.
Finally, suppose the network is not stable, synchronization could be interrupted. When Vincent and cellphone is exchanging data through WIFI, the cellphone is moved far away.
So the connection is lost. Then both site managers in the cellphone and Vincent detect this connection problem and stop synchronizing. When connection is recovered, both site managers resume synchronization from the terminate point. This is possible because information of last successful message is kept in site managers.
6.3 Storage and memory use
In my implementation, all data is kept in Sqlite. From the graph above, the trend of used space is linear. Every message takes around 230 bytes. So 1MB space could store over 4 thousand pieces of data. As all data has not been compressed, this value is quite good.
Figure 6.2: Measurement of storage
Site manager does not take too much memory. When site manager is idle, it takes around 7MB memory. Memory use goes up to more than 20MB when site manager is synchronizing, and it maintains this number until the synchronization finishes. During the test, sometimes the memory use suddenly rises too much and site manager uses more CPU consumption than average. The problem I find is that site manager receives too much data in unit time. In this case, too many messages are needed to insert into database at the same time, but it is unable to handle these messages right away. So the queue of queries gets longer which causes more memory use and CPU consumption. Appendix A.3 gives more information of this queue.
This problem had never happened before I tried to simulate transferring data from VClient to site manager using Python script. According to further test, site manager works perfectly if the interval of two pieces of data is longer than 1 ms. As long as the number of interval configuration for VClient is always larger than 1 ms, it is no problem to use new site manager in the testbed.
6.4 Overhead
Messages in the testbed are described in section 4.2. Messages transferred in the network are different from the original messages from sensor nodes. Next, I will analyze overhead per message and session overhead.
6.4.1 Message overhead
2cc2f40e95db75de9d403449f9833e43 172800000 1300803228892 130.238.8.154 7000 vincent trapper
PING {position=>wgs84,17.647102456850032,59.83816340033785, 62.99983961507678;platform=>ASUS WL-500GP;type=>sensorhost;}
This is a typical message in the network. At first glance, it may not make much sense, but it contains several fields and its format is presented below:
<Hash><TTL><Time><IP><Port><Source><Destination><Type><Data >
Compared to its original message from sensor node, some fields are newly added in the message. They are ‘Hash’, ‘TTL’ and ‘Destination’ fields. ‘Hash’ field takes constant 32 bytes.
There are no limitations of how long ‘TTL’ and ‘Destination’ fields are, but in a reasonable situation, they are no longer than 16 bytes. Therefore, each message is about 64 bytes longer than original message. Compared to the length of ‘Data’ field, these three fields take very little proportion of one message.
Even though message overhead costs very few, it is still possible to optimize. If the amount of data is small or uniqueness is not a major consideration, it can be easily cut down the length of ‘Hash’ field by applying another hash generate function. ‘TTL’ field can be shortened by applying unitized sign in the whole network instead of using numbers explicitly. If user requires shorter messages, compression is need and it will be talked in the next chapter.
6.4.2 Session overhead
The whole process of synchronization can be divided into three parts: initialize part, synchronize part and terminate part. The overhead of synchronize part is also the message overhead which is talked above. Session overhead contains the overhead in both initialize and terminate parts.
Initialize part is presented in section 5.7. During this part, only identity information of both site managers is exchanged. For this reason, compared to the huge amount of data in synchronize part, the overhead here can be ignored. Moreover, the overhead in terminate part only contains two terminate messages from each site manager. So in the testbed, the cost of session overhead is quite small.
6.5 Measurement of sending time
In the testbed, the transmission is bidirectional. This means transmission from each end carries out at the same time. According to test, bidirectional transmission could make use of band- width more efficiently.
The graph shows how long is used to send different amount of data. From the figure, the more data to send, the more time it takes. In addition, the time used is almost linear. The speed of sending is less than 2ms per message. Figure 6.3 also shows that the time used to transfer message in new testbed is a little more than that in old testbed. This is reasonable because of the overhead talked in the previous section.
Figure 6.3: Measurement of sending time
Chapter 7
Future Work
The design and implementation of DTN testbed is a prototype, so it can be improved in a number of ways and more work remains to be done to make it a fully featured DTN.
7.1 Protocol
Messages are transferred through TCP in Sensei-UU WSN testbed. Whether one message reaches its destination or not depends on the features of TCP. If code detects one error, it means there is something wrong with this message and retransmission is needed. Besides, only one hash value is used to detect if one message is available.
Therefore, it is better to find a practicable protocol and apply it to Sensei-UU testbed.
Then the testbed could handle more complicated message and deal with more sophisticated operations. Nevertheless, this may bring a heavier overhead, so a comprehensive evaluation and compare must be made.
7.2 Security and compression
As mentioned in chapter 4, two nodes exchange information first before they synchronize. In this process, some information, which will be used in next step, is exchanged. But this infor- mation is in plain text. And messages transferred in the network are in plain text. Transferring plain text in the network may cause two problems: security problem and efficiency problem.
People who can access our testbed can monitor network traffic by using tools like Wireshark.
If messages in the network are in plain text, they can easily read them thus cause data leakage.
Since data got in this project has little safety value, it is not a big problem. However, it may
be not allowed in reality.
No matter it is wireless network or Ethernet network, it takes more time to send uncom- pressed data. With a period of time, more available data could be sent if we send compressed data. Besides, hacker spend more time in figuring out how to extract compressed data, hence our testbed is safer.
Besides, in this thesis, there are routers and cellphones where different kinds of site managers run. In these devices, the storage is constrained. In the test network, OpenWrt is installed to routers. After installation, only several MBs left in the router, even there is no space to install Python. So USB sticks are attached to routers while micro SD memory cards are inserted into cellphones in order to extend capacity. These external storage devices are big enough to keep GBs of data. But how much memory is needed and how to hold more data using fixed amount of memory are still problems.
In general, taking security and compression improvement into account in future is valuable.
7.3 Multiple connections
Currently, one site manager is not able to connect multiple site managers at one time. One drawback of this could be explained in the next scene. Considering many sensor hosts distribute in one area where human hardly gets, one helicopter flies over this area to gather data. If only one connection can be handled, the bandwidth may not be utilized efficiently. So it takes much more time and costs more. Together with compression problem, how to make use of bandwidth is a great direction to work on next.
7.4 Opportunistic networks
Opportunistic network is a type of Delay Tolerant Networks where network communication opportunities appear opportunistic. It focuses on how to route in the network. In our testbed, nodes get close to each other and synchronize while in opportunistic network, nodes will choose who they want to connect to. Opportunistic network is out of the scope of this thesis, but it is a good idea to continue working in this direction.
Chapter 8
Conclusions
In this thesis, I presented a solution that enables Sensei-UU testbed to deliver delay-tolerant data. A systematical analysis was adopted to understand how Sensei-UU testbed works. More- over, I improved the testbed upon the pre-existing architecture and designed a concise architec- ture. The new Sensei-UU testbed can work in a connection unstable environment. Every sensor host has a site manager running on it. Site manager keeps all data in local database. Once there is available connection, data is fetched from database and sent to others. A mobile de- vice is introduced into the testbed. It aims at transferring data between different site managers.
The new architecture makes as little changes as possible on existing testbed. Python is used to implement site manager in the network because for one thing, code of previous testbed can be reused as much as possible and for another new testbed is able to run in various areas where Python interpreter is available. New data type can be easily introduced to the testbed because database design provides sufficient flexibility. Therefore, all the requirements in the first chapter have been fulfilled. And with lots of testing, we ensured that the testbed meets all requirements and is stable enough.
REFERENCES
[1] “Sensei-uu wsn testbed : http://www.it.uu.se/research/group/coregroup/projects/sensei- uu,” Sept. 2011.
[2] O. Rensfelt, F. Hermans, L.-˚A. Larzon, and P. Gunningberg, “Sensei-UU: A Relocatable Sensor Network Testbed,” p. 8, 2010.
[3] K. Fall, “A Delay-Tolerant Network Architecture for Challenged Internets,” p. 14, 2003.
[4] A. R. Pollak, “Challenges and Considerations for a Delay-Tolerant Wireless Sensor Network Deployment,” p. 68, 2010.
[5] “Delay-tolerant network : http://searchnetworking.techtarget.com/definition/delay- tolerant-network,” Sept. 2011.
[6] O. Rensfelt, F. Hermans, P. Gunningberg, L.-˚A. Larzon, and E. Bjornemo, “Repeatable Experiments with Mobile Nodes in a Relocatable WSN Testbed,” p. 14, 2010.
[7] “Sqlite : http://www.sqlite.org,” Sept. 2011.
APPENDIX
Appendix A
Code implementation
A.1 DTNDevice
A.1.1 Constructor
First of all, DTNDevice is a subclass of threading.Thread.
Every DTNDevice should have a unique indication, by which user can identify the device.
In DTNSiteManager, this indication is the id of sensor host where this site manager is running.
In DTNMobileSiteManager, this indication is generated by the system. But it can also be set manually.
Another important variable keeps a list of unique indications mentioned above. It is used for stating what information the device is interested in. For example, when a cellphone connects to a site manager, it will receive data from the site manager. However, if the user who carries this cellphone only wants to get data from another specific site manager, there is no need to synchronize with this site manager. Thus this variable is introduced. The two devices from two ends start to exchange the interested data after the connection is initialized between them. If one has the data the other one is interested in, this data will be sent when synchronization gets started. Usually, the value in a DTNMobileSiteManager is ‘*’ which means no matter what this cellphone is connecting to, all information from the other end will be requested. The value in a DTNSiteManager is usually the indication of itself which means it only cares about itself.
The history of connection is also kept in DTNDevice. A typical scenario is two devices exchange information twice in a row. Same data would be sent and stored twice without the history of connection. By having recorded history of synchronization, transmission can start from the point of last transmission. Consequently, it saves a lot of energy and has a great boost
