IN
DEGREE PROJECT COMPUTER ENGINEERING, FIRST CYCLE, 15 CREDITS
STOCKHOLM SWEDEN 2017,
Visualization of IoT Data on Mobile Devices and Modularity of
Applications
ADRIAN GORTZAK
JIMMY FJÄLLID
Visualization of IoT data on mobile devices and modularity of
applications
Adrian Gortzak
Jimmy Fjällid
Abstract
Wireless sensor networks used in smart cities and elsewhere collect large amounts of sensor data that needs to be made available to respective audiences and have proper visualization.
There is a lack of mobile applications capable of visualizing sensor data coming from multiple sources using different protocols and data formats in a clear way; an application that is easy for anyone to use. This paper will present the development and evaluation of Smart City IoT which is an Android application for visualization of IoT data collected from wireless sensor networks. The resulting application Smart City IoT is designed to be modular to the core to allow for easy extension with new sensor types, communication protocols and data formats. It demonstrates that an Android application can be developed to support multiple sources with different protocols and have a unified visualization without requiring extensive set up.
Keywords
Android, IoT, Visualization, modularity
Sammanfattning
Trådlösa sensornät som används i smarta städer och på annat håll samlar in stora mängder data som behöver göras tillgängliga för respektive publik och använda sig av lämpliga
visualiseringar. Det är brist på mobila applikationer som är kapabla att visualisera sensordata som kommer från flera källor med olika protokoll och data format på ett tydligt sätt; en
applikation som är enkel att använda. Den här rapporten kommer att presentera utvecklingen och valideringen av Smart City IoT vilket är en applikation till Android för att visualisera IoT- data hämtat från trådlösa sensor-nätverk. Den resulterande applikationen Smart City IoT är designad för att vara modulär i sin kärna för att tillåta enkel utbyggnad av nya sensortyper, kommunikationsprotokoll och dataformat. Den demonstrerar att en applikation för Android kan utvecklas med stöd för flera källor, med olika protokoll och ha en gemensam visualisering utan att kräva en omfattande konfiguration.
Table of Contents
1 Introduction 8
1.1 Background and problem 8
1.2 Purpose 9
1.3 Goal 9
1.4 Delimitations 9
1.5 Sustainable Development 10
1.6 Outline 10
2 Background and Related work 11
2.1 Internet of Things 11
2.2 Publish-Subscribe 12
2.3 Client/Server 13
2.4 Client/Server vs Publish-Subscribe 13
2.5 MQTT 14
2.6 CoAP 14
2.7 Application programming interface 14
2.8 IoT data 15
2.9 Mobile development 15
2.10 Similar applications 16
3 Methodologies and Methods 18
3.1 Project methods 18
3.1.1 Extreme Programming 18
3.1.2 Scrum 19
3.2 Information Architecture 20
3.2.1 Sitemap 20
3.2.2 Wireframe 20
3.3 Development environment 20
3.3.1 Version control system 20
3.3.2 Integrated development environment 21
3.4 Testing and Evaluation 21
3.4.1 Evaluation 21
3.4.2 Guerrilla testing 22
3.4.3 Usability testing 22
3.4.4 Unit tests 22
4 From idea to application 23
4.1 Sprint 1 23
4.1.1 Sitemap 23
4.1.2 Wireframe 25
4.2 Sprint 2 26
4.2.1 Database 26
4.2.2 Activities and fragments 28
4.2.3 Background service 28
4.2.4 Message retrieval 29
4.2.5 Limit the working data 30
4.2.6 Sensor limits 30
4.2.7 Topic data retrieval 31
4.2.8 Data structure support 31
4.3 Sprint 3 31
4.4 Sprint 4 32
4.4.1 Generated data 32
4.4.2 Sensor data from the residence 33
4.5 Sprint 5 33
4.6 Sprint 6 35
4.6.1 API 35
4.6.2 Type alias 35
4.6.3 Updating sensor types 37
4.6.4 Improvement of activity structure 37
5 The Application 39
5.1 Start screen 39
5.2 Map visualization 40
5.3 List visualization 41
5.4 Individual visualization 42
5.5 Add sources 43
5.6 Update sources 44
5.7 Update location 45
5.8 Update sensor types 46
5.9 Update sensor type aliases 47
5.10 Change active sensor types 48
6 Conclusion 49
6.1 Discussion 49
6.2 Security considerations 51
6.3 Future work 51
7 References 53
Terminology
View - A view refers to a visual component occupying the screen.
Broker - A server that receives publish messages from clients and allows other clients to subscribe to them, thereby acting as a middleman.
Service - A task that runs in the background without a user interface.
Activity - Main component in Android application typically responsible for user interaction.
Fragment - Represents a behavior or portion of the user interface in an activity
Sprint - A limited time period with assigned goals that should be completed by the end of it.
API - A software interface to communicate with a program or a service.
API call - A call made to an API to communicate with the underlying program or service
1 Introduction
The idea of using IoT technologies to develop a smart city is currently under investigation in Sweden, (among other places), in a project named GreenIoT[1]. A smart city and all its services should be designed to be more intelligent and improve the quality of life for its citizens[2]. Another phenomenon that is emerging is what is called the Internet of Things (IoT) which refers to the possibility for small devices such as refrigerators, sensors and smartphones to connect to the Internet to send and receive data[3]. A smart city built using IoT to collect data will produce high amounts of information and somehow that data should be presented to the users. One of the utilities of collecting sensor readings in a city is that they can be beneficial for city planning as they give indications of current conditions.
Wireless sensor networks are being constructed to gather and make available various sensor readings such as weather data and air quality in the GreenIoT project[4]. What is missing are good ways to represent the data to the users which in a smart city might be the civilians.
Today a majority of the population in Sweden owns a smartphone[5], making a mobile application a viable way to present the available sensor data. While there are mobile applications that can gather real time data and visualize it, they are often limited to using only one communication protocol or data format. Because there is not just one protocol in use, a suitable mobile application would have to support multiple protocols and data formats or at least be easily extended to include them.
This paper will discuss the development process of a mobile application that can visualize IoT sensor data and various problems that were encountered. It will attempt to fill the gap of an application that can visualize sensor data coming from multiple sources using various protocols and data formats.
1.1 Background and problem
This project is a part of a collaboration project by, among others, the Royal institute of Technology in Sweden (KTH), Uppsala University and the Uppsala municipality called GreenIoT[1]. It is a study and an implementation of a city wide sensor network to measure air quality using sensors distributed over Uppsala and interconnected wirelessly. The sensor data should be collected and made available both for individuals and organizations in a useful manner. Individuals should be able to retrieve sensor data such as the temperature, humidity or pollution measurement from a specific sensor. Organizations should be able to get batch data from multiple sensors, including historic data, and use it to build models of cities for city planners.
For a long time there has been multiple sources from which you could retrieve sensor data. No matter if it is geological data delivered through a georeport, temperature and humidity information from the weather report or pressure from a barometer. Much of the information
and readings originates from different sources and a user would likely have to use multiple applications to read the desired sensor data.
With the Internet of Things the amount of potential data sources available has greatly increased and some of them are dedicated to taking environmental readings such as temperature, humidity, pressure and air quality. While it could be a great source of data, it leads to the question of how to represent it in one application and make it easily accessible by end users.
When trying to gather different types of data from multiple senders the problem of data formatting and communication protocols presents itself. There is not just one communication protocol standard to rule them all that is used for everything, which means that the application would need to support multiple ones at present and perhaps new ones in the future. Which data format should be used to contain the data? It would probably be hard to convince all senders of sensor data to use the same formatting which means the application would have to support different kinds of formatting to suit everyone and preferably be easily extended to add a new data format whenever it is required. The problem is then how to build a useful mobile application simple enough for a people to use but with compatibility to support connecting to data sources using different protocols and data formats that is easily extendable in the future if new protocols and standards arrive.
1.2 Purpose
The purpose of this report is to describe the work process and result of developing an android application. The application should be suitable for visualizing real time sensor data collected by the wireless sensor network in the GreenIoT project to citizens of Uppsala. It should be easy to use for users without previous Android experience. Furthermore, the application should be usable by other projects, whether it is smart cities or smart homes, and it should allow extension with new data formats and new communication protocols.
1.3 Goal
The goal of this project is to develop a framework application for android that is available to individuals for displaying different types of sensor data. It should be able to collect data from multiple sources simultaneously and by using different protocols. The application should be able to visualize both real time data and historical data and source locations where applicable.
1.4 Delimitations
The application will only be developed for the Android operating system. The initial version will only support Message Queue Telemetry Transport protocol (MQTT)[6] and REST[7] API calls as communication protocols. The SenML data structure used in the GreenIoT Project will
be the only supported data format for MQTT. There will only be three representations of the data; in the shape of the latest values shown on a map, a list and history displayed in a graph.
1.5 Sustainable Development
The application is developed as a flexible framework which can be extended to support a new communication protocol or data format. Allowing easy extension of the application reduces the risks of becoming obsolete when a new protocol or data format appears which might be the case if it is easier to develop a new application. New sources of sensor data from another project or from a smart home using our supported communication protocols and data formats could be added from inside the application with just a few clicks making it very versatile.
Another aspect is that the code is being developed as open source. By releasing the entire source code as open source, any developer is free to continue developing the application and change it which thus reduces the risk of abandonment.
1.6 Outline
Chapter two of this report will start by describing previous work that exists for visualization of IoT data, discussing common problems and advantages of these solutions. These are things to learn from when developing our own application. It will move on with the theoretical background connected to the project. This includes information about IoT in general including protocols and communication structures, IoT data, data visualization on a mobile device and things to take into consideration when developing a mobile application.
Chapter three describes the methods and methodologies that will be used in the project. This includes the project methods that will be used during the project, the methodologies that the modeling of the application will follow and what development environment will be used. It also includes which testing methods will be used to ensure stable operation of the application.
Chapter four will describe our use of the methodologies from chapter three. It contains detailed accounts of the problems we encountered during the development of the application and our solutions.
Chapter five will show the resulting application and the features that it supported at the end of this project.
Chapter six is the last chapter and will contain the discussion about the result, security considerations and potential future work.
2 Background and Related work
This chapter will start by introducing The Internet of Things and what that is as well as the Client/Server and Publish-Subscribe architectures. It will move on to describe the MQTT protocol and the Constrained Application Protocol (CoAP) followed by a description of what an Application Programming Interface (API) is. At the end it will discuss mobile application development and existing similar applications.
2.1 Internet of Things
Kopetz describes The Internet of Things (IoT) as the connection of physical things, both animate and inanimate, to the Internet[3]. He describes them as smart objects and as the building blocks of the Internet of Things and that smart objects is another name for embedded systems connected to the Internet. Salman describes IoT as a technology that allows devices that are not computers to act smartly and make collaborative decisions[8].
Salman also states that the Internet of Things introduces the need for specialized standards and communication protocols.
Two common models for the Internet protocol stack are the OSI model and the TCP/IP model as seen in Figure 2. The IoT protocol stack seen in Figure 1 differs from the others by the inclusion of an adaptation layer below the Network layer. IEEE 802.15.4 is a popular technology for wireless sensor networks to communicate but comes with a limitation of a maximum packet length of 127 bytes[9]. 6LowPAN is a protocol defined to enable IPv6 over 802.15.4 because the 16 bytes IPv6 addresses add much overhead[9]. IPv6 requires a
Figure 1: Illustration of
the IoT Protocol Stack Figure 2: Comparison between the TCP/IP and OSI model
minimum Maximum Transmission Unit (MTU) of 1280 bytes and the 802.15.4 max packet length is 127 bytes which is why 6LowPAN also defines fragmentation to support this requirement[9].
Salman describes specialized protocols and standards used for the IoT protocol stack and some of the application protocols used are the Constrained Application Protocol (CoAP) and the Message Queue Telemetry Transport protocol (MQTT)[8].
2.2 Publish-Subscribe
The Publish-Subscribe model is centered around the idea that the sender (publisher) is not directly connected to the receiver (subscriber) and instead a middleman, also known as a broker, is used to relay the information[10]. Thus, the publisher does not have to keep track of all subscribers but rather only keep track of the broker. An advantage with Publish-Subscribe for IoT devices in a wireless sensor network is that a sensor can sleep most of the time and only wake up to collect a sensor readings and publish it to a broker which conserves
resources[8].
A publisher sends the data to the broker to a specific topic selected by the publisher and a client can subscribe to that topic to get the messages from the broker as soon as they are published as seen in Figure 3.
Figure 3: Illustration of the Publish-Subscribe architecture
2.3 Client/Server
Hanson describes the Client/Server architecture as a model where a client can request a resource or service from another system called a server which in turn receives requests from clients as seen in Figure 4 [11]. Hanson goes on to say that the client is the initiator of any activity and a server waits for clients to initiate communication. A client can be connected to multiple servers and a server can handle multiple clients.
2.4 Client/Server vs Publish-Subscribe
In the case of a mobile device, the Client/Server architecture relieves the mobile device from having to perform calculations and the need to always be connected to a broker such as in the publish-subscribe system. Instead it can act as a dummy that requests the needed data at the time from the server. This could result in the benefit of prolonged battery life and less
consumed mobile data which might be limited or have an associated cost. A disadvantage compared to the publish-subscribe system is that there will be a variable delay between when a message is sent to the server and when that message is retrieved by a client. This delay will depend on how often the client requests data from the server. Long polling[12] can be
implemented to solve this but the previously mentioned benefits would be removed.
While the Client/Server architecture could be suitable for a mobile device to retrieve data from a server, it is less suited in a wireless sensor network built of constrained IoT devices.
Using this architecture, each sensor node could act as a server such that various clients could connect to them to retrieve new sensor readings. This would mean that a sensor would have to
Figure 4: Showing the Client/Server architecture
always listen for requests from new clients and have enough resources to handle connections from potentially many clients. Another problem is that IoT devices communicating over 802.15.4 using 6LowPAN requires a bridge to translate the compressed 16 bit IPv6 addresses in 6LowPAN to 128 bit addresses used on the Internet[13]. Instead of using the Client/Server model, the Publish-Subscribe architecture could be used. This would allow for better
conservation of resources as described in section 2.2. The broker could be located at the gateway to the sensor network thereby removing any need for the sensor nodes to be directly accessible from the Internet.
2.5 MQTT
MQTT was designed as a lightweight Publish-Subscribe message protocol for low-bandwidth devices and uses the machine to machine (M2M) structure[14]. An MQTT client can publish messages to a topic and another client can then subscribe to that topic to receive the published messages[6]. A broker such as Mosquito[15] acts as a relay by allowing clients to publish and subscribe to topics and keeping track of which topics exists and which clients are subscribed to which topics[16]. When a message is published to a topic, the broker will relay it to all the clients currently subscribed to that topic[16]. MQTT is a suitable protocol for IoT devices to communicate due to being a lightweight Publish/Subscribe protocol with all the benefits described in section 2.2 [14]. One drawback with MQTT is that it runs over TCP which compared to UDP adds some latency and complexity due to requiring to set up a connection before transmitting data[14].
2.6 CoAP
RFC 7252 describes CoAP as a protocol designed for constrained devices operating on low- power lossy networks[17]. The RFC describes CoAP as realizing a subset of REST[7] common with HTTP and to have low overhead and be suitable for constrained devices. The RFC goes on to say that CoAP uses a compact binary header format and runs over UDP. Using UDP instead of TCP allows for less delay and complexity as no connections needs to be set up before transmitting data. The RFC further describes that because CoAP was designed according to the REST architecture, it is quite straightforward to map CoAP to HTTP which makes it a good candidate as an IoT protocol. A drawback with CoAP as specified in the RFC is that it is using the Client/Server architecture as previously discussed. A draft for a CoAP broker describes how CoAP could be extended to support the Publish-Subscribe
architecture[18]. Implementing this draft would remove the drawbacks from the Client/Server architecture and thus make CoAP a good alternative to MQTT as an IoT protocol.
2.7 Application programming interface
Application programming interface (API) refers to a standardized way of communicating with a service or application[19]. There are different kinds of APIs that follow different principles.
For example REST follow the principles that the URLs should be directory structured and easily understood. The API should transfer data objects or attributes in the form of Json or XML[20]. The request should use HTTP methods explicitly, such as GET, POST, PUT and DELETE[20]. It should also be stateless such that it does not store any client context on the server between requests. The client is the holder of the session state[20].
2.8 IoT data
Widely used communication protocols such as HTTP use human readable headers and are as such very spacious and inefficient in regards to packet size. Devices that together build the Internet of Things (IoT) are usually known to be small energy efficient devices with a constrained amount of resources available. The radio transmitter and receiver on the IoT devices are usually the most power hungry[21] and special communication protocols and data structures are developed to minimize the time they need to be active. To keep the transmissions few and messages small the GreenIoT project decided to use the publish subscribe protocol MQTT to transfer data from sensor nodes to a broker which relays the data. To keep the messages small they use the SenML formatting[22] .
SenML is a data structure suited for sensor nodes to encapsulate measured data and send it in small packets over a mesh network. It is designed to be efficient and fit in the constrained resources of an IoT device. JSON[23] is used to represent the data and the message is limited to containing just minimum amount of metadata to describe the measurements such as time, unit, sensor type, node identification and some additional properties.
2.9 Mobile development
When developing an application to visualize data on a mobile device attention must be given to the potential problem of information overload. As the screen is typically small there is only so much room where to show data and it is a trade-off between too much data, making it look cluttered, and too little data, making it less useful. Besides information overload, focus should also be given to context of the data. When searching for environmental sensor readings the physical location of the one requesting the data might be used to locate relevant data of that and surrounding areas instead of some place far away.
To learn about mobile application development for Android the book “Android 6 for Programmers An App-Driven Approach, 3rd Edition”[24p. 6] was used, which explains the different parts of an application in the form of eight different applications with explanations of and full access to source code. This book served as a learning foundation for everything needed to be able to create a simple Android application.
Mobile Design and development by Brian Fling[25] notes some important aspects that should be taken into consideration while developing mobile applications. Fling points out how important research is when starting a new mobile development project despite having years of
experience, as the mobile development industry is a highly competitive one, with new information being published all the time. This means that whatever information you already have is probably out of date[25pp. 51–60]. He also emphasizes how mobile developers should focus on the device most suitable for the core customer, rather than on supporting all devices.
The most suitable device might not be the one easiest to develop for[25pp. 65–66].
Fling sets weight to the idea of creating a new application specifically for a mobile device, rather than just converting an application to mobile application[25pp. 65–66]. He looks at the mobile platform as a unique medium and encourages developers to tailor the application to reap the benefits of the mobile platform rather than striving for it to look like, for example, an earlier web page[25pp. 56–66]. Fling also says that the key to mobile developing is simplicity and encourages developers to keep it simple[25p. 67].
To keep up to date with newer interfaces and design practices we used the official site for Android developers[26]. This resource was very useful as it provided documentation, APIs and examples of implementation of all Android features and was continuously used as a reference throughout the development.
A major part of the application would be to display sensor data on a map and as such we studied the official Google Maps API documentation[27]. We used the documentation to learn everything we needed to know about the API to implement and work with it in Android.
2.10 Similar applications
MQTT Dash[28] is an Android application developed by Routix software to control things via the MQTT protocol and can be used to display real time sensor value. While it offers a lot of features and customizability, it is hard to understand and everything has to be setup from scratch leaving an inexperienced user confused. IoT MQTT Dashboard[29] by Nghia TH is another Android application that can display MQTT data which is a bit less complex than MQTT Dash but with more limited functionality and which is still hard for an inexperienced user to set up. Both these applications are limited to the MQTT protocol.
Weather Underground: Forecasts[30] by Weather Underground is an Android application that visualizes weather data that is gathered from privately owned weather stations[31]. A community of weather station owners publish their weather data to this application making it available to anyone. This application offers good visualization of the sensor data and works from the start. The big drawback is that it is limited to displaying the data provided by servers from Weather Underground and can not connect to new sources such as an MQTT broker[32]
to get real time data.
Many other applications exist which can display MQTT data but we have not found a versatile application that could display sensor data coming from different sources with varying data
formats. Nor have we found an application that is easily extended to support different data structures and protocols.
3 Methodologies and Methods
This section will describe what methodologies will be used in the project to develop the android application. First the project methods will be described that will be used to structure the work. Next, the modelling methods will be described which are used to develop a model of the application to be created. Last, we will go through the testing methods to be used for verifying that the application works according to specifications.
3.1 Project methods
To develop a mobile application that is highly modular and able to collect and visualize real time sensor data the workflow will follow an iterative approach. By following an iterative approach we will start with developing a core application with limited functionality and perform testing to make sure it works. Each following iteration will develop a new feature and test it before moving on to the next to verify that there is always a fully functional application after every iteration. The alternative would be to follow the old Waterfall method[33] where all development is linear and no working product is released until the end. This method is highly inappropriate for software development due to the requirement of a full specification at the start and then continuous development until the product is finished. When developing software the specifications might be more vague in the beginning and changed mid development, requiring an agile method that is quick to respond and to adapt to changes. Our work will be agile in the core and include parts from Scrum[34] and parts from Extreme Programming[35] to improve the development process.
3.1.1 Extreme Programming
From Extreme Programming (XP) there are four concepts that are particularly useful for this project: pair programming, collective ownership of the code, a simple design and user stories.
One goal of the project is to develop an android application that is modular and easy to extend. By keeping to a simple design the application code will be easy to maintain. It will also be more readable and easier to understand allowing for future extension of the code to go smoother. The other two principles of pair programming and collective ownership of the code are well suited to our project team, where one of us has experience of developing mobile applications and the other has none. Co-writing the code will start slower but the learning process will proceed faster. Further along the project it will pay off as there is a solid foundation where both team members can contribute individually. Another aspect of it is that at the beginning there are many decisions about how to structure the code and how to solve potential problems that have to be made. This gives the opportunity to discuss and arrive at common conclusions and it helps to keep a good spirit. Pair programming will also help to promote a collective ownership and responsibility of all code as it is written together. By having a collective ownership of the code we also get the benefit of collective responsibility of
all parts of the code. It also helps to avoid the problem where everyone looks out only for themselves by only caring for the code they have written. The last principle of user stories is to write down what the system is supposed to do for the user and how to test that the behavior is acceptable. This practice will help focus the development on features that a user wants as the use cases will partially be collected from the clients. In this project the clients are our mentor and supervisor from the GreenIoT project group[36].
3.1.2 Scrum
The parts from Scrum that we will use is the idea of a Scrum board, a product backlog, sprints, daily scrum and sprint reviews. We will use an online resource at Trello[37] as a digital Scrum board where we can keep track of our individual tasks and completion status to give a good overview of the ongoing work process. The user stories from XP will be broken down into concrete tasks whose completeness can be measured. When all of them are done the user story is ready to be tested. A user story could be about reading the value of a specific sensor node and include the specific steps of how it should be done. When all the accompanying tasks are completed the user story will be moved to a testing phase. Each user story has steps defined of how to test the feature properly and in the testing phase they are followed. After it is verified that the feature is working, the user story and related tasks are changed to status
“completed”. By using user stories there is a concrete description of what a user might want to do and a way to verify that it works.
The project will be divided into sprints which are short periods of time with specific goals and tasks that needs to be completed. Dividing the time into sprints follows the overall idea to work iteratively and helps to complete a few things at a time and make sure they work before moving on to the next one. The sprint ends with a sprint review where the tasks and user stories that have been completed and any unfinished ones that might need to continue in the following sprint are looked over. It will also lift any particular problems encountered and a discussion about why they happened will follow. Doing this provides an overview on what was good in the previous sprint and allows for the possibility to take actions to improve for the next one. After every sprint the currently active tasks should be completed or moved to the next sprint.
The product backlog will be used to list all the unfinished user stories that are not actively being worked on in this sprint and will help to keep track of how much work remains for future sprints. Every morning will be started with a daily scrum which is a morning meeting where what was done the previous day and what will be done today will be gone through. This way every day always starts focused on specific things that should be worked on during the day which helps to complete all the tasks assigned to the current sprint.
3.2 Information Architecture
Modeling is a way to get a good structure and overview of the project and is helpful to keep everyone involved on the same page. It structures a clear plan of where the project is going and as stated by Fling: “The truly successful mobile products always have a well-thought-out information architecture.”[25p. 89]. This projects will be using the methods of sitemaps and wireframes.
3.2.1 Sitemap
Sitemaps is a visual representation of the structure of the application shown as the relationship between different contents[25pp. 94–95]. These contents will in our case be different views. Using a sitemap provides the developer with a chance to limit the risk of mistakes by categorizing content so that the user is greeted with fewer options at every view[25pp. 95–97]. The sitemap will be used as a visual representation of the relationships between all the views and helps divide the different parts into suitable groups.
3.2.2 Wireframe
Wireframes is the direct interaction between the user and the informational space while sitemap designs the organization of the content in information space[25pp. 101–102]. Fling points out that the wireframe should not only be seen as a visual representation of the sitemap but also show how the user would interact with the experience[25pp. 101–102]. He notes that this should be used to design a navigation and show information in a suitable way in an assumed context for the user. The project will use Wireframes to get a united view on the design of the application to strive for the same goal design.
3.3 Development environment
To facilitate collaboration the team decided to use a version control system and an Integrated Development Environment (IDE) that would be used by the entire team. This way, encountered problems with the IDE could be solved together because everyone used the same one.
3.3.1 Version control system
Git is a version control system that makes it possible to record changes on files to later be able to recall a specific version[38]. In this case these files will be code and images. The action of adding a new version to the history tree is called a commit which is a text message describing what changes have been made[39]. The commit is saved locally until the developer pushes the commit or commits to a git server[39]. Git implements a feature known as a branch, which is a version of the code with its own history tree[40]. At the start there is only the master branch. When creating a new branch the history tree is split in two and any further changes will be local to the currently selected branch[41]. This allows a developer to work on a copy of
the code when developing a specific feature without interference from other developers. When the feature is done the branch can be rejoined with the master or any other branch through a process called merging[42]. This project will use Git for version control and to divide the code into multiple feature branches that have their own goals and merge them back into the main branch when completed and tested.
3.3.2 Integrated development environment
Android studio is the official integrated development environment for android that is made for developing applications compatible with every type of android device[43]. With android studio the development team can create the application and then run it either on an emulator or directly on an android device[44]. There is also a debugger integrated to help solve any issues[45]. The project will use Android studio as its main tool to create the application because of the simplicity of running the application directly on the phones. Another reason is that it is the official developing tool for Android and will include support for the newest features available.
3.4 Testing and Evaluation
Testing the application will show how the user will interact with the application and also show potential problems in the structure. This project will be testing according to guerrilla testing, usability testing and unit tests. The product will be evaluated towards the end to verify that the goals specified in section 1.3 have been fulfilled.
3.4.1 Evaluation
Evaluation of the application is an important stage of the project. This provides a way of verifying that the developed application conforms to the specified goals. To evaluate the application, the goals are broken down into the following sub goals:
• The application should be available to individuals
• The application should be able to visualize real time data
• The application should be able to visualize historical data
• The application should be able to visualize source locations where applicable
• The application should be able to collect data from multiple sources simultaneously
• The application should be able to collect data using different protocols
• The application should be able to display different types of sensor data
Dividing the goals into sub goals allows for easier verification of conformity. It would help to ensure that the functional behavior of the application that is sought is available in the resulting product.
3.4.2 Guerrilla testing
Guerrilla testing means that the application is tested on phones on display at local operator stores[25p. 283]. These stores are common and allows the developer to test the application’s functionality on a wide range of phones without having to own them[25p. 283]. The Guerrilla testing is mainly used in this project to evaluate the behavior of the application on different screen formats without needing to own the phones.
3.4.3 Usability testing
Usability testing is a method where real users are asked to test the application[25pp. 295–
296]. This always provides invaluable information as it gives an outside perspective from someone who does not have as intimate knowledge of the application as the developer and gives feedback directly from the targeted group of users[25pp. 295–296]. It is important to attempt to make the scenario as realistic and genuine as possible as this method can often run into issues with test subjects changing their behavior to try to please or be overly critical towards the application[25pp. 295–296]. Usability testing will be used as a guideline to make the application something the target group wants.
3.4.4 Unit tests
Unit tests[46] are small code fragments used to test the logic in an application. These are organized in a pass and fail structure[46]. It enables the developer to easily test if the latest changes might affect any of the already existing functionality in an unwanted way[46]. If that is the case, the developer can use Unit tests to locate the unwanted behavior and save debugging time. The project is planning to use JUnit[47] test as a way to verify that the application is working before every push to the main branch of the git repository. This forces the developer to fix any issues before the code gets accepted. By doing this we will be assured that the latest code on the main branch is in a working state.
4 From idea to application
The application was planned to be developed during a period of six weeks that encompassed six sprints with the length of one week each. Testing of the application was done continuously during development. Usability tests were used to test the new features added but also to get a sense of the application as a whole. The logic of the application was tested by automated Unit tests and the visual parts of the application could be tested by the Guerrilla testing method when the application hit beta stage. This is because the application should be able to be installed on a device in a store without a certain account connected to it which must have an invite to the application.
4.1 Sprint 1
The first Sprint of the the application development had the goal to build the core structure, designing how the user would navigate the application. During this Sprint the goal was to provide a Wireframe and Sitemaps that shows the structure of the application and can be used further throughout the project as a visual guideline. The design models were tested and changed until the product owner was satisfied. All the design documents that were developed in this sprint were living document that were updated along the way.
4.1.1 Sitemap
Figure 5: The resulting sitemap from the first sprint
Figure 5 shows the first version of the sitemap and a decision was made to have three different types of visualization of the sensor data. One in the shape of markers on a map that displays an overview of where the sensor nodes are located together with the most recent sensor values from every location. The second representation is in the shape of a list that show all sensor nodes. The list representation does not require the sensor data to have a location and can therefore include nodes that the map can not. Touching a list item or a map marker redirects to a detailed view of the particular sensor node which is the third visualization. It contains a line graph with the recent history of this sensor node of the specified type. Addition of other representations such as a bar-diagram was considered but put aside because it was concluded that it would not provide any new information for the user.
4.1.2 Wireframe
The first idea was to display all sensor types with the most recent values on the map. However, the idea was quickly rejected as it might overwhelm the user with too much information.
Instead, it was decided to show only nodes of a certain type in each view. The user can easily change sensor type from the options menu located in the top right corner of the screen as seen in Figure 6, on phone number five. To change from list representation to the map
representation and the other way around the user presses the top left menu also seen on phone number four. From the menu the user also has the option to go to settings. The settings page was planned to be used as a view to add new connections to brokers of the MQTT and CoAP[17] protocol.
Figure 6: The resulting wireframe from sprint one
4.2 Sprint 2
The Second Sprint had the goal of creating the application’s data structure. Unlike the first sprint that designed the visual representation of the application, the second one created models of the application’s classes, connection and database structure.
4.2.1 Database
The application will store all its values in an Sqlite[48] database stored on the phone. The idea was that the system would create the structure at the start and then add the standard sensor types and locations specific for the GreenIoT project. This would only occur during the first run of the application and would save the user from having to add these locations to the nodes by showing data on the map directly. The alternative would have been to not pre-configure any sensor types or location and leave it to the user to configure as desired. However, pre- configuring them helps inexperienced users to get started and lowers the required technical expertise to use the application. This makes it suitable for both advanced and novice users.
The message structure in the database (seen in Figure 7) was designed to try to save everything of importance and skip the rest. This decision was made to shift the focus from what extra features one source could provide and aim for the most necessary and useful parts
Figure 7: Showing the sensor message entity relationship in the database
of the application. The message itself has the sensor value, timestamp and a unique id that is automatically incrementing. If a message is received without a timestamp the current time should be used. Each message should come from a node, a node that has some kind of identification label as its name. Each node has a location with a name and optionally GPS coordinates. The location coordinates can be updated or set from the settings view which will update the database. Separate organizations might use the same id to identify a sensor node.
To differentiate between two nodes with the same id, the organization name is combined with the id to create a unique identifier, requiring that the id is unique within the organization. The alternative would be to only support unique ids such as the device’s mac-address but that would set an unnecessary limit on what can be an id. If a message is received for a node without an organization specified, the connection URL will be used as the organization name.
The connection database structure (seen in Figure 8) was designed to support MQTT but not limit the system to have the same structure for another protocol. MQTT needs a subscription topic to select which messages it should listen to but a REST API connection does not. Instead of having an MQTT specific field for subscription topic, a field arg0 was created instead. This serves as an optional value that can be used for all connection types. The data structure and topic structure is how the received values are handled. An MQTT message comes with information in the topic and thus the topic structure is needed to retrieve it. In the case of a Figure 8: Showing the connection entity relationship diagram in the database
REST API connection, the topic structure will contain information used to retrieve all the supported sensor types.
4.2.2 Activities and fragments
The application consist of activities which are ultimately responsible for what is displayed on the screen. Similar content was grouped to the same activity leading to one activity handling the settings menu, one for the part of the application showing a map and one for the sensor list. Each activity contains one or more fragments which in turn draws content on the screen.
As seen in Figure 9, the SettingsActivity contains multiple fragments and each fragment represents a specific view. The LocationFragment is responsible for a view where the user can update the coordinates for a location and the MQTTFragment is responsible for updating or creating an MQTT connection. They are both classified as modifying settings and thus belongs to the SettingsActivity. Some behavior such as the creation of the drawer menu or back button was identical across all activities. To avoid duplication of the code a base activity was created which all other activities extended.
4.2.3 Background service
For the application to be able to subscribe and store the data from a broker the application always needs to listen. Having this thread on one of the activities would result in it restarting every time the activity was changed. The decision was made to create a background
Figure 9: The activity and fragment structure of the application
service[49] that would listen and then store the data in the database. To store only a specified amount of sensor values in the database, a cleanup service was created. A thread that starts in the beginning of the background service periodically removes any sensor values from the database that exceeds the specified threshold. The threshold specifies a maximum amount of messages for each sensor node for each sensor type. For example, if a node has twenty sensor types with one value each, the thread would not delete any records unless the threshold was zero. For the application to access this data and always display the latest sensor readings, the base sensor activity will retrieve them from the database every ten seconds. The system will use a timestamp to limit the data retrieved to make sure that the same data is not retrieved twice.
4.2.4 Message retrieval
A problem that occurred early on was incoming messages with unknown sensor types which were not supported. Either the message could be ignored, or the data type could be automatically created and all the values connected to it and saved in the database. A third alternative would be for the sensor type to be automatically created but the sensor values ignored. It was decided that the system should add the sensor type but not the data because it would save memory and computing power on the limited capabilities of a mobile device. A supported flag was added for the sensor types to indicate if data should be stored or not and automatically created sensor types would default to unsupported. To only show the sensor types of interest for the user the sensor types needs one additional flag which was called active. This flag would decide if a sensor type would be shown in the application. If a sensor type is flagged as active and supported the user can choose to display data of that type in the list and map representation.
When retrieving a new message the retrieval logic would first check in the database if the sensor node has been encountered before. If it is a new one, it would be created. The SenML structure supports multiple sensor values and sensor types in the same message[22]. First the node information is taken out, then it loops through the sensor data and adds it to the database according to type. In the case of an unsupported or unknown sensor type the data will be ignored as described above. The path each message needs to take can be seen in Figure 10.
4.2.5 Limit the working data
To limit the data stored in the application and to avoid needing to override the current data list used by the system every time more data was retrieved, a special data structure was needed. When new values are added to the list the oldest ones should be removed if the list size exceeds the threshold. The circular queue structure worked as a solution to the problem and each sensor type node class could hold one circular queue.
4.2.6 Sensor limits
To give the user some context to the data shown on the map, limits were added to all sensor types. The limits would be there as visual help that would give the markers on the map Figure 10: Event chart of received message
different colors depending on how the sensor value correlated to the minimum, medium and maximum values for that particular sensor type. If the limits are not specified for a sensor type they default to zero.
4.2.7 Topic data retrieval
When using MQTT for the connection in this project some of the data is stored in the topic[6].
That data will be retrieved from the topic by using regular expressions with capture groups. To specify which capture group contains what information there will be a need for variables to store that information. These would be stored in the database connected to the topic structure itself (seen in Figure 8).
4.2.8 Data structure support
One of the goals of the application was modularity and to easily being able to add new sensor data structures. There is one class that is responsible for mapping incoming data formats to the data structures used in the application internally. The mapping will be done with a switch statement to map the name of a data structure to a conversion method. To add support for a new data format the developer could first add a database entry in the data structure table with a name for the new structure. Then what is left is to extend the switch statement with the name of the new data structure and map that to their own converter method. When a connection is created the new data structure can be selected which would call the newly added converter when the data type name is retrieved. The converter is given the message in the shape of text as an argument. By sending the message in text format to the converter, the message could be of any structure. The implementation design could support JSON, XML[50], text or even a self-written structure as long as the developer could convert the data into the internal structure used in the application.
4.3 Sprint 3
The base of the android application took its shape in the third sprint where it was usable with one connection. The sprint ended with a release through google play store in alpha stage and by invite only [51]. This facilitated testing of the application on other devices as they did not have to enable developer options and install the application by file or by connecting to a computer [52].
While testing the application a new problem presented itself related to visibility of the sensor readings drawn on the map. If multiple map markers are located in close proximity the values will be written on top of each other making them hard to read. To solve this problem a map marker cluster could be used[53]. The advantage with this was if two markers are too close to each other they would merge into one. The new value that was shown was the average of the two. Zooming out would result in more and more sensor nodes merging, allowing the user to see sensor data more clearly over larger areas. This new merging ability resulted in a new
problem that needed to be solved. What should happen if one of these cluster was clicked. One alternative could be to take the average of all values and display them on a graph, but that has potential for other problems. The values from all the nodes might not have been read at the same time and the average would depend on how many values was given for that particular time. The solution became to go to the sensor list and list all the sensor nodes that the map marker cluster contained instead.
In the Wireframe seen in Figure 6, on phone 3 and 6, a green light was used to indicate the status of the sensor. The idea was that the light would be green if the sensor was online and red if it was offline. The idea was rejected and the light was replaced by a display of the latest value. The reason for rejecting the status light was due to the fact that there was no good way to determine if a sensor was offline or if a packet had been lost.
4.4 Sprint 4
The fourth Sprint focused on test data. During the project we had one connection with incoming data from the GreenIoT project that we could listen to but the number of messages using the correct sensor structure were few. To test the application with more than two sensor nodes present on the map we needed to generate fake messages or get the date from another source.
4.4.1 Generated data
The first decision was to start with generating fake messages containing random values because it was a fast and easy approach. A disadvantage for the implementation was that the data for all sensor types was generated using the same pseudo random function with an interval of 0 - 100. This gave unrealistic data both because readings could take too big of a leap in a short time and because the sensor types allow different ranges of values.
Temperature data allows for negative numbers when using Celsius and humidity readings can only be between 0 - 100. The fake data generator could have been made more advanced to solve the problems but having easy access to real sensors, it was decided that real data would be more useful than fake.
4.4.2 Sensor data from the residence
Two additional sources were added, each one connected to a residence. In each residence the new sensor data came from two different kind of sources: wireless 433MHz temperature sensors[54] and moisture sensors connected to WiFi enabled NodeMCU development board[55] as seen in Figure 11. The 433 MHz sensor was connected to a Tellstick Duo[56] to retrieve the sensor values and a script was written to bridge it to MQTT and publish to a broker[57]. The NodeMCU development board was programmed to periodically take sensor readings and publish them to an MQTT broker[58]. The application could then connect to that MQTT broker to retrieve values from all the sensors.
4.5 Sprint 5
Going from one source to many made it possible to visualize not only the GreenIoT data but also the data from the residences created as well in the same view. The fifth sprint focused on mashing all the different data from different sources into one visualization. The goal here was to be able to add new sources and modify existing connections.
Figure 11: Residence sensor test data collection
Having more than one source meant that the messages from a node might be retrieved by the application from multiple sources with different timestamps. This means that the circular queue that was used earlier did not work anymore. If there was an older value received than the oldest one in the queue, it could not be positioned at the front of the queue due to limitations in the implementation. A structure was needed that was similar to a circular queue with respect to its predetermined size, but when queuing it would also sort the messages by a timestamp. The oldest message would be removed if the queue’s length exceeded the maximum length. An example upon the wished behavior can be seen in Figure 12. No existing solutions was found which led to the development of a self-written implementation using a linked list that would search after the correct position and then add the message. If the queue length exceeded the threshold it would remove the sensor value in the front.
Figure 12: Enqueuing process of sensor values
4.6 Sprint 6
The last Sprint goals were to add an API connection that would be similar to the modular broker engine and also to make it possible to make a sensor type into an alias for an already existing sensor type. During the development we encountered situations when the same sensor types were named differently. The application did support this and treated them as separate types leading to, for example, temp and temperature values being displayed on separate views.
4.6.1 API
To limit the calls to the API we decided to only ask sensor data from the currently active sensor type. The API structure used for a particular source might not implement all possible requests and what is supported might vary. One API might have the option to ask for the ten latest values, another only the request for messages after a certain timestamp and some might have implemented the option to have both in the query. To follow the goal of modularity the API calls for sensor data uses an interface. Each API call structure is defined in a separate class which extends the interface such that a standardized request from the application is translated to match what is required by that API. The interface has also got a parameter for when the last request was made and how many messages can be stored locally to customize the API request where applicable. It is up to the person creating the support for an API to use these in the best suitable way for the structure they are creating. Even if the API call would result in more sensor values then the queue can hold, the exceeding ones would simply be removed and only the most recent values remain.
With the API the smartphone does not get callbacks with new messages and can therefore not make an easy sensor type discovery. To get all the sensor types periodically, with a default period of 10 seconds, seemed a bit much. A decision was made that the call to retrieve all sensor types would be put in the background service and would be run at the start of the application. This would retrieve all sensor types from all API sources and then add them in the same way that the broker types were added.
The resulting interface that all the API structures would need to follow requires a way to get all sensor types, a way to retrieve sensor of a certain type and also more information of one specific sensor node.
4.6.2 Type alias
A problem we encountered was that of the same sensor types being reported with different names such as temperature data being submitted by some sensors with the name temperature and by others with the name temp. The difference in name led to them being handled as separate types and thus not combined in the visualization. As our application is built to
display one type of sensor values at a time this led to an unintuitive separation of temperature data to different views. To solve this problem a new feature of type aliases was developed where a user can associate aliases with a sensor type to allow sensor data with different names to be handled as the same type. As an example the temp data can be associated with the alias temperature thereby allowing them to be combined and represented as one sensor type.
For every sensor type there is also an alias with the same name so with the sensor types temperature and temp there are also the aliases named temperature and temp. At the start there is only one alias per sensor type and it has the same name. As shown in the Figure 13 an alias can be updated to be associated with another sensor type and thereby disconnecting it from the previous one. The circle named temperature represents an alias previously associated with the sensor type temperature but now associated with temp instead. If a sensor type has no aliases associated, it will automatically be removed as illustrated by the dashed line for the sensor type named temperature. There is also an option to release an alias which will remove the current association and re-associate with the original one and recreate that sensor type if it does no longer exist.
When adding new sources and new sensor types there might be times when certain sensor types are detected that are not going to be used or that are renamed. To prevent the sensor type alias list in the application from listing sensor type aliases that it detected once but will
Figure 13: Illustrating alias - sensor type association and re-association
not detect again, a feature was added to delete an alias. This will also automatically delete the sensor type if no other aliases is currently associated. If sensor data is received using the name of a deleted sensor type both the type and alias will be recreated and associated with the new sensor readings.
4.6.3 Updating sensor types
When a new sensor type is detected it is automatically added but with the detected name only.
There is no unit associated with the sensor type or threshold values also known as limits to help categorize sensor readings as low, medium or high. A new setting was added to allow the sensor types to be updated such that a unit can be set which will be displayed next to the values and limits which are used for visualization on the map. If a sensor type is recreated following the release of an alias it will be missing the unit and limits which can then be updated manually. There is also the option to change the name of a sensor type. This is the display name that will be shown in the application to represent the data type but not what will be matched against for incoming data.
4.6.4 Improvement of activity structure
The map view and sensor list view are very similar and intertwined as seen in (Figure 9, The activity and fragment structure of the application) and should be handled by the same activity.
It was decided that it would be better to merge the sensor map activity and sensor list activity into the base sensor activity. The reason as to why they were separate to begin with was that it allowed the views to be changed differently depending on which activity that was in control.
At that time, starting an activity with a variable was unknown to us.
Figure 14: New activity and fragment structure
The improvement to be gained by having it in the same activity (seen in Figure 14) is that the data would not have to be reloaded between the map and list view. The settings activity handles all the views for reading and updating settings which means that all the settings fragments can use the activity to retrieve data from the database. Switching between views and fragments in the same activity does not wipe the data which is the case when changing activity.
The improvement was accomplished by merging the two activities into the BaseSensorActivity. Variables were used as arguments when switching between views to get separate behavior depending on if the switch was made from the map view or the sensor list view.
5 The Application
The six sprints resulted in the application Smart City IoT[59] available on Google Play Store[60]. Below the functionality of the application will be shown and explained.
5.1 Start screen
Figure 15 shows the start screen of the application which is displayed while the services are starting up. It is available for multiple screen resolutions to avoid possible distortions from stretching to fit other screens.
Figure 15: Start screen
5.2 Map visualization
Figure 16 shows the map view of the application where the user can see sensor data visualized on the map tied to the sensor location if available. Humidity is currently selected and the latest values are displayed. Sensors without a location is not shown in this view. Figure 17 displays the options menu on the map view where the user can select another data type to be displayed on the map.
Figure 16: Map view with
Humidity selected Figure 17: Options menu on Map view
5.3 List visualization
The list view as seen in Figure 18 displays all connected sensor nodes and their data of a specific type which can be changed in the options menu in the upper right corner. This view contains all sensor nodes including those without GPS coordinates. Only the most recent sensor readings is displayed for every sensor.
Figure 18: List visualization of sensor readings
5.4 Individual visualization
Figure 19 displays the detailed view of a specific sensor of a selected type. In this case temperature is selected and this shows the recent history of a newly added sensor node and the latest value.
Figure 19: Detailed view of a sensor displaying historic data of specified type
5.5 Add sources
Figure 20 displays the settings view with all the existing connections. From this screen new sources can be added and existing connections can be modified. They can also be toggled between active and inactive.
Figure 20: Settings view where data sources can be added and modified
5.6 Update sources
Figure 21 displays the MQTT connection view where new ones can be created and existing ones be updated. In the figure there is a preconfigured connection for the GreenIoT MQTT broker using the project specific topic structure and SenML data format. Figure 22 displays the REST API view where a new connection can be created.
Figure 21: MQTT connection view
Figure 22: REST API connection view
5.7 Update location
Figure 24 displays the settings location view where all sensor locations are listed. All locations with GPS coordinates are grouped allowing for easy detection of the ones missing coordinates.
Clicking on a location item enters the update location view displayed in Figure 23. This view allows the GPS coordinates to be updated or set for a specific location.
Figure 24: Settings location
view Figure 23: Update location
view
5.8 Update sensor types
Figure 25 displays the sensor type list view where all detected and preconfigured sensor types are listed. Figure 26 displays the update sensor type view where all the attributes of the selected sensor type can be updated. An automatically discovered sensor type will only have the name specified and can be updated with relevant unit and thresholds from this screen.
Figure 25: Sensor type list view Figure 26: Sensor type update view
5.9 Update sensor type aliases
Figure 28 displays the sensor type alias list with all discovered and preconfigured aliases.
Figure 27 displays the Sensor type alias update view where an alias can be associated with a new sensor type. The alias “temp” is currently used for the sensor type “temperature”. From this view the alias can be updated to be associated with a new sensor type, released or deleted.
Figure 28: Sensor type alias list Figure 27: Sensor type alias update view
5.10 Change active sensor types
Figure 29 displays the active sensor type list where all the supported sensor types are listed.
This view allows the status of a sensor type to be switched between active and inactive.
Switching a type to inactive will remove it from the options menu in the map and sensor list views.
Figure 29: Active sensor types list
