Mobile Client for SMART Reaction

(1)

M A S T E R ' S T H E S I S

Mobile Client for SMART Reaction

Zaheer ul Hussain Sani

Luleå University of Technology Master Thesis, Continuation Courses

Computer Science and Engineering

Department of Computer Science and Electrical Engineering

(2)

Mobile Client for SMART Reaction

Zaheer ul Hussain Sani

Luleå University of Technology

Department of Computer Science and Electrical Engineering Division of Software Engineering

June 2007

Supervisor: Kåre Synnes, Ph.D.

Luleå University of Technology

(3)

Today users often do not provide suggestions or feedback for the environment around them because of uncertainty on who to contact, where to contact, time and effort required to participate actively. SMART system is intended to provide solution to facilitate users to easily recognize things and the objects around them for their suggestion or feedback.

This thesis presents Mobile Client for the SMART system with focus on employees where they can participate actively in an industrial context, which is the replacement of manual system. Finding feasible way to annotate an object in an industrial context is one of the major tasks of the thesis work.

Mobile Client identify objects by different ways, it can be a unique identification number of an object, name of object or by looking at the latest suggestions provided by other employees. Mobile Client provides many ways to react and provide suggestions to facilitate employees. It provides access to native resources of mobile terminal within the Mobile Client to add interactive suggestions. A suggestion can be provided by one of or all of the available options, which are Text, Audio, Video or Image capturing. Mobile Client submits the suggestion using internet, which is further available to other employees; they can view and provide their suggestion or feedback.

Mobile is a low memory device and causes application crash on adding too much media elements while suggestion. Client has been observed for platform independency and figured out the problems that are faced while deploying this system on other mobile terminals having different platform.

Test results have shown that the system provides quick and easy solution for SMART reactions. System outcome is satisfied and can provide smarter solution for the different situations. It can be enhanced by providing more options and features in the future.

(4)

1. Introduction ...1

1.1 Brief... 1

1.2 Background ... 2

1.2.1 Current System...2

1.2.2 Previous Work ...2

1.3 Feasibility Study... 2

1.3.1 MPS Positioning ...3

1.3.1.1 Request format for Telia MPS...3

1.3.1.2 Analyzing Telia MPS Service...4

1.3.1.3 Conclusion ...6

1.3.2 GPS Positioning ...7

1.3.2.1 Bluetooth GPS Receiver...8

1.3.2.2 Analyzing Bluetooth GPS Receiver...8

1.3.2.3 Conclusion ...10

1.3.3 RFID ...11

1.3.3.1 NFC...11

1.3.3.2 Nokia 6131 NFC ...11

1.3.3.3 Conclusion ...12

2. Related Work...13

2.1 Nokia 3220 Bus Pass... 13

2.2 Taxi Tracking Service ... 14

2.3 Buy Movie Tickets Using RFID ... 14

3. Design and Architecture ...16

3.1 Use Case Diagram... 16

3.2 System Architecture ... 17

3.3 Activity Diagram... 18

3.4 System Design Technology and Motivation ... 19

3.2.1 J2ME ...19

3.2.2 PHP ...19

3.2.3 XML...20

3.2.4 MySQL ...20

4. Implementation ...21

4.1 Development Environment ... 21

4.1.1 Development Kits ...21

4.1.1 Tools ...21

4.1.1.1 NetBeans IDE 5.0...21

4.1.1.2 NetBeans Mobility Pack 5.0...21

4.1.2 Emulator...22

4.1.2.1 SDK for Nokia 6131 NFC ...22

4.1.3 Mobile Terminal ...23

(5)

4.2 Mobile Client... 23

4.2.1 Object Selection ...24

4.2.1.1 Responz ID ...24

4.2.1.2 Responz Object...24

4.2.1.3 Latest Suggestion ...24

4.2.1.4 Open Suggestion ...25

4.2.1.5 RFID Tag Detection...25

4.2.2 Add Suggestion...26

4.2.2.1 Audio Capturing...27

4.2.2.2 Video Capturing...27

4.2.2.3 Image Capturing ...28

4.2.3 Submit Suggestion ...29

4.3 Web Server ... 29

4.3.1 Get Objects...30

4.3.2 Get Latest Suggestions...30

4.3.3 Insert Suggestion...31

4.3.3.1 Insert Text Data ...31

4.3.3.2 Insert Media Data ...31

5. Testing and Evaluation...32

5.1 Test Cases... 32

5.2 Test on Nokia N91 ... 35

5.2.1 Audio Capturing...36

5.2.2 Video Capturing...36

5.2.3 Image Capturing...37

5.3 Evaluation... 37

6. Conclusion and Future Work ...38

6.1 Conclusion... 38

6.2 Limitations and Known Issues ... 38

6.3 Future Work ... 38

References ...40

(6)

Figure 1: Mobile Positioning System ...3

Figure 2: Google Earth Image X, Y co-ordinates and out radius ...6

Figure 3: GPS satellites around the earth orbit ...8

Figure 4: Google Earth image indicating Bluetooth GPS Receiver results ...10

Figure 5: Nokia 6131 NFC ...12

Figure 6: Nokia 3220 uses NFC as bus pass...13

Figure 7: KTF GREEN TAXI Service...14

Figure 8: Get movie tickets using RFID tag ...15

Figure 9: Use Case Diagram - Overview...16

Figure 10: Use Case Sub-Diagram - Add Suggestion ...16

Figure 11: Use Case Sub-Diagram - Add Media Elements ...17

Figure 12: Mobile Client for SMART Reaction, System Architecture ...18

Figure 13: Activity Diagram...18

Figure 14: Nokia 6131 NFC Emulator...22

Figure 15: Mobile Client Main Screen ...25

Figure 16: View Suggestion Screen...26

Figure 17: Add Suggestion Screen ...28

Figure 18: Suggestion Submission Details ...29

(7)

Table 1: Add Suggestion by Responz ID Test Case ...32

Table 2: Add Suggestion by Responz Object Test Case...33

Table 3: Add Suggestion by Viewing Latest Suggestion Test Case...33

Table 4: Audio Recording, Playback and Delete Test Case ...34

Table 5: Video Recording, Playback and Delete Test Case ...34

Table 6: Image Capture and Delete Test Case...35

Table 7: Submit Suggestion Test Case ...35

(8)

All praises to Almighty ALLAH, the most merciful and compassionate, who enable me to complete my thesis work.

I would like to extend my heartfelt gratitude and appreciation to my family especially my loving parents, whose support and encouraging words made me sail smoothly through my difficult times.

I thank my supervisor Kåre Synnes and my project manager Johan E. Bengtsson for their invaluable advices and for their subtle directions of my efforts throughout the implementation of my thesis. Thesis work could have been very hard without their motivation and encouragement throughout my work.

Special thanks go to my fiancé. Without her continuous support and motivation it was really impossible to complete my thesis work.

And lastly, I thank to my friends who always wished me best of luck in my thesis. I am indebted to all those not mentioned, who have contributed in completing my thesis within the allocated timeframe.

Zaheer ul Hussain Sani May 2007

(9)

Thesis is Dedicated To My

Loving Parents and Fiancé

(10)

1. Introduction

1.1 Brief

Suggestions and feedbacks are important for growth, stability and improvement of any organization or an industry. People want to give suggestion and feedback for the things that are around them in daily life. Either it is on the way on cycling track or outside their house, when they see something need to be fixed, they want to react. But today users often do not provide suggestions, opinions and alarms, due to uncertainty on who to contact, and the effort needed to take active part. SMART system is to explore the concept of the "reaction media", allowing individuals to engage themselves and take active part in many situations.

Thesis work is to develop a SMART Mobile Client (Java Client) with focus on use by employees in industrial context. It includes study of different technologies that can help in object detection. These possible technologies are:

• Mobile Positioning

• GPS Positioning

• RFID

Mobile Client also provides different options to give suggestion for a SMART object.

These options are:

• Text

• Audio

• Video

• Image

Mobile Client submits suggestion to the system and that is available for other users using Mobile Client or Web Client.

(11)

1.2 Background

1.2.1 Current System

Within an industry in Sweden, there is a manual system for responding to an object. If employee wants to give suggestion or feedback to any object, he has to take a form from a specific department, fill that form with the object information (like id, name, location etc) and then submit that form back to that department.

Sometimes employees do not provide feedback for objects due to that much long process and effort. Moreover, employees have to explain everything with textual information and have no option to provide any interactive information (audio, video, image etc).

1.2.2 Previous Work

SMART Mobile Client was previously developed and implemented on Nokia N91 mobile terminal. It was developed in J2ME and contains following features:

1. Display list of all SMART objects 2. Create new SMART object 3. View SMART object details

4. Record and playback audio in PCM format (baud rate: 8000, bits = 8)

5. Provide suggestion for any SMART object by entering text for it, audio recording or both.

6. Suggestion could be stored or submitted. Store means, it will not be viewed by other users, only viewable by sender and administrator. Submit means, it will be available for all other users.

7. Submitted suggestion is viewable from the web client. Web client can also play audio that was submitted by the mobile client.

1.3 Feasibility Study

Best possible technique for SMART object detection is discussed and analyzed. On the basis of conclusion, it has been decided that either studied technology is feasible

(12)

1.3.1 MPS Positioning

Mobile Positioning System (MPS) provides the Location Bases Services (LBS) in which Mobile Positioning Center (MPC) is the gateway between mobile network and the location depended services.

Mobile Position Protocol (MPP) defines a URL that is used by any location dependent application for sending request to the mobile station. In response, MPC reply with the estimate position of mobile terminal.

MPP define some set of parameters on the basis of those, estimated location of the mobile terminal is replied by MPS. Those parameters are username, password, service id and Msid (mobile numbers) of the mobile terminals to be queried.

MPP replies with the set of information elements, which contains area where mobile terminal is located. Response time is usually 2 to 5 seconds depending upon the traffic load on the server. [1]

Figure 1: Mobile Positioning System

1.3.1.1 Request format for Telia MPS

This is the request format that Telia use, to get the position of a mobile terminal. In parameters, service_owner is the registered account id/name, password parameter is the password to access the service and service_id is an id to access the service, it could be the mobile number of the service owner. Msid is the phone number of the mobile terminal which position is needed to retrieve. [2]

<?xml version="1.0" encoding="ISO-8859-1" ?>

<!DOCTYPE svc_init SYSTEM "MLP_SVC_INIT_300.DTD">

<svc_init>

<hdr>

(13)

<id>service_owner</id>

<pwd>password</pwd>

<serviceid>service_id</serviceid>

</client>

</hdr>

<slir>

<msids>

</msids>

</slir>

</svc_init>

1.3.1.2 Analyzing Telia MPS Service

Position of a mobile terminal and its accuracy was analyzed in this test. MPS service of Telia was used to get the estimated position of mobile terminal in Luleå.

XML is formed and sent to the Telia server through HTTP request and a response was received in the form of XML. Request sent and received using Java.

In XML, Service Owner and the Password was supplied to access the MPS service.

Service Id is the same as Service Owner Id in Telia. Dummy values are written here instead of actual values for these three attributes. A subscribed mobile phone number was given as Msid.

<!DOCTYPE svc_init SYSTEM "MLP_SVC_INIT_300.DTD">

<svc_init>

<hdr>

<pwd>xxxxxxxx</pwd>

</client>

</hdr>

<slir>

<msids>

</msids>

</slir>

</svc_init>

Following response received in the form of XML from Telia server containing position of the requested Msid. Received XML contains <msid> tag that contains the mobile terminal number. <time> tag contains the position time in UTC. <shape>

(14)

shows the shape of position, it could be the circle, ring, circle sector or circle sector arc. In this case, its circular arc area. It also contains latitude and longitude values in representation of <X> and <Y> respectively. [2]

<!DOCTYPE svc_result SYSTEM "MLP_SVC_RESULT_300.DTD">

<svc_result ver="3.0.0">

<pos>

<pd>

<shape>

<coord>

</coord>

</CircularArcArea>

</shape>

<lev_conf>80</lev_conf>

</pd>

</pos>

</slia>

</svc_result>

Response returns the shape of the cell as Circular Arc Area with in radius, out radius, start angle and stop angle values. Values shoes that start angle of arc is 14 and stop angle is 120 and the out radius is 2758 meters. It has been observed that the circular arc area returned by the Telia MPS is quite big i.e. 2758 meters. In that much big area, it’s very hard to locate the nearby objects that are near by mobile terminal.

Following image shows the actual position and estimated position by Telia MPS.

(15)

Figure 2: Google Earth Image X, Y co-ordinates and out radius

1.3.1.3 Conclusion

MPS result tells that it does not provide accurate mobile terminal position. Difference is too much. It might be useful for outdoor purposes where SMART objects are far from each other.

For industrial context, SMART objects are very close to each other and it’s very hard for a user to annotate a SMART object. Moreover, even MPS provide exact and accurate mobile terminal position; it does not provide the level or height of mobile terminal from ground. In industrial context, there are different stories in an industrial building, its almost impossible for MPS to determine that in which floor the user is right now.

(16)

So, it is concluded that MPS is not very useful to implement Mobile Client for SMART reaction in an industrial context. However, it could be useful for the outdoor SMART objects.

1.3.2 GPS Positioning

Global Positioning System (GPS) is fully functional Global Navigation Satellite System (GNSS) to get the receiver’s location, speed and the direction. It is made up of 27 satellites (24 operational, 3 reserved if any of 24 stop functioning) that are place into the world orbit by the U.S. Department of Defense. To use GPS, user doesn’t need to go through any subscription procedure or pay any charges.

GPS satellites circulate around the orbit of earth and transmit signal information to the earth. That signal information is received by the GPS receivers and processed according to the type of the GPS receiver. For good results, GPS receiver must locked with at least 4 GPS satellites. When the GPS receiver finds 3 satellites, it calculates receiver’s longitude, latitude values and tracks the movement; it is also called 2D position. On finding 4 GPS satellites, it can calculate longitude, latitude and altitude;

it is also called 3D position. When user position has been calculated, it can also tell other information; like speed, movement time, stationary time, distance, time etc. [3]

(17)

Figure 3: GPS satellites around the earth orbit

1.3.2.1 Bluetooth GPS Receiver

It is a small device that has GPS receiver and built-in Bluetooth to connect with other devices. Bluetooth GPS Receiver can connect with any device through Bluetooth connection; it can be the mobile, pc or any other device to display the GPS information. There are many companies which are developing Bluetooth GPS Receivers.

1.3.2.2 Analyzing Bluetooth GPS Receiver

GPS Track is an open source J2ME application that installed on Nokia N91 and the socket Bluetooth GPS Receiver (socket is the brand name which is known for their Bluetooth devices [4]) used with this application to display the GPS information. GPS Track can be used on any mobile that support J2ME and Bluetooth API (JSR-82). [5]

We need to install the GPS Track application on mobile and turn on socket Bluetooth GPS Receiver. It takes around 8 to 10 minutes for Bluetooth GPS Receiver to find satellites. When GPS Track executed on mobile, it search for Bluetooth device, once connected, it shows the GPS information of Bluetooth GPS Receiver.

(18)

When connection established and 3 to 4 satellites found by Bluetooth GPS Receiver, it displayed the GPS information including, speed, time, longitude, latitude, altitude, number of satellites found etc. Longitude and latitude values are useful for the implementation of our Mobile Client.

GPS information that was of our concern has been observed. Observed values are:

Latitude: 65.6171 Longitude: 22.1366

Actual position of the Bluetooth GPS Receiver device and the position information received by GPS satellite are slightly different when these values observed using Google Earth. Position difference was only 0.01 miles which is shown by following Google Earth image: [6]

(19)

Figure 4: Google Earth image indicating Bluetooth GPS Receiver results

1.3.2.3 Conclusion

GPS is mainly used in cars and outdoor for the positioning and path finding. It is also helpful specially for positioning purposes. As a result of Bluetooth GPS Receiver analysis for the position, there is not such a big difference between the actual and the satellite positions. Moreover, Bluetooth GPS Receiver works well for outdoor usage.

While using indoor, sometimes it loose the satellites and sometimes it doesn’t work in the basement.

Actual Bluetooth GPS Receiver and Mobile Terminal Position

Position Information Received by GPS Satellites

(20)

One of the major drawbacks is that it takes some time to start working and finding the satellites. If we implement Mobile Client using Bluetooth GPS Receiver for the industrial context where employee has to respond for SMART object then each employee must have the receiver. That will cost much and not a smarter solution for SMART Reaction system.

1.3.3 RFID

Radio Frequency Identification (RFID) is a small electronic device that consists of chip and the antenna. Small chip can carry 2000 Bytes of data or less. RFID device is scanned to retrieve relevant information from RFID Tag similarly as it is retrieved from bar code, magnetic tape of credit card or ATM card.

RFID is not precisely positioned to scan it as bar code is scanned relative to the scanner and positioned properly or like credit card that is swiped through a special reader.

RFID devices work within few feet from scanner depending upon the device frequency. RFID is not so much common in the industry because of the lack of standards. [7]

1.3.3.1 NFC

Near Field Communication (NFC) is a wireless technology that is based on RF Technology. NFC doesn’t require line of sight. It is a short distance technology and works at 13.56 MHz frequency. It works at shorter distance that is less than 20 cm with approximate data exchange rate of 1 Mbit depending upon the compatibility. [8]

1.3.3.2 Nokia 6131 NFC

Nokia announced its new NFC integrated mobile on 08 January, 2007. Nokia 6131 NFC supports Contactless Communication API (JSR 257) for 3^rd party NFC application development. It supports lots of other Java APIs that are useful for mobile application development. [9]

(21)

Nokia 6131 NFC can act as a tag or reader/writer. It also works with other features like, touch a poster to get information, touch an image frame to transfer image via Bluetooth, touch a printer to print file, write tags etc. It supports only specific RFID tags for reading and writing. Tag detection works in range of 0 to 5 cm with the frequency of 13.56 MHz. [10]

Figure 5: Nokia 6131 NFC

1.3.3.3 Conclusion

Nokia 6131 NFC gathered RFID and NFC technologies together for 3^rd party NFC application development. Although, RFID has different standards and Nokia 6131 NFC is compatible with only specific standardized RFID tags, even then it is possible to follow single standard to implement J2ME application.

For industrial context, SMART objects are already recognized by the object numbers/labels. If we replace SMART object number/label with RFID tag that is compatible with Nokia 6131 NFC then we can easily read RFID tag information into our Mobile Client. As RFID tags are very cheaper, so that it does not cost much if we specify RFID tags for every object.

Once the SMART object information is read by Mobile Client through RFID tag, then it can be further processed for providing feedback and suggestion.

(22)

2. Related Work

This chapter contains the study of different implemented and experimented systems by different companies that uses Mobile and RFID together.

2.1 Nokia 3220 Bus Pass

Transportation application provider Buscom in Oulu, Finland has made a trial to upgrade their fair collection system in order to facilitate users to use their mobile phone instead of smart bus cards.

Nokia 3220 phones are used for payment purposes with their NFC shells. NFC tags are embedded and connected with mobile application loaded into Nokia 3220 handset.

Nokia 3220 handsets use NFC at 13.56 MHz frequency to communicate with upgraded hardware system in the buses.

According to Buscom, this system will be beneficial for transportation companies and passengers. Also, this system will save millions of dollars that Buscom spend on smart bus cards and tickets. Passengers will be facilitated from the services that smart bus card cannot provide such that recharge credit through internet, balance inquiry and other user options.

Buscom has decided that if up to 30 phones will be equipped to use this new payment system till March 2005, they will provide payment option to buy travel credit or deduct amount from their bank account or by credit card. [11]

Figure 6: Nokia 3220 uses NFC as bus pass

(23)

2.2 Taxi Tracking Service

A Korean Company KTF has announced its Green Taxi service. It enables user to get information of a taxi by scanning RFID tag which is embedded within seats. RFID reader is embedded with mobile phone that retrieves the information from RFID tag.

Once the information is retrieved, it can be send back to friends or family to track taxi on the basis of retrieved information for extra security. [12]

Figure 7: KTF GREEN TAXI Service

2.3 Buy Movie Tickets Using RFID

U-theater Service is by SK Telecom. In this service, people can buy movie tickets from mobile phone by scanning RFID tags attached with movie poster. By scanning RFID tag, user can view movie video clips, buy cinema tickets, download movie

(24)

Figure 8: Get movie tickets using RFID tag

(25)

3. Design and Architecture

3.1 Use Case Diagram

User

Specify Object ID

Select Object from List

Select Latest Suggestion

Add Suggestion

View Suggestion

Server Send Suggestion

Figure 9: Use Case Diagram - Overview

View Object Detail

Add Text Add Media Elements

Add Subject Add Suggestion

«uses»

«uses» «uses»

Figure 10: Use Case Sub-Diagram - Add Suggestion

(26)

Add Media Elements

Add Audio Add Video Capture Image

«uses» «uses» «uses»

Record

Playback

Delete

«uses» «uses»

Capture

«uses»

Figure 11: Use Case Sub-Diagram - Add Media Elements

3.2 System Architecture

Mobile Client sends HTTP request to the Web Server for SMART objects. Script runs on Web Server and connects with the database to retrieves SMART objects. Web Server generates XML and return to Mobile Client as a result of response. For submitting suggestion, Mobile Client sends information to the Web Server and it store media files locally on the Web Server and saves information into Database.

(27)

Figure 12: Mobile Client for SMART Reaction, System Architecture

3.3 Activity Diagram

Load UI Components [Launch Application]

Connect with Internet

Get Objects List

Get Latest Suggestions

Update UI [Refresh]

[connecting]

[successfully connected]

[not connected]

[Exit]

[Add Suggestion]

Fetch Object Detail [Specified Object ID]

[Select Object From List]

Add Suggestion

Suggestion Detail

[View Latest Suggestion]

[Add Suggestion]

Object Detail

[View Object Detail]

[Audio]

Audio Recorder

[Video]

Video Recorder

Image Capturing

[Image]

[Send Suggestion]

Insert Text Upload Media Files

Send Log [Home]

Figure 13: Activity Diagram

Read RFID Tag

HTTP Server (PHP scripts)

Database Server

Generates XML (to return as a HTTP response)

Database Operations

RFID XML Tag Mobile Client

HTTP Request/

Respons

(28)

3.4 System Design Technology and Motivation

Different technologies used depending upon the feasibility for each part of the system.

• Mobile Client: J2ME (Java 2 Micro Edition)

• Web Server: PHP, XML

• Database Server: MySQL

Motivations behind using these technologies are as follows:

3.2.1 J2ME

J2ME used to develop mobile client. We can develop mobile applications by using J2ME and install them into mobile terminal. Today, many mobile terminals support J2ME applications.

Sun Microsystems has developed many APIs for J2ME. These APIs facilitates the developer to access the native resources of a mobile terminal. J2ME has Mobile Media API, Mobile 3D Graphics API, Trust Services API, Location API, Wireless Messaging API, Java API for Bluetooth and many other APIs. To use a specific API on mobile terminal, that terminal has to support that API as well.

Motivation behind using J2ME is to enable user to capture Audio, Video and Image.

Mobile Media API provides audio, video and other time-based multimedia support. It is an optional package that allows you to gain access to the native multimedia services available on a given device. Nokia 6131 NFC supports MMAPI to access its native multimedia resources, like camera, microphone and speaker.

Flash Lite does not support Audio, Video and Image capturing. Although it can launch media elements of a device but user cannot access these resources within the Flash Lite application.

3.2.2 PHP

(29)

PHP is used as a scripting language for the database operations. Apache is installed on the Web Server and PHP is flexible enough to work with Apache. It is open source and free of cost. PHP is fastest scripting language and its good for a mobile terminal which would be using GPRS connection to access the Web Server.

3.2.3 XML

XML is platform independent and could be used to store data. It looks like HTML and use same tag format. XML is properly structured and web browser can execute it.

Due to platform independency and good data presentation features of XML, it has been selected to fetch the information from the database. Regardless of mobile application, any other application can get structured data from the database and parse it easily for the further implementation.

3.2.4 MySQL

Database Server is installed with MySQL. It is the common database technology that is used for the implementation of the other part of SMART system (Web Client).

MySQL is typically distributed with Linux systems which is a big plus. Further, there are lots of free and inexpensive management tools available for this platform on both Windows and Linux. And finally, the licensing of MySQL on Windows and Linux costs a tiny fraction of what the other database vendors charge.

(30)

4. Implementation

4.1 Development Environment

This section will explain the Kits, Tools, Emulators and Mobile Terminals that are used during the implementation of Mobile Client.

4.1.1 Development Kits

To develop mobile application in J2ME, we need following tool kits to be installed on our system.

• J2ME Wireless Toolkit (WKT) Version 2.2 or greater

• Java Development Kit (JDK) Version 1.4.2 or greater

J2ME Wireless Toolkit contains tools that are important in generating MIDlets (Mobile applications developed with J2ME are called MIDlets). This Toolkit provides the development environment for the MIDP 2.0 and CLDC 1.1 (also for MIDP 1.0 and CLDC 1.0, since these parameters are backwards compatible). It provides the optional packages required for the optional libraries, like 3D and Mobile Media applications.

4.1.1 Tools

Following tools were used for the programming environment.

4.1.1.1 NetBeans IDE 5.0

It is a free user friendly tool for quickly design and coding Java Applications. It can be downloaded from NetBeans website http://www.netbeans.org/

4.1.1.2 NetBeans Mobility Pack 5.0

It is also a free pack that could be installed with NetBeans IDE for the mobile application development. It provides fast mobile application UI development with nice and helpful coding environment. We can also integrate different mobile

(31)

emulators for platform specific development. It can be downloaded from the NetBeans website.

4.1.2 Emulator

Nokia provides different emulators according to the target device. Emulator can be installed and integrated with any mobile application development tool (like NetBeans, Eclips). Emulator helps a lot to develop the platform specific applications and testing without transferring application into mobile. Emulator has some limitations in which we have to install the application on mobile terminal for testing video capturing, audio recording and image capturing.

4.1.2.1 SDK for Nokia 6131 NFC

Nokia S40 emulator includes emulator, APIs, sample MIDlets and the documentation.

Once it is installed, we can add its emulator into our development tool by specifying its installation path. Contactless Communication API (JSR-257) included in the Nokia 6131 NFC SDK enables the usage of the NFC features of Nokia 6131 NFC mobile phone. SDK is based on Prototype SDK 4.0 S40 240x320 emulator.

Figure 14: Nokia 6131 NFC Emulator

(32)

4.1.3 Mobile Terminal

Nokia 6131 NFC used as mobile terminal. It is a Series 40 3^rd Edition mobile with Nokia OS having resolution 240x320. Java Technology is MIDP 2.0 and CLDC 1.1 with many APIs support (Wireless Messaging API, Mobile Media API, Web Services API, Mobile 3D Graphics API, JTWI, Wireless Messaging API, Scalable 2D Vector Graphics API, Contactless Communication API, FileConnection and PIM API, Bluetooth API, Nokia UI API). [9]

See also Nokia 6131 NFC under section 1.3.3 RFID.

Mobile Media API (MMAPI) and Contactless Communication API were required to develop Mobile Client.

4.1.3.1 Mobile Media API (JSR-135)

Mobile Media API provides audio, video and other time-based multimedia support. It is an optional package that allows user to gain the access to the native multimedia services available on a given device. [14]

4.1.3.2 Contactless Communication API (JSR-257)

Contactless Communication API provides the communication between two mediums to transfer small amount of data. It allows applications to access the information on contactless targets. It can be used for communicating with RFID, NFC or bar codes.

Small amount of data between two NFC devices can be exchanged using this API.

RFID tag can be read/write using a RFID reader/writer without a physical contact but with the close distance between two devices. [15]

4.2 Mobile Client

Mobile client is implemented on Nokia 6131 NFC. When user launches the application, it asks for permission to access the internet, it fetches the objects and the latest suggestions from the server on providing permission. Mobile Client uses HTTP

(33)

protocol to retrieve/send data. Mobile Client has different parts that are explained as follows:

4.2.1 Object Selection

In order to provide suggestion, user has different options to annotate an object. In Mobile Client, SMART object is known as Responz Object.

Options are as following:

4.2.1.1 Responz ID

User can provide suggestion by specifying a Responz ID. It is a numeric ID. After entering a Responz ID, select Add Suggestion.

Add Suggestion connects with the Responz Server using GPRS and execute PHP script; it fetches the object details from the database and takes user to the Add Suggestion screen for entering the suggestion information.

4.2.1.2 Responz Object

If user does not know object id, then it can be selected from a Responz Object choice list. By opening Responz Object choice list, a list of 8 latest objects displayed. User can select one of these objects and select Add Suggestion. It takes user to the Add Suggestion screen for entering the suggestion information.

If user has entered a Responz ID and also selected a Responz Object from the list, then Mobile Client considers Responz Object as a default selection.

4.2.1.3 Latest Suggestion

Topics of three latest suggestions are listed under Latest Suggestion. User can select one of the suggestions and select View Suggestion. It takes on View Suggestion screen for suggestion details. View Suggestion screen displays the details of the suggestion (i.e. object name, topic, suggestion, user, date, priority, action).

(34)

By selecting Suggest option, it takes to the Add Suggestion screen and use suggestion object as Responz Object.

4.2.1.4 Open Suggestion

When user does not specify Responz ID or Responz Object and selects Add Suggestion then it takes to Add Suggestion where user can add an open suggestion.

Open suggestion means that it does not suggest for any registered Responz Object but an Open Responz Object.

4.2.1.5 RFID Tag Detection

This feature is not yet implemented due to unavailability of RFID tags. Idea is that when user brings Nokia 6131 NFC near RFID tag, it will automatically launch mobile application with Responz ID and user can proceed for the further procedure to add suggestion.

Figure 15: Mobile Client Main Screen

(35)

Figure 16: View Suggestion Screen

4.2.2 Add Suggestion

Add Suggestion screen has different options to add the suggestion material. View Object option shows Responz Object Details on View Object screen. Object detail contains object name, description, creation date, modified date, expiry date, latitude, longitude and radius.

User can add subject, message/text and also can add different media elements for a suggestion. Media elements are audio, video and image. Subject of maximum 50 characters can be entered and suggestion message length is maximum 1000 characters.

While adding suggestion, priority can be set to Red, Yellow or Green. These priorities represent High, Normal and Low priorities. When priority is not specified then such

(36)

suggestion will not have any priority and suggestion will be analyzed by semantic analysis that is out of scope of this thesis.

4.2.2.1 Audio Capturing

Selecting Audio from options will open audio recorder. It records audio in AMR format and keeps the recorded data in memory. It records for the unlimited time. It has been experimented that the recording in WAV format records only for few seconds. That is the reason AMR encoding is selected for the audio recording.

Audio recorder records audio. It asks for permission to access necessary mobile resources for capturing. Once recording is completed, selecting Stop to stop the recording. Selecting Playback plays the recorded audio. Selecting Delete removes the recorded audio. Going back without delete adds Audio under Media Elements at the end of the Add Suggestion screen.

When Audio is added under Media Elements list, selecting Audio will open Audio Recorder and provide options to Record, Playback and Delete.

4.2.2.2 Video Capturing

Video captured in 3gpp format. It has limitation to record only for 6 seconds. Due to low mobile memory of Nokia 6131 NFC, it does not allow to record more than 6 seconds. 3gpp is the only encoding scheme that Nokia 6131 NFC supports. It saves the recorded stream into the memory same as Audio Capturing.

Selecting Video goes to the Video Capturing screen. It takes permission to access the mobile camera. Pressing Record starts the recording and it automatically stops the recording after 6 seconds if user does not select Stop. By selecting Playback plays the captured video. Selecting Delete removes the captured video and goes back to Add Suggestion screen.

When Video is added under Media Elements list, selecting Video opens the Video Capturing and gives options to Record, Playback and Delete.

(37)

4.2.2.3 Image Capturing

Nokia 6131 NFC supports JPEG encoding for image capturing. It captures image and hold in the program’s memory. It captures image in 480x640 pixels resolution.

Memory overflows and application quits if image is captured in higher resolution. So that is the reason for specifying 480x640 pixels resolutions. On Add Suggestion screen, when user selects Image option, it goes to the Image Capture screen to capture an image. It asks for permission to access the mobile camera. On providing permission, it captures the image. After capturing image, Image added under Media Elements list at Add Suggestion form to indicate that user has added an image.

Selecting Image again from options previews captured image in the canvas. It gives Delete option; deleting image removes image data from the memory and also from the Media Elements list. After Delete, it comes back on the Add Suggestion screen.

Figure 17: Add Suggestion Screen

(38)

4.2.3 Submit Suggestion

Selecting Submit asks for the confirmation about submitting. On providing permission sends data to the Web Server. PHP script runs and uploads the media files, if any. After uploading media files, it inserts the information into the database. It saves each media file with a unique name starting with the object name.

Submit log shown when the submit process completes. It gives the detailed information about the amount of data and which data is uploaded and which data failed. On Submit screen, selecting Home takes user to the first screen.

Figure 18: Suggestion Submission Details

4.3 Web Server

PHP scripts are hosted at http://demo.responz.nu. There are 4 PHP scripts that handle Mobile Client requests.

(39)

4.3.1 Get Objects

To retrieve the objects list, Mobile Client sends HTTP request to execute getObjectXML.php PHP script. It fetches the objects and their details from the database and returns an object detail XML. In order to fetch the object details of a specific object, we have to assign object id to the smartid variable of the query string.

URL looks like this;

http://demo.responz.nu/Mobility/getObjectXML.php?smartid=200 where 200 is the object id.

Request returns XML of the following format that is further parsed by Mobile Client.

<type>Förslag</type>

<name>Unimob AB</name>

<description>Här sitter vi och gör objekt.</description>

<keywords>hej,hopp,lingontopp</keywords>

</Object>

</Responz>

4.3.2 Get Latest Suggestions

Mobile Client retrieves 3 latest suggestions by sending HTTP request to getSuggestionsXML.php script. Script connects with the database and fetches 3 latest suggestions, generate XML and return.

Script returns XML of the following format; that is parsed by Mobile Client.

<username>testuser</username>

<OBJECTNAME>Luleå Tekniska Universitet</OBJECTNAME>

(40)

<priority>yellow</priority>

<maintext>B building cycle parking has no proper arrangement for cycle parking!</maintext>

<status>Inskickad</status>

</Suggestion>

</Responz>

4.3.3 Insert Suggestion

There two different scripts for handling the text and the media data. When the text data is inserted into the database, it returns blogid of recently entered data, which is further used in the insertion of the media data.

4.3.3.1 Insert Text Data

Mobile Client sends the required information to submit a text suggestion through the query string to insertTextData.php script. It sends the user id, user name, smart id, subject, suggestion text and the priority. Script gets all these parameters and inserts the information into the database. On the successful insertion, it returns the blogid of inserted data. It returns the blogid value 0 if insertion failed.

4.3.3.2 Insert Media Data

If there is also media data that need to be uploaded, Mobile Client sends the information to the insertMediaData.php script. It sends file type (jpg, 3gpp etc), smart id, blogid (returned by insertTextData.php script) and user id.

It uploads the file and inserts the suggestion information into the database. Script returns the status of the operation that is read by Mobile Client.

(41)

5. Testing and Evaluation

Each part of the system has been tested during the implementation. In the final testing, the whole system is tested from the object selection to the suggestion submission.

Small test cases designed to test each part of the system.

5.1 Test Cases

Add Suggestion by Entering Responz ID

Objective Get Responz Object Name and details by entering the Responz ID

Prerequisite

Object exists in the database and the mobile is configured to access the GPRS. Responz Object is not selected from the list.

Input Data Valid numeric Responz ID Steps 1. Enter Responz ID

2. Press “Add Suggestion” button or select from the options

Expected Result

Go to Add Suggestion screen and display Object Name. Add options “Audio”, “Video”, “Image”, “View Object” and

“Submit”.

Test Result Successful

Table 1: Add Suggestion by Responz ID Test Case

Add Suggestion by Selecting Responz Object

Objective Get Responz Object Name and the details by selecting the Responz ID from the Choice List.

Prerequisite Object(s) exists in the database and the Responz Object list is not empty.

Input Data None

Steps 1. Select Responz Object from the List

2. Press “Add Suggestion” button or select from the options Expected Result Go to the Add Suggestion screen and display the Object

Name. Add options “Audio”, “Video”, “Image”, “View

(42)

Object” and “Submit”.

Table 2: Add Suggestion by Responz Object Test Case

Add Suggestion by Viewing Latest Suggestion

Objective Get the Responz Object Name and the details by viewing the latest suggestion.

Prerequisite

Object(s) and suggestion(s) exist in the database and the Latest Suggestion list is not empty. Select a suggestion and press “View Suggestion” button or select from the options, it takes on the View Suggestion screen.

Input Data None

Steps

1. Select Latest Suggestion from the Choice List

2. Press “View Suggestion” button or select from options 3. It will take to View Suggestion screen

4. Select “Suggest” from options

Expected Result

Go to Add Suggestion screen and display Object Name. Add options “Audio”, “Video”, “Image”, “View Object” and

“Submit”.

Table 3: Add Suggestion by Viewing Latest Suggestion Test Case

Record, Playback and Delete Audio

Objective Check Audio capturing, playback and delete functionality Prerequisite Object selected and current screen is “Add Suggestion”.

Input Data Open Audio Recorder by pressing “Audio” from options.

Steps

1. Select Record

2. Select Stop to stop recording

3. Select Playback to playback recording 4. Select Stop to stop playback

5. Select Delete to delete recorded audio

Expected Result Selecting Record will start recording and selecting stop will

(43)

stop it and display data recorded in number of bytes and duration in seconds. It will provide Record, Playback and Delete options.

Playback will start playing back and pressing Stop will stop playback.

Pressing Delete will delete the recorded data, take back to the Add Suggestion screen and remove Audio from Media Elements list.

Test Result All operations Successful but it show time duration 0

seconds. In AMR encoding, we cannot get the time duration.

Table 4: Audio Recording, Playback and Delete Test Case

Record, Playback and Delete Video

Objective Check Video capturing, playback and delete functionality Prerequisite Object selected and current screen is “Add Suggestion”.

Input Data Open Video Recorder by pressing “Video” from options.

Steps

1. Select Record

2. Select Stop to stop recording

3. Select Playback to playback recording 4. Select Stop to stop playback

5. Select Delete to delete recorded video

Expected Result

Selecting Record will start recording and selecting Stop will stop it. It will provide Record, Playback and Delete options.

Playback will start playing back and pressing Stop will stop playback and provide options to Delete or Record again.

Pressing Delete will delete the recorded data, take back to the Add Suggestion screen and remove Video from Media Elements list.

Table 5: Video Recording, Playback and Delete Test Case

(44)

Capture and Delete Image

Objective Check Image capturing and delete functionality

Prerequisite Object selected and current screen is “Add Suggestion”.

Input Data Open Image screen by pressing “Image” from options.

Steps 1. Select Capture to capture image 2. Select Delete to delete captured image

Expected Result

Selecting Capture will capture image and provide Delete option.

Selecting Delete will delete image data, take back to Add Suggestion screen and remove Image from Media Elements list.

Table 6: Image Capture and Delete Test Case

Submit Suggestion

Objective Test Submit suggestion functionality

Prerequisite Current screen is “Add Suggestion”. GPRS is working.

Input Data At least one suggestion element has entered (Subject, Text or any Media Element)

Steps 1. Select Submit option 2. Wait for submit completion

Expected Result Show Submit screen and display status log Test Result Successful

Table 7: Submit Suggestion Test Case

5.2 Test on Nokia N91

Mobile Client has been tested on Nokia N91 phone as well. Nokia 6131 NFC is Series 40 3^rd Edition and Nokia N91 is Series 60 3^rd Edition developer platform.

(45)

There were no such differences observed in the user interface, expect only two. Nokia 6131 NFC has 3 menu options in the display, right, left and center whereas Nokia N91 provides only 2 menu options that are on the right and the left sides only. In Nokia 6131 NFC, we assign highest priority to the command/option that we wish to display in the center. If no priority is assigned and all have equal priority, then it will put the first command option in the center.

Buttons functionality does not work on Nokia N91. On pressing a button, it shows command option associated with that button, but it does not work. We have to use that command from options.

The major difference has become while Audio, Video and Image capturing. Although Nokia N91 also supports all encoding schemes that Nokia 6131 NFC follow for capturing.

5.2.1 Audio Capturing

Audio capturing went fine on Nokia N91. As Nokia 6131 NFC records audio in AMR format by default and no need to provide special encoding parameters to tell the application about the encoding scheme. Nokia N91 records audio in the WAV encoding by default. It also displays duration in form of seconds as it records in WAV encoding scheme. Moreover, size differs with respect to the record duration as WAV format has very good quality than AMR.

5.2.2 Video Capturing

Nokia N91 behaves very strange in the video recording. Nokia 6131 NFC records video in 3gpp encoding scheme by default and no need to tell the application about the encoding scheme. But in Nokia N91, it records video very well and create problem in the playback sometime. It does not play back sometimes. After some playback requests, when it create problem, application just quit.

(46)

5.2.3 Image Capturing

Nokia 6131 NFC captures image in only JPEG encoding. Nokia N91 also supports JPEG encoding but it does not capture image and creates the problem. Might be this is due to the reason that Nokia 6131 NFC and Nokia N91 has different screen dimensions. [16]

For image capturing, when we create player using the Manager Class, it uses different datasource for the initialization in Nokia 6131 NFC and Nokia N91. For Nokia 6131 NFC, we have to use capture://image and for Nokia N91, we have to use capture://video. On the basis of platform information, we can decide which datasource should be used.

Platform information can be checked by System.getProperty("microedition.platform") function. When platform recognized then different image capturing encoding scheme and dimensions mentioned. Nokia N91 captures image using JPEG encoding with 640x480 pixels resolution whereas Nokia 6131 NFC captures in 480x640 pixels.

Nokia N91 also creates some problem while capturing, it does not capture image some times and when it captures, it does not show still image on the canvas.

5.3 Evaluation

It has been observed that system behaves different in aspect of audio, vide and image capturing. It goes out of memory because all the media elements are stored in the program’s memory. As mobile is a low memory device, sometimes it goes out of the memory and the application quits.

For now, Mobile Client is platform dependent. As the test results of Nokia N91, Mobile Client fully functional on Nokia 6131 NFC only and creates some problems on Nokia N91.

Regardless of the development limitations and low resources of Nokia 6131 NFC, Mobile Client works properly and the expected results have been achieved.

(47)

6. Conclusion and Future Work

6.1 Conclusion

It has been concluded from analyzing best possible way to annotate objects that MPS and GPS are not so much fruitful because of accuracy in context of within an industry.

But these technologies would be effective for outdoor purposes.

NFC technology has opened a new world for mobiles and it is expected that till year 2009, NFC technology would be integrated in every mobile phone. Besides NFC technology, mobile phone manufacturer companies also started GPS integration in their mobile phones, Nokia N95 is one of the examples.

Mobile Client for SMART reaction has been developed successfully and fulfilled the requirements regardless of some features that were not implemented due to the shortage of time.

6.2 Limitations and Known Issues

Mobile Client has some limitations and there are some issues that have been figured out while testing.

• Video capturing limit is 6 seconds

• Mobile Client goes out of memory

• When one of media capturing (Audio, Video or Image) causes problem then other stop working too

• Mobile Client is platform dependent and works on Nokia 6131 NFC

• Some times Refresh command option does not load objects and suggestions

• While submitting a suggestion, if media file has been stored on the server and script could not insert data into the database then file will remain on the server

6.3 Future Work

Lots of future work is ahead after analyzing and implementing current Mobile Client for SMART Reactions. Many features and options are missing in the current Mobile

(48)

Client and that can be implemented to add value into current system. Future recommendations and work could be focusing on:

• MPS and GPS integrations for the user location and the object detection

• User authentication

• RFID tag detection and object selection

• Provide options to view the location map for confirmation

• View media suggestions (Audio/Video Streaming)

• Platform independent Mobile Client. Enable features and options of the application according to the mobile platform

• Solution for AMR and 3GPP decoding for the Web Client

(49)

References

[1] Sergio Ríos, Location Based Services: Interfacing to a Mobile Positioning Center http://www.wirelessdevnet.com/channels/lbs/features/lbsinterfacing.html, May 2007 [2] Telia, Application guide for Service Integrator v 3.0

[3] What is GPS?

http://www8.garmin.com/aboutGPS/, May 2007

[4] Bluetooth GPS Nav Kit from Socket Communications

http://www.mobiletechreview.com/tips/GPS/socket_bluetooth_GPS.htm, May 2007 [5] GPS Track 1.1 for J2ME

http://qcontinuum.org/gpstrack/, May 2007 [6] Google Earth

http://earth.google.com/, May 2007 [7] What is RFID?

http://www.technovelgy.com/ct/Technology-Article.asp, May 2007

[8] Near Field Communication (NFC). ECMA International Standardizing Information and Communication Systems

ECMA/TC32-TG19, Geneva, October 2002 [9] Nokia 6131 NFC, Device Details

http://www.forum.nokia.com/devices/6131_NFC, May 2007

[10] Software Development Kit for NFC device. SDK for Nokia 6131 NFC.

Mikko Saarisalo, 19.4.2007

[11] RFID Journal, News, RFID-enabled Phones Take the Bus in Oulu http://www.rfidjournal.com/article/articleview/2062/1/1/, May 2007 [12] KTF, Green Taxi Service

Aving Global News

http://aving.net/usa/news/default.asp?mode=read&c_num=28634&C_Code=01&mn_

name=news, May 2007 Mobile Mentalism

http://mobilementalism.com/2006/11/16/mobile-phone-and-rfid-taxi-tracking- service/, May 2007

[13] Aving Global News, Buy Movie Tickets using RFID

(50)

http://aving.net/usa/news/default.asp?mode=read&c_num=28623&C_Code=01&mn_

name=news, May 2007 [14] Mobile Media API

http://java.sun.com/products/mmapi/overview.html, May 2007 [15] Contactless Communication API

http://jcp.org/en/jsr/detail?id=257, May 2007 [16] Nokia N91 Device Details

http://www.forum.nokia.com/devices/N91, May 2007