Cross-platform Development for Wearable Devices

(1)

Institutionen f¨

or datavetenskap

Department of Computer and Information Science

Final thesis

Cross-platform Development for

Wearable Devices

by

Gustav Beck-Nor´

en

LIU-IDA/LITH-EX-A–15/038–SE

2015-06-10

(2)

(3)

Link¨opings universitet

Institutionen f¨or datavetenskap

Final thesis

Cross-platform Development for

Wearable Devices

by

Gustav Beck-Nor´

en

LIU-IDA/LITH-EX-A–15/038–SE

2015-06-10

Supervisor: Magnus B˚ang, Linköping University Niklas Björkén, Accedo Broadband AB Examiner: Rita Kovordanyi, Linköping University

(4)

(5)

Abstract

The market for wearable devices is continuously growing and has seen an in-crease in interest and demand this past year, specifically smartwatch devices. With several big players entering and trying to take place in the market the number of devices and platforms grow. This leads to device and software fragmentation like the one seen in the world of smartphones. In this paper I discuss and compare the two smartwatch platforms Android Wear and Apple Watch in terms of possibilities, limitations and differences. Research is done to find cross-platform development possibilities for these platforms. Extensive theoretical background of both APIs is researched and presented. An app for both smartwatch platforms is developed with integration of the WebSocket protocol to function as a remote control for a Video-On-Demand web service. This is done to showcase the cross-platform possibilities and differences of the platforms. As a result the biggest differences are out-lined and a conclusion is made that cross-platform development for these platforms can be challenging but is possible on certain levels.

(6)

Preface

I would like to begin by thanking Accedo Broadband AB for giving me the opportunity to perform this master thesis within such an exciting area. A special thanks to my Accedo supervisor Niklas Björkén and also Niklas Lavrell for making me feel welcome at Accedo and helping me with feed-back and ideas during this thesis. I would also like to thank my examiner, Rita Kovordanyi, and my opponents, Oskar Eriksson and Emil Rydkvist, at Linköping University. It has been very exciting working with smartwatches and it has been a real learning experience. Finally I would like to thank my family and my friends for the encouragement and support during this thesis work and during my studies at Linköping University.

Gustav Beck-Nor´en Stockholm, June 2015

(7)

List of Figures

2.1 Interaction pattern using a smartphone (Jordan, 2014) . . . . 8

2.2 Interaction pattern using a smartwatch (Jordan, 2014) . . . . 8

2.3 Google Play Services diagram (Google, 2015) . . . 11

2.4 High level architecture of Android Wear API . . . 12

2.5 The Data API . . . 13

2.6 The Data Layer API stack (Hahn, 2015) . . . 14

2.7 Communication between a WatchKit app and WatchKit ex-tension (Apple, 2015b) . . . 18

2.8 WatchKit app life cycle (Apple, 2015c) . . . 19

2.9 Accedo VIA (Accedo, 2015b) . . . 20

2.10 MEAN stack (Sevilleja, 2015) . . . 21

2.11 High level architecture of VIA Web . . . 22

4.1 Use case solution idea . . . 26

4.2 High level architecture of solution . . . 27

5.1 Final Native Android Wear Application . . . 38

5.2 Home screen with notification on wearable and phone . . . . 39

5.3 Final Android Wear Application . . . 39

5.4 Final Apple Watch Application . . . 40

(10)

Chapter 1

Introduction

1.1 Motivation

This section covers some short background information about the topics and motivations to why this thesis was performed.

1.1.1 Wearables

Wearable technology has been around for a very long time and is a very broad term but can be described as follows. A wearable device is simply a piece of technology that is incorporated in a piece of clothing or in an accessory that can be worn. This includes anything from pedometers em-bedded in shoes and trackers in bracelets, to something like microphone earrings. The wearable technology market had a short upswing and caught the normal consumer’s eye in the mid 80s with Casio’s wristwatch calcu-lator (Wikipedia, 2015b), but has been fairly absent from the consumer market since. Until now. The launch of Google Glass and the new trends in tracking our movements and habits has sparked a new interest in wearable technology. Healthier lifestyles have sparked the fitness tracker market and the need to be more connected now directs the consumer’s gaze towards the smartwatch. In May of 2012 the record-breaking (Kickstarter, 2015) success of the Kickstarter campaign for the Pebble Watch confirmed the indications that the demand for smartwatches is increasing. With technology getting more powerful and affordable the smartwatch, although still struggling, is finally starting to find its area of use and niche. Forbes writes that statistics from GlobalWebIndex says that 71% of those aged 16 to 24 would like to own a piece of wearable tech (smartwatch, smartband or Google Glass) (Lip-man, 2014). Google’s (2015) announcement of Android Wear in March of 2014 with partners such as Samsung, LG, Motorola, HTC and Asus, and the revealing of Apple Watch by Apple in September the same year makes it clear that non wants to miss the launch of the smartwatch market. With

(11)

1.1. MOTIVATION CHAPTER 1. INTRODUCTION

all of the big players in the mobile industry diving into the area of smart-watches the innovation and number of devices available on the market is increasing rapidly. A report released in February by the market intelligence firm Tractica (2015) says that the ”Smart Watch Shipments Will More than Quadruple in 2015, Reaching 24.4 Million Units Worldwide”. This follows the smartwatch market forecast done by NextMarket (2013) which states the growth will continue year over year. The demand to wear a smartwatch on the wrist never been higher.

With these new platforms launching on hardware devices that have not existed in this format before a lot of questions arises. As of starting this thesis work the Android Wear platform is fairly immature and Apple’s WatchKit is only in early Beta. Not much is known about the respective API’s and the limitations and differences between these new platforms. This is why this has to be thoroughly investigated.

1.1.2 Cross-platform

In the world of mobile application development fragmentation between dif-ferent platforms and operating systems is becoming an increasingly large challenge for developers to overcome (Arthur, 2014). The term cross-platform refers to code, components or entire applications which can be run on on multiple different platforms. Having an application function on both the Android platform and the iOS platform simultaneously for instance is con-sidered cross-platform. This is no different in the world of wearables, specif-ically smartwatches. The smartwatch platform brings additional aspects into consideration with even more variety between models and brands than smartphones, with different form factors, functionalities and sensors. A key challenge for app developers across the world will be to ensure device com-patibility for maximum reach. This can be more complicated than it might seem at first glance. There are three major platforms today in the smart-watch market: Android Wear, Pebble OS and Watch OS (Apple Watch). But confusingly, even though LG and Samsung are listed as Android Wear partners, some of their devices are running their own operating systems. For example the LG Watch Urbane LTE is running an operating system based on WebOS and multiple Samsung devices are running a Tizen-based operat-ing system usoperat-ing the Samsung Gear SDK. The company Accedo Broadband faces this problem of platform fragmentation on a daily basis.

1.1.3 Accedo

Accedo Broadband (henceforth referred to as Accedo) is a global company based out of Stockholm, Sweden and a market leading enabler of TV ap-plications (Accedo, 2015a). Accedo provides apap-plications, tools and ser-vices to media companies, consumer electronics and TV operators globally, to help them deliver the next-generation TV experience. Accedo’s cloud-based platform solutions enable customers to cost-efficiently roll out and

(12)

1.2. PURPOSE AND AIM CHAPTER 1. INTRODUCTION

manage application offerings and stores for multiple devices and markets. The company works with customers like Netflix, HBO, Viaplay and Disney. They have produced well known consumer video-on-demand applications e.g. HBO Nordic and Viaplay for various platforms. One of Accedo’s prod-uct solutions is called VIA. It is built to provide the customer with a highly customizable digital entertainment service and has the ability to deploy to multiple platforms (TVs, Game Consoles, mobile devices, the web and more). Accedo is always interested in the newest technology and in try-ing to find new use cases for these and incorporate them in their solutions and products. This is where smartwatches come into the picture. A paper written a the University of Oslo by Pradhan and Sujatmiko (2014) states that there is a value to and a demand for ways to minimize the interaction steps and effort required to perform certain tasks on the phone. The report suggests that the smartwatch can be a viable solution for this. This is some-thing Accedo sees value in. With many of their products already designed to function cross-platform e.g. VIA, what are the possibilities to extend these to the wrist of the user of every smartwatch platform? Are there any relevant and valuable use cases in a media consumption context? These are some of the questions they hope this thesis work can answer.

1.2 Purpose and Aim

This thesis work will focus its efforts within the area of wearables, specifically smartwatches. With the smartwatch market and number of devices growing, and facing the challenge of major fragmentation an investigation in cross-platform development for smartwatches will be done.

To do this a smartwatch application for both the Android Wear and Apple Watch platform will be developed towards Accedo’s existing prod-uct solution Accedo VIA. Accedo explains this prodprod-uct as follows: ”Accedo VIA is a state of the art multi-screen application solution, which enables TV operators, media companies, and broadcasters to launch full-featured digi-tal entertainment services quickly and efficiently across multiple connected devices, including Mobiles, Tablets, Smart TVs, Game Consoles, and also the web. Accedo VIA provides a comprehensive set of reference applications that encompass all the major types of over-the-top (OTT) TV viewing” (Ac-cedo, 2015b). Multi-screen refers to the usage of multiple screens e.g. TV, web browser, smartwatch etc. in the same use case. One of Accedo’s biggest strengths today is their knowledge in a wide range of different platforms. With applications available across a large amount of different platforms it is relevant for them to investigate the possibilities in the new area of smartwatches. The VIA application is chosen as target for integration with smartwatches because by doing so this investigation becomes relevant and generates value for Accedo. They would like to include smartwatches in their product portfolio. The test application will be created cross-platform to illustrate a relevant key use case and be used for demonstration of the

(13)

1.3. RESEARCH QUESTIONS CHAPTER 1. INTRODUCTION

projects progress. The web solution of VIA explained above will be the tar-get for this use case. The use case used during the entirety of the project is as follows:

”The user shall easily be able to control media playback (play/pause/vol-ume up/vol(play/pause/vol-ume down) in the web browser containing VIA from his/her smartwatch”.

Multi-screen in this use case is partly what drives the need for cross-platform functionality. I want to use one screen, the smartwatch, to control another screen, the web browser, and be able to do this for every smartwatch platform (Android Wear and Apple Watch).

A solution to a similar use case is presented by Seettharamu Keshav et al. (2014) but their approach is to utilize the sensors in the smartwatch to produce a gesture based remote control for Smart-TVs.

This thesis should provide insight to where abstractions to enable cross-platform development are relevant and possible across smartwatches. By implementing this relevant use case and integrating it with Accedo’s VIA Web, the pain points for cross-platform development should be discovered and solutions should be proposed. The finished thesis should also serve as a proof of concept for Accedo.

1.3 Research questions

With the thesis purpose and aim as a basis the relevant research questions can now be formulated.

• Which are the integration points with the wearable devices APIs and external systems1, which can maximize portability and development synergies cross-platform?

• Which are the similarities and differences between the smartwatch platforms from both a developer’s and a user’s perspective?

By investigating and implementing a relevant use case using smartwatches in a media consumption context, the key areas where cross-platforms ab-stractions are possible and desirable will surface.

1.4 Scope and Limitations

Due to the time limitations of this thesis work, the broad subject of cross-platform development and the vast amount of devices encompassed by the term wearables limitations must be set to narrow the scope of the research. The following limitations will be followed throughout the thesis work:

• Research and development will focus solely on the area of smart-watches.

(14)

1.5. APPROACH CHAPTER 1. INTRODUCTION

• Priority will lie on two smartwatch platforms: Android Wear and watchOS (Apple).

• Only the use case stated in Purpose and Aim will be implemented and work as a proof of concept.

• No major changes will be needed to the existing solution VIA save for integration of the communication with the smartwatch.

• The aim is not to produce a complete and production ready application but to use the use case to find cross-platform possibilities.

1.5 Approach

As stated before, the thesis is performed at Accedo and is focusing on the area of smartwatches. During the thesis work the problem will be ap-proached from two different angles. Use case and literary studies, and a deep dive into the different wearable platforms and APIs as well as into Ac-cedo VIA Web. This will be done as follows: Firstly constraints and clear definitions of the concepts of wearables and smartwatches must be estab-lished. Then an investigation of relevant and universal use cases performed. Academic research regarding smartwatch interaction, design and APIs will be reviewed and presented. Challenges must be identified and considered during this phase.

Next the test application realizing the aforementioned use case towards VIA Web will be developed on each platform as part of research in finding the common elements and integration points in the different APIs. The finished thesis should serve as a proof of concept for Accedo and provide answers to previously stated research questions.

Evaluation will be done and conclusions will be drawn based of the results produced by the pre-study and implementation.

(15)

Chapter 2

Theoretical Background

This chapter introduces the reader to some basic concepts, techniques and theoretical background from research relevant to this report. Technical in-formation regarding both Android Wear and Apple Watch ecosystems is retrieved from each respective developer’s sites (manuals, API guides and references) unless otherwise stated.

2.1 Smartwatches

For decades the smartwatch has been a popular accessory in the world of science fiction as a gadget used for communication, tracking and control-ling other devices. In the real world is smartwatches in fact nothing new, but only recently is the technology and the functionality starting to match consumers expectation. There have been several attempts to get the con-sumer interested in smartwatches: Fossil’s wrist PDA and IBM’s WatchPad are examples of these (Rawassizadeh et al., 2015). Microsoft also made an effort in 2003 with the launch of the Microsoft SPOT (Albanesius, 2014). This gave the user access to feeds about news, sports, stocks, weather and more. This limited functionality was not enough to keep the users interested in wearing a bulky mini computer on their wrists. During the decade since, the progress necessary to intrigue the average user has been made in the field of wearables. Functionalities such as 3G/4G connectivity, advanced touch screens, voice commands, navigation and the need to feel even more con-nected than before makes the smartwatch highly desirable. The smartwatch are meant to give us the ability to stay more connected to the digital world, but more engaged in what is happening around us. This can be explained by the interaction patterns illustrated in Figure 2.1 and Figure 2.2.

Figure 2.1 represents a user’s normal day interaction with their phone. Most smartphone users today can recognize their phone usage in this pattern agree that there is a lot of time wasted on interactions that should only take a second. This is where smartwatches can reduce the time overhead

(16)

2.1. SMARTWATCHES CHAPTER 2. THEORY

Figure 2.1: Interaction pattern using a smartphone (Jordan, 2014)

per interaction by giving the user the right amount of information, ready for interaction, directly to the wrist. Figure 2.2 show the same set of interactions on a smartwatch.

Figure 2.2: Interaction pattern using a smartwatch (Jordan, 2014) This is what Timothy Jordan, developer advocate at Google, meant by saying: ”The user’s more present in the real world, and yet more connected to the virtual world.” (Jordan, 2014)

2.1.1 Challenges and Limitations

It is very important to recognize the limitations and challenges associated with smartwatches and not go overboard and make it out to be something it is not. When developing applications and searching for new ways to utilize the smartwatch platform one has to rethink the design and interaction patterns used for smartphones. Currently smartwatches are not stand-alone devices but rather an extension of our smartphones to make interactions easier and more accessible. Some notable challenges and limitations:

• Very limited screen size. • Limited computing power.

• Currently dependent on companion smartphone for connectivity and computing power.

• Finding relevant use cases.

• Designing applications using only micro interactions. • Immaturity of the platform.

(17)

2.2. CROSS-PLATFORM CHAPTER 2. THEORY

2.1.2 Second screen

The term second screen refers to a device which purpose is to enhance the viewing experience that is taking place on another device. This can for example be a user’s smartphone or tablet working as a companion device (second screen) to for example a TV-set. Since the smartwatch currently more or less acts only as an extension, a second screen, to the companion smartphone it is relevant to look at how the smartphone itself is used as a second screen to other devices. This could be seen as one step further in the area of companion devices. The smartwatch act as an additional companion device to other devices by extension through the smartphone, a sort of third-screen. By considering how the smartphone acts as a second-screen interesting areas for the smartwatch can be found.

2.2 Cross-platform

As stated briefly in the introduction many of the challenges faced with cross-platform compatibility with smartphone application development are the same for smartwatch development. The fragmentation of mobile operating systems is due to the fact that all major actors on the market has developed their own non-standard methods for developing towards their platform. This includes using different operating systems (using unique programming lan-guages and SDK’s) and different hardware configurations making the user experience and user interaction different from platform to platform. Fur-ther fragmentation exists within the various platforms with applications not being able to run on certain different software versions of said platform. Limited development resources often lead to developers having to choose which specific platforms, devices and operating systems to target for the applications.

There are several different development tools available to handle this problem for smartphones. These tools support various amounts of platforms and approach the problem of cross-platform development from different an-gles. According to Ohrt and Turau (2012) there are three main approaches to cross-platform development: mobile web applications, cross-compilation to separate native code and a hybrid approach using the web-view element on each platform. All these approaches and techniques will be taken under consideration when investigating the smartwatch platforms. A presentation and evaluation of these in a smartwatch context is available in the chap-ter Cross-platform Evaluation 6.

2.3 Android Wear

Android Wear is a version of Google’s operating system Android that is designed to run on smartwatches and other wearable devices. It was an-nounced along with the release of a developer’s preview on March 18, 2014.

(18)

2.3. ANDROID WEAR CHAPTER 2. THEORY

Since Android Wear extends the Android operating system much of the same functionalities are available, including Google Now, but with a differ-ent form factor. Much of the user experience of Android Wear is cdiffer-entered around Google Now. Supplying the user the right information, at the right time, using contextual information such as location, time and events. And giving the user the ability to see this information just at a glance. As al-ready mentioned due to the limitations of the smartwatch form factor user interaction is focused solely on micro interactions. Wearable devices run-ning Android Wear needs a companion Android smartphone with Android version 4.3 or later to communicate with.

There are differences that need to be considered when developing ap-plications for Android Wear in comparison to Android. First of all there is a sleep timer that the system enforces. This means that the device will go to sleep if the user does not interact with the displayed activity. When the device wakes back up the home screen will be displayed, not the pre-vious active activity. And the difference that cannot be stressed enough is the limited size, both screen and functionality wise, a wearable application has. Generally the wearable application will contain only a subset of the functionality provided by the corresponding app on the handheld device. Consequently, the term micro interactions and supplying the user with the right amount of information, at the right time, once again is important. An-other noteworthy difference is the way apps are downloaded and installed in Android Wear. The Android Wear device cannot access the Google Play store so wearable applications need to be bundled together with a handheld application. When the handheld application is installed the system auto-matically installs the application on the wearable. However this handheld companion application can also be useful for handling heavy computation, performing network operations or performing other tasks and sending the results to the wearable. In addition to the limitations and challenges stated earlier regarding smartwatches, there are also certain limitations where some API’s available in Android simply are not available in the wearable device. The following API’s are not supported for Android Wear applications:

• android.webkit - Provides tools for implementing browsing of the web. • android.print - Provides all classes and abstractions for implementing

print support.

• android.app.backup - Contains the backup and restore functionality available for applications.

• android.appwidget - Components necessary to develop application wid-gets.

• android.hardware.usb - Provides support to communicate with USB hardware that are connected to Android devices.

(19)

The absence of these API’s puts some limits on the scope of what a wearable app can be and also limits the cross-platform possibilities explained in the Discussion chapter (7).

2.3.1 Google Play Services

Google Play Services is a background service and an API package that let’s applications access Google-powered features such as Maps, Google+ and also Wear. The Google Play service client library contains interfaces that give the developer access to the individual Google services. That is the client library is used to interact with the Google Play service APK (Android application package) that runs as a background service in the Android OS (Figure 2.3). The Google Play store automatically distributes the Google Play services to all devices running Android 2.3 (Gingerbread) or later, and keeps them updated, which frees the developer from worrying about device support. To make a connection to one of the API’s provided by Google Play services an instance of GoogleApiClient (Google API Client) must be created. This serves as a common entry point to all services and manages the connection between device and service. Once connected, the client can make read and write calls using the service-specific API that the application has authorized, as specified by the API and scope added to the GoogleApiClient instance.

Figure 2.3: Google Play Services diagram (Google, 2015)

The use of the Google Play services is not required to extend notifications from handheld applications. It is used to send and sync data to and from the wearable/handheld with the Wearable Data Layer APIs. All Android Wear specific APIs are described in detail in subsection 2.3.2.

(20)

2.3.2 Architecture

Android Wear supplies the developer with a high level of abstraction and a set of APIs to handle operations. These APIs consist of the Message API, Node API and Data Layer API. They handle all the communication needed between devices in addition to the Notification API which can provide the wearable with notifications from existing handheld applications out of the box. The high level architecture of the communication between handheld and wearable and the API’s used is illustrated in figure 2.4.

Figure 2.4: High level architecture of Android Wear API

Node API

The Node API in Android Wear handles events related to local nodes and nodes connected to the Android Wear network. A node in this context is simply a Android or Android Wear device. It tells the developer about which nodes exists in the network and gives information about these. It is used to identify and list available nodes during communication.

Data Layer API

The Data Layer APIs allow for automatic data synchronization between nodes. This works by allowing the handheld and the wearable to take on the role of data sender, data receiver or both. A DataItem is a base object of data stored in the Android Wear network and is synchronized across all nodes in the Android Wear network automatically. DataItems can even be sent to non-connected nodes which will become synchronized one they come online. The DataItem itself consists of two different objects: the payload

(21)

object, which contains the actual data, and a path object which acts as a unique identifier string for the DataItem. The DataItem is only meant to send smaller batches of data back and forth between handheld and wearable. For larger amounts of data, e.g an image, an Asset object can be attached to the DataItem. The large amount of data is still sent over the bluetooth connection and the handheld device does all the heavy lifting, e.g resizing of said image. A useful feature with asset objects is that they can handle caching of data, which prevents retransmissions of objects and preserves computing power, battery and bluetooth bandwidth. The Data Layer API also contains the Message API which is described below. How the DataItem with an asset object is represented is illustrated in Figure 2.5.

Figure 2.5: The Data API

Message API

The Message API exposes an API for components to send messages to other connected nodes in the wearable network. Messages are by default sent in the remote procedure call (RPC) type. That is, once the sender has sent the message there is no validation that the message was actually received at the other end. If this is not good enough messages can be set up to use the Request/Response model. The Message API is used to do common tasks like e.g telling the wearable to start a new activity, or tell the handheld to skip to the next song. Once again only small amounts of data should be sent as payloads through the Message API, for larger amounts use the aforementioned Data API using assets. Since the Message API is limited to sending messages to connected nodes the Data API is also used for sending messages that is to be delivered on node connection. Figure 2.6 shows an example how a message sent from the handheld to the wearable travels through the API-stack from device to device.

(22)

2.4. APPLE WATCH CHAPTER 2. THEORY

Figure 2.6: The Data Layer API stack (Hahn, 2015)

Notifications API

The notifications in Android Wear works right out of the box. The wearable and the companioned handheld devices automatically share notifications. Notifications that appear on the handheld device appears as a card in the wearable devices context stream. To enrich the user experience develop-ers can add wearable-specific functionality to the notifications. This can be functions such as, actionable notifications, receive voice input, adding additional pages and stacking multiple notifications.

Limitation: ongoing notifications will not show up on the wearable, these needs to be done manually on the smartwatch. An ongoing notification is a type of persistent notification which cannot be dismissed by a traditional swipe motion. This type of notification is always on top and can be useful for e.g. media applications for control of playback.

2.4 Apple Watch

Apple Watch is Apple’s smartwatch device and was first announced on the the 9th of September 2014 and later released on the 24th of April 2015. The Apple Watch is the first new device added to Apple’s product range since the iPad was released 2010. The Apple Watch has the most features one would expect in a smartwatch in 2015 but features some unique additions which demands some specific UI patterns. The most distinctive and, for now, unique feature of the Apple Watch is the digital crown. The digital crown is a dial placed on the side of the watch which, when turned, lets the user scroll through content, zoom and when pressed navigates back to the home screen. This allows for precise scrolling and zooming interaction on the watch. Another platform unique feature available on the Apple Watch is “force touch”. This technology allows the touch display to distinguish

(23)

a soft screen-press from a hard one and consequently implement different functionality for each event. This is some compensation in the absents of multi-touch functionality in Apple’s SDK which is available on the Android Wear platform. Other features available on the Apple Watch are: speakers, Wi-Fi connectivity, NFC and haptic feedback (precise vibrations from no-tifications and user interaction). The Apple Watch uses Apple’s operating system designed for the watch called watchOS. To develop applications for watchOS developers are provided with WatchKit, a software development kit for Apple Watch. This section will explain the design, functionalities and limitations of the Apple Watch platform.

2.4.1 WatchKit

WatchKit is the API and SDK provided by Apple to use for development of Apple Watch applications as well as for extending existing iOS applications to the watch. watchOS is the operating system, based upon iOS, which runs on the Apple Watch. It is developed by Apple and shares much of the func-tionality and design of iOS but with some limitations. It is designed around smaller simpler interactions and gestures to suit the Apple Watch’s interface better. WatchKit provides three different ways for the user to interact with and developer to extend applications: apps, glances and notifications. WatchKit Apps

The first thing that needs to be established is that third party apps for the Apple Watch are, as of writing this report, not stand alone applications. That is, they need a paired companion iPhone to function at all. Initially WatchKit apps are meant to extend and/or complement the iOS apps, not replace them. In addition to the obvious limitations in the form of screen size and user interactions etc. there are other challenges that need to be considered when developing WatchKit apps. Developers are for instance not able to access the sensors of the Apple Watch when developing third party apps. The range of unavailable sensors includes the gyroscope, ac-celerometer, heart rate sensor and NFC. The build in speakers, microphone, the Taptic Engine and the functionality of the digital crown are unavailable in WatchKit as well.

WatchKit apps always consist of two parts: the WatchKit extension which runs on the user’s paired iPhone and a set of interface resources which are installed on the Apple Watch. With the limited number of styles and interfaces in WatchKit there are only two basic types of navigation available: hierarchical style and page-based style. Both navigation styles closely resemble their iOS counterparts but cannot be combined. The limited number of UI components available is due to the fact that WatchKit does not make use of the UIKit framework used for iOS apps. Here Apple has replaced controllers and elements with WatchKit-specific classes. Classes such as UIViewController, UITableView and UIButton for example are replaced by

(24)

WKInterfaceController, WKInterfaceTable and WKInterfaceButton. These new interface objects are a lot alike the ones in the UIKit framework except that they are more lightweight, and not as customizable. One UIKit view of importance from a cross-platform perspective that is not available in Watchkit is UIWebView. Other notable differences from iOS development are the life cycles of the WatchKit applications and their inability to run as background services. Both these topics are described in more detail further into the report.

Glances

Glances on Apple Watch are meant to provide the user with short, useful information only meant for reading. They could be seen as feed of contex-tually relevant information and a lightweight view, a ”glance”, of the user’s WatchKit applications. Glances are accessible from anywhere on the watch by doing a swipe-up gesture from the bottom of the screen. They are not scrollable and cannot contain any other action than launching the app it represents. A glance is not required but limited to one per WatchKit app. Notifications

Apple Watch can make full use of notification interactivity supported in iOS, both remote and local. So if the containing iOS app supports notifications extending them to the Apple Watch only requires an implementation of the notification interface in WatchKit. The notifications are meant to provide the user with small amount of information at an appropriate time and can contain small lightweight actions. The user, if they choose to view notifi-cation, can decide to dismiss the notifinotifi-cation, launch associated WatchKit app, or interact directly on the notification by touching the provided action button.

There are two types of interfaces for notifications: the short-look in-terface and the long-look inin-terface. The short-look inin-terface tells the user that a notification has been received and gives only a very brief view of the notification. The short-look interface is not scrollable and cannot be customized. It always displays the app icon, app name and the title string from the notification. If the user interacts with the notification or simply continues to look at the notification (keeps their wrist raised) the watch will transition into the long-look interface. The long-look interface is scrollable, displays more detailed and comprehensive information about the notifica-tion and it is somewhat customizable. If a custom notificanotifica-tion interface is not provided, Apple Watch displays a default interface that includes the app icon, the title string of the notification, and the alert message. If a custom notification interface is provided, this will be displayed instead. The layout of the long-look notification interface is divided into three different areas:

• The system-provided sash at the top is an overlay that contains the app name and app icon. Only the color is configurable.

(25)

• The app content area contains the detailed information about the in-coming notification.

• The bottom area contains the app-defined action buttons from by the containing iOS app and the system-provided dismiss button.

User interaction with notifications works as follows: Tapping the app icon in the sash will launch the WatchKit app. Tapping one of the action buttons in the bottom area will start the linked action on either the WatchKit app or the corresponding iOS app. Since the Apple Watch is limited to only running foreground actions any background actions will be offloaded to the iOS app. The dismiss button simply closes the notification without any further actions. Tapping anywhere else on the notification does nothing.

2.4.2 WatchKit architecture and life cycles

As previously stated the WatchKit app runs as an app extension on the users iPhone device. App extensions gives the user access to content and functions from apps running on iOS 8 or Mac OS X Yosemite. Leverag-ing the extensions functionality in iOS 8 the iPhone contains and runs the code while the watch contains the application storyboard. This means that the watch always needs to be connected, via Bluetooth or Wi-Fi, to an iPhone for the app to function. This is the situation in the initial release of WatchKit but Apple (2014) has stated that third-party developers will be given the ability to develop native applications for Apple Watch in the future. Figure 2.7 illustrates how the architecture of the extension looks and how communication with the watch functions. The sharing of data and how the devices communicate is described more in detail in the section below.

Although UIKit is unavailable in WatchKit the framework still conforms to the MVC (Model-view-controller) design pattern. As mentioned previ-ously the UIKit classes have been replaced by watch-specific ones. Each scene in a WatchKit app’s storyboard is managed by a single interface con-troller object in the form of an instance of the class WKInterfaceConcon-troller. Consequently only one interface controller can be displayed on-screen at any given time. With the limited screen size available the system handles layout of UI-elements automatically and most often stacks them from top to bottom. This can be slightly customized by the usage of Groups that can put elements side-by-side in the top-down hierarchy. The life cycle of a WKInterfaceController is fairly straightforward, see figure 2.8, and does not allow any long-running task without user interaction. This leads to the deactivation and suspension of the app.

The sharing of common data between the WatchKit extension and the containing iOS app is done through a shared app group. An app group is a secure container that multiple processes can access and is common practice when developing iPhone extensions. This makes it possible for WatchKit and the iOS app to share files and user defaults. Apps that work even more

(26)

Figure 2.7: Communication between a WatchKit app and WatchKit exten-sion (Apple, 2015b)

closely with their containing iOS app can use a method that sends messages using a request/response model. This enables the watch app to run activities that require extra time to complete or be run in the background by running them on the iPhone. This is done by sending a request to the phone, which starts the task, and the result is then communicated back to the WatchKit extension when complete. The app delegate in the containing app performs the request using a provided dictionary and then returns a reply to the WatchKit extension.

(27)

2.5. ACCEDO VIA WEB CHAPTER 2. THEORY

Figure 2.8: WatchKit app life cycle (Apple, 2015c)

2.5 Accedo VIA Web

Accedo VIA is one of Accedo’s product solutions. It is built to provide the customer with a highly customizable product and the ability to deploy to multiple platforms. Accedo markets their VIA product as follows: ”Ac-cedo VIA is a multi-screen application solution that allows TV operators and media companies to launch full-featured digital entertainment services quickly across multiple connected devices, Mobiles, Tablets, Smart TVs, Game Consoles and the web.”

Key features of Accedo VIA application: • Content Promotions • Live TV • Program Guide • Video on Demand • Search • Settings • My Account • Recommendations • Favorites • Social (Accedo, 2015b)

(28)

Figure 2.9: Accedo VIA (Accedo, 2015b)

The Accedo VIA Web application is built on a set of JavaScript frame-works that together are referred to as the MEAN-stack (Figure 2.10). The MEAN-stack consists of three JavaScript frameworks together with a NoSQL database solution and they are all described briefly below. The MEAN ab-breviation is a contraction of the framework names MongoDB, Express.js, Angular.js and Node.js.

2.5.1 MEAN Stack

Node.js

Node.js is a JavaScript platform which enables very low effort building of fast, scalable web applications. It is a back-end framework which uses a event-driven architecture and it is very lightweight. The scalability and platform independence makes it well suited for a application of this type. Express.js

Express.js is a web application framework for Node.js. It helps the developer to organize the project into a MVC pattern on the server side and can help build single-page applications. Like Node.js it is very lightweight and has a low effort installation process. It is a tool for handling routes, requests and views in the Node.js back-end.

AngularJS

AngularJS is also a JavaScript framework for web applications but functions as the client side front-end. It also uses the MVC pattern and adapts to and

(29)

lets the developer extend the HTML vocabulary when developing single-page web applications.

MongoDB

MongoDB is a cross-platform document-oriented NoSQL classified database. It enables queries to be made in JavaScript and data provided in JSON format which makes it ideal to work together with the rest of the frameworks.

Figure 2.10: MEAN stack (Sevilleja, 2015)

2.5.2 Architecture

The high level architecture for Accedo VIA Web is built up as illustrated in figure 2.11. The figure (2.11) shows how the app, built using the aforemen-tioned MEAN-stack, runs from a high-level perspective. The VIA Web-site, is a Single-Page-Application web app built with Angular.js. It is rendered in the end-users browser after continuously fetching data, in JSON format, through AJAX requests to the server-side back-end component built with Node.js. The back-end Node.js app is responsible for integrating with 3rd party APIs. OVP is an abbreviation of Online Video Provider and AppGrid is Accedo’s app management solution.

(30)

(31)

Chapter 3

Method

This chapter briefly describes the method and processes used when carrying out this thesis work.

3.1 Process

Preferred by myself along with Accedo all of the stages during the thesis work was done iteratively following the agile methodology. As mentioned before in the Approach section in the Introduction the work was preformed in four phases. The pre-study (and analyzing the results), implementation and evaluation. These in turn where divided into sprints to more easily track the progress of the work and follow the time constraints. The high level view of the phases in which the work was performed is as follows:

• Perform pre-study research • Analyze the pre-study results • Start implementation

– Specifying the use case and requirements – Solution architecture and design

– Code and implement solution • Evaluate the implementation results

3.1.1 Pre-study

Initially the pre-study consisted of researching and familiarizing myself with the concept of wearables. Defining the limitations of the thesis work and gaining an overview of the topic at hand was the first step of the thesis work. The later pre-study was performed with two different approaches: literary

(32)

3.1. PROCESS CHAPTER 3. METHOD

research to find related work and a deep dive into the device APIs and manuals. The results of the pre-study and information needed to understand the final results, discussions and conclusions is presented in the Theoretical Background chapter.

3.1.2 Implementation

With the necessary amount of knowledge acquired during the pre-study the implementation portion of the work could begin. The implementation part of the thesis work was a quite large and important part. By working extensively hands-on with development on the smartwatch platforms knowledge of the APIs and their differences could be discovered efficiently. As shown above the implementation phases begun once an analysis of the pre-study had been performed. With the theoretical background as a foundation the solution design and architecture was done followed by the actual implementation. The phase started with the Android Wear platform. This because of the fact that WatchKit was still in early beta at this stage. I wanted to hold off with the development for Apple Watch hoping for further updates to the SDK. This development process is presented in its entirety in the Implementation chapter.

3.1.3 Evaluation

With the implementation completed along with the deep theoretical knowl-edge about the platforms a relevant and qualified evaluation of the work could be performed. A high level discussion and concrete conclusions is be presented in the Discussion and Conclusion chapters.

(33)

Chapter 4

Implementation

This section showcases the implementation of the use case previously ex-plained in the report. It includes an exposition of the planning and design process as well as the development using the technologies from the theo-retical background. Conclusions and differences between the process and implementation of the different platforms are discussed in chapter 7. To reiterate, the target use case used for the investigation during this thesis work is as follows:

“The user shall easily be able to control media playback (play/pause/vol-ume up/vol(play/pause/vol-ume up) in the web browser containing VIA from his/her smart-watch”.

4.1 Web Server

The design and implementation of the environment outside of the smart-watch app context will be explained. To enable communication between the smartwatch app and the Accedo VIA Web application a server was needed as a go-between. This was set up using the Node.js framework. The sim-plicity and low effort to set up a server and add a WebSocket library made this the optimal platform to work with for this thesis work. Additionally is the VIA application back-end built with the same framework which limited the scope of frameworks and languages used during development. Figure 4.1 illustrates a high level view of the solutions idea for the use case.

Once the server was up and running an implementation of the WebSocket protocol was added to both the server and VIA Web application (Figure 4.2). The WebSocket protocol is an independent TCP-based protocol that enables full-duplex communication over a single TCP-connection. After a hand-shake has been performed between client and server it uses a stream of messages in real time which can be associated with event listeners to handle these. The implementation of WebSocket protocol on the different platforms will be explained within this chapter. Some further modifications was also

(34)

4.2. VIA WEB APPLICATION CHAPTER 4. IMPLEMENTATION

Figure 4.1: Use case solution idea

required in the VIA Web application to handle the events in both directions when sending and receiving WebSocket messages. This was necessary to be able to control the media playback and also to keep the smartwatch and web application in sync.

A WebSocket implementation for Node.js called WebSocket-Node devel-oped by McKelvey (2015) was used. This enables the developer to require the library and start the WebSocket server as shown in Appendix A.1. The server requires connecting devices to identify themselves when the connec-tion opens. This way the server can keep track of which node is the web application and which is the mobile device and relay messages accordingly. Messages are configured to use the JSON format across all platforms to be able to bundle more information within each message. The implementation, how the messages are handled and forwarded to the right device is shown in Appendix A.2.

4.2 VIA Web application

Since the current VIA Web application does not implement the WebSocket protocol this had to be added to enable remote communication to and from the app. With the massive code base that is VIA I looked for a minimum effort way of doing this. With this in mind I decided to implement the WebSocket as a service in the front-end of application. The VIA Web ap-plication is built on the previously explained MEAN-stack so a WebSocket

(35)

4.2. VIA WEB APPLICATION CHAPTER 4. IMPLEMENTATION

Figure 4.2: High level architecture of solution

library for AngularJS was required. I decided to use a WebSocket library de-veloped by Ferrari (2014) called ng-websockets. This is a WebSocket library specifically for AngularJS and can be installed right into the application via the package and dependency manager Bower (Appendix B.1). The service, webSocketService.js, handles the communication with the server. It is im-plemented into the video element in VIA in a way that starting a video opens up the WebSocket connection. The service sends a message identifying itself as a browser when the connection opens. The videoplayer registers listeners to the webSocketService and is now ready to receive and send messages. webSocketService is available in Appendix B.2 and the video player element in Appendix B.3. The WebSocket messages are configured to be sent in the JSON format to bundle more information in each message. This way in ad-dition to the desired command meta data such as title, description and URL to the cover image of the media being played could also be delivered to the smartwatch. How the messages are handled by the different mobile devices is explained under their respective chapters. Listing 4.1 shows an example of how a typical message from the browser client VIA Web application to the server could look.

(36)

4.3. ANDROID WEAR CHAPTER 4. IMPLEMENTATION

Listing 4.1: Example of JSON formatted WebSocket message sent from the browser { ” e v e n t ” : ” webPlay ” , ” d a t a ” : { ” p o s t e r ” : ” h t t p : / / ovp . c l o u d . a c c e d o . t v / i m a g e s / d v d c o v e r /616 a 6 6 7 3 6 c d 7 8 1 b 1 2 9 3 c 7 b 9 b 7 c 7 3 9 6 2 7 . j p g ” , ” t i t l e ” : ” I n t e r s t e l l a r ” ,

” d e s c r i p t i o n ” : ”A gr oup o f e x p l o r e r s make u s e o f a newly d i s c o v e r e d wormhole t o s u r p a s s t h e l i m i t a t i o n s on human s p a c e t r a v e l and c o n q u e r t h e v a s t d i s t a n c e s i n v o l v e d i n an i n t e r s t e l l a r v o y a g e . ” } }

4.3 Android Wear

Initially a native Android Wear app was developed which utilized the Mes-sage API (2.3.2) to send and receive commands to and from the phone which was connected to the server. The final product however is a lightweight An-droid app which uses enhanced notifications to extend functionality to the smartwatch. This solution was the most logical, from a users perspective, to solve the use case provided. But to get a wider view and more depth of the development process using Android Wear both implementations will be explained along with the implementation of the WebSocket library.

4.3.1 WebSockets

For the implementation of the WebSocket protocol in the Android app the client library Java WebSockets (Rajlich (2009)) was used. This was easily done by including the code from Apache Maven (Wikipedia (2015a)) by listing it in the gradle dependencies for the mobile module and importing it to desired file (Listing 4.2). This enables the Android application to connect and communicate with a WebSocket server of our choosing.

Listing 4.2: Include WebSocket library

b u i l d . g r a d l e ( module : m o b i l e ) d e p e n d e n c i e s { c o m p i l e ” o r g . j a v a −w e b s o c k e t : Java−WebSocket : 1 . 3 . 0 ” } M a i n A c t i v i t y . j a v a i m p o r t o r g . j a v a w e b s o c k e t . c l i e n t . W e b S o c k e t C l i e n t ; i m p o r t o r g . j a v a w e b s o c k e t . handshake . S e r v e r H a n d s h a k e ;

(37)

With this client library included in the project an instance of the ab-stract class WebSocketClient can be instantiated with a WebSocket URI and used to set up the connection and event listeners. The class using WebSocketClient must implement the following methods related to the con-nection to be useful: onOpen, onClose and onMessage. To send a message to it’s connected server the instance can use the send method. The onOpen and onClose events allows us to perform certain tasks when the WebSocket connection opens or closes. This is useful for alerts when disconnected or for example sending a message identifying the device to the server when the connection opens. The actual full implementation of this library in the applications can be found in the Appendices C.3 and D.1.

4.3.2 Native Android Wear

As mentioned above initially a native Android Wear application was devel-oped to solve the use case. This means that the applications code is stored, started and runs entirely on the smartwatch device. But due to the nature and limitations of Android Wear explained in the Background section some parts of the code base must be run in the companion app on the phone. An-droid Wear does for instance not have Wi-Fi support as of yet. And since the current use case in question needs to communicate with external systems i.e the web server this had to be handled by the paired phone. This means that the code base for the application is divided into a ’mobile’ module (runs on the phone) and a ’wear’ module (runs on the smartwatch). To be able to communicate the actions performed on the watch to the phone the Message API (2.3.2) was used. This is done by letting your class, in both modules, implement the GoogeApiClient and MessageApi interfaces (Listing 4.3).

Listing 4.3: Implemented interfaces in ’mobile’ module p u b l i c c l a s s M e d i a C o n t r o l S e r v i c e e x t e n d s I n t e n t S e r v i c e i m p l e m e n t s G o o g l e A p i C l i e n t . C o n n e c t i o n C a l l b a c k s , MessageApi . M e s s a g e L i s t e n e r { p u b l i c M e d i a C o n t r o l S e r v i c e ( ) { s u p e r(” M e d i a C o n t r o l S e r v i c e ”) ; } @Override p r o t e c t e d v o i d o n H a n d l e I n t e n t ( I n t e n t i n t e n t ) { i n i t G o o g l e A p i C l i e n t ( ) ; connectWebSocket ( ) ; } . . .

In Listing 4.3 it should be noted that I have chosen to use a IntentService to handle the communication from the phone to the server. This is because IntentService’s in Android can be run in a background thread allowing

(38)

op-4.3. ANDROID WEAR CHAPTER 4. IMPLEMENTATION

erations to be done even though the app is not open in the foreground. This is crucial to improve the user experience. Both in terms of app responsive-ness but more importantly having the ability to send messages between the devices even though the phone is not running the app in the foreground. In this case the background service MediaControlService is started from the MainActivity directly when the app starts by using the provided service method: startService(mediaControlService). The steps to set up the con-nection, done in onHandleIntent above, is as follows: first do the necessary initialization of the GoogleApiClient, connect and then add the MessageAPI listener (Listing 4.4). This is done for both devices.

Listing 4.4: Initialize GoogleApiClient, connect, add listener p r i v a t e v o i d i n i t G o o g l e A p i C l i e n t ( ) {

m G o o g l e A p i C l i e n t = new G o o g l e A p i C l i e n t . B u i l d e r (t h i s) . addApi ( Wearable . API )

. a d d C o n n e c t i o n C a l l b a c k s (t h i s) . b u i l d ( ) ; i f( m G o o g l e A p i C l i e n t != n u l l && ! ( m G o o g l e A p i C l i e n t . i s C o n n e c t e d ( ) | | m G o o g l e A p i C l i e n t . i s C o n n e c t i n g ( ) ) ) m G o o g l e A p i C l i e n t . c o n n e c t ( ) ; } @Override p u b l i c v o i d onConnected ( Bundle b u n d l e ) { Wearable . MessageApi . a d d L i s t e n e r ( m G o o g l e A p i C l i e n t , t h i s) ; } @Override p u b l i c v o i d o n M e s s a g e R e c e i v e d ( MessageEvent m e s s a g e E v e n t ) { //Do s t u f f } . . .

Once the connection is established a message API, not very unlike the one described for WebSockets, can be used. The Wearable.MessageApi.addListener( mGoogleApiClient) will register a listener that will be notified when receiv-ing a message. Handlreceiv-ing of said message is done in the method onMes-sageReceived (MessageEvent messageEvent) which can perform the intended task depending on the message. Messages are sent asynchronously by ex-tending the AsyncTask class to prevent blocking of the UI thread. Each button in the wear app has a click listener which sends a message to the phone via the MessageAPI. This in turn converts the message to JSON and forwards the command to the server. The server registers that the phone has sent a command and relays it to the VIA application. The message handlers in VIA mentioned above performs the desired command e.g. paus-es/starts video playback. Since the wearable app’s layout does not require any updates depending on interaction done on the phone or VIA no events need to be implemented to handle message receiving. Implementation of

(39)

the ’wear’ module of this application is shown in Appendix C.2. The im-plementation of the ’phone’ module including MainActivity.java and the background service MediaControlService.java handling WebSockets can be found in Appendix C.3 and Appendix C.4 respectively.

4.3.3 Extended Notification

As previously stated an alternate solution to the one explained above was developed to solve the use case. This application utilizes the ability to ex-tend notifications from the phone to the wearable device. This functionality work out of the box with Android Wear and is easily implemented if the existing Android application already supports notifications. By implement-ing the media control in the notification interface the user is able to control the media from both the phone and the wearable, and they are automat-ically in sync. With this solution of extending an interactive notification from the phone the entire code base is written and runs on the phone. The notification is simply extended, customized and displayed on the wearable through the inner workings of Android Wear using WearableExtender. In Android development there is a builder class for creating notification objects (Notification.Builder) and a manager class for managing these (Notification-Manager). When creating the notification a set of parameters is required to specify the attributes of the notification. This includes but is not limited to attributes such as: title, icon, content title, content text, style, actions and extenders. Notification.Builder shown in Listing 4.5.

Listing 4.5: Notification.Builder // C r e a t e a W e a r a b l e E x t e n d e r t o add f u n c t i o n a l i t y f o r w e a r a b l e s N o t i f i c a t i o n . W e a r a b l e E x t e n d e r w e a r a b l e E x t e n d e r = new N o t i f i c a t i o n . W e a r a b l e E x t e n d e r ( ) . s e t C o n t e n t A c t i o n ( 1 ) ; // C r e a t e t h e n o t i f i c a t i o n N o t i f i c a t i o n . B u i l d e r n o t i f i c a t i o n B u i l d e r = new N o t i f i c a t i o n . B u i l d e r (t h i s) . s e t S t y l e (new N o t i f i c a t i o n . M e d i a S t y l e ( ) . s e t M e d i a S e s s i o n ( m S e s s i o n . g e t S e s s i o n T o k e n ( ) ) . setShowActionsInCompactView ( 0 , 1 , 2 ) ) . s e t V i s i b i l i t y ( N o t i f i c a t i o n . VISIBILITY PUBLIC ) . s e t S m a l l I c o n (R . d r a w a b l e . a c c e d o l o g o ) . s e t C o n t e n t T i t l e ( name ) . s e t C o n t e n t T e x t ( d e s c r i p t i o n ) . a d d A c t i o n (R . d r a w a b l e . i c a c t i o n v o l u m e d o w n , ” ”, lowerVolumePending ) . a d d A c t i o n ( p l a y i n g ? R . d r a w a b l e . i c a c t i o n p a u s e : R . d r a w a b l e . i c a c t i o n p l a y , p l a y i n g ? g e t S t r i n g (R . s t r i n g . a c t i o n p a u s e ) : g e t S t r i n g (R . s t r i n g . a c t i o n p l a y ) , p l a y i n g ? p a u s e P e n d i n g I n t e n t

(40)

4.3. ANDROID WEAR CHAPTER 4. IMPLEMENTATION : p l a y P e n d i n g I n t e n t ) . a d d A c t i o n (R . d r a w a b l e . i c a c t i o n v o l u m e u p , ” ”, r a i s e V o l u m e P e n d i n g ) . s e t L a r g e I c o n ( p o s t e r A r t ) . e x t e n d ( w e a r a b l e E x t e n d e r ) ; // Get an i n s t a n c e o f t h e N o t i f i c a t i o n M a n a g e r s e r v i c e N o t i f i c a t i o n M a n a g e r n o t i f i c a t i o n M a n a g e r = ( N o t i f i c a t i o n M a n a g e r ) g e t S y s t e m S e r v i c e ( NOTIFICATION SERVICE) ; // B u i l d t h e n o t i f i c a t i o n and i s s u e s i t w i t h n o t i f i c a t i o n manager . n o t i f i c a t i o n M a n a g e r . n o t i f y ( n o t i f i c a t i o n I d , n o t i f i c a t i o n B u i l d e r . b u i l d ( ) ) ;

The three attributes to make special note of in this case are style, actions and extenders. The setStyle method lets the developer use the pre-defined Notification.MediaStyle which automatically gives us a notification layout style suitable for media control.

The ability to add actions to the notification lets the user interact di-rectly on the notification i.e. control the media content of VIA Web. An action consist of a icon, a title and a PendingIntent. Intents in Android are abstract descriptions of operations to be performed. When the user interacts with a action the PendingIntent is triggered which launches the IntentService ActionIntentHandler. The IntenService can handle operations asynchronously and uses the attribute added to the intent with .putEx-tra() in the PendingIntent to identify and broadcast the desired operation back to MainActivity. This in turn sends the corresponding message to the WebSocket server in the same way as previously explained for the native application (Appendices D.1, D.2).

As stated before one needs to implement the WearableExtender to cus-tomize and extend the notification to the wearable. This is as simple as creating an instance of Notification.WearableExtender and adding this to the extend method of the notification as shown in Listing 4.5. The setCon-tentAction method sets which notification action shall be available directly on top of the wearable notification.

Unlike the native application this solution handle messages received from the web server and update the UI of the notification accordingly. The app reads the command received from the server and changes the play to pause (both icon and action) in the notification, or vice versa depending on the command. This way the Web application and the Android application is in sync and it is obvious to the user if the content is playing or not. To enhance the user experience and appearance of the application the title and poster art of the media that is actually playing have been added to the notification. This is done by using the meta data sent in the JSON message (Listing 4.1) from the server to set the title and retrieve the poster. The poster art is retrieved from the web application through a InputStream

(41)

4.4. APPLE WATCH CHAPTER 4. IMPLEMENTATION

using the provided poster URL and created with a BitmapFactory before it is assigned as the notification icon. A check to see if new content is playing prevents unnecessary requests and retrievals of the poster art.

Entire implementation of this solution is available in Appendices D.1 and D.2.

4.4 Apple Watch

What follows is the implementation of the aforementioned use case on the Apple Watch platform. The section is divided into a subsection of the WebSocket library and its implementation along with a subsection with the implementation of the actual WatchKit application.

4.4.1 WebSocket

With the design approach of the solution for this use case a WebSocket li-brary is needed for the app to function. There are a variety of client side implementations working with iOS devices, the one I have chosen is called SocketRocket. SocketRocket is a WebSocket client side implementation that is written in Objective-C and works with both iOS and OSX. The easiest way to include this library and get the right dependencies is to use CocoaPods. CocoaPods is a dependency manager with a great deal of libraries avail-able which make including and managing these more simple. CocoaPods is available to install through Ruby and can later add desired dependencies to the project easily by editing a dependency file and installing them in the terminal. Once the SocketRocket dependency is in place the library can be imported in the standard way (Listing 4.6.

Listing 4.6: Import SocketRocket

#i m p o r t <S o c k e t R o c k e t / SRWebSocket . h>

Once this is done the WebSocket connection can be established and the SocketRocket API used to receive and send messages to the web server as shown in Listing 4.7.

Listing 4.7: The SocketRocket API

SRWebSocket ∗ webSocket ; // Connect t o s e r v e r − (v o i d) connectWebSocket { i f( webSocket == n i l ) { webSocket . d e l e g a t e = n i l ; webSocket = n i l ; N S S t r i n g ∗ u r l S t r i n g = @”ws : / / 1 9 2 . 1 6 8 . 1 . 5 2 : 1 3 3 8 ”;

(42)

4.4. APPLE WATCH CHAPTER 4. IMPLEMENTATION

SRWebSocket ∗ newWebSocket = [ [ SRWebSocket a l l o c ] initWithURL : [ NSURL URLWithString : u r l S t r i n g ] ] ; newWebSocket . d e l e g a t e = s e l f ;

[ newWebSocket open ] ; }

}

#pragma mark − SRWebSocket d e l e g a t e

// webSocketDidOpen

− (v o i d) webSocketDidOpen : ( SRWebSocket ∗ ) newWebSocket { webSocket = newWebSocket ;

[ webSocket s e n d : [ N S S t r i n g s t r i n g W i t h F o r m a t :@” H e l l o from %@”, [ U I D e v i c e c u r r e n t D e v i c e ] . name ] ] ;

}

// d i d F a i l W i t h E r r o r

− (v o i d) webSocket : ( SRWebSocket ∗ ) webSocket d i d F a i l W i t h E r r o r : ( NSError ∗ ) e r r o r {

[ s e l f connectWebSocket ] ; }

// didCloseWithCode

− (v o i d) webSocket : ( SRWebSocket ∗ ) webSocket didCloseWithCode : ( N S I n t e g e r ) c o d e r e a s o n : ( N S S t r i n g ∗ ) r e a s o n wasClean : (BOOL) wasClean {

[ s e l f connectWebSocket ] ; }

// d i d R e c e i v e M e s s a g e

− (v o i d) webSocket : ( SRWebSocket ∗ ) webSocket d i d R e c e i v e M e s s a g e : ( i d ) m e s s a g e { // NSLog (@” R e c e i v e d m e s s a g e : %@” , m e s s a g e ) ; } // Send a m e s s a g e [ webSocket s e n d : /∗ Payload h e r e ∗/] ;

4.4.2 WatchKit App

As we have learned from the theoretical background the entire code base for third party applications for Apple Watch is both stored and run on the companion iPhone. It is located as an extension to the containing iOS appli-cation. The only thing stored on the actual watch is the Storyboard which the phone can send updates to. Since the code already is stored and exe-cuted on the phone only the elements on the Storyboard and the WatchKit extension need to communicate for this solution. The UI elements sends actions to the extension and the extension send updates to the UI. The WebSocket implementation explained above is simply placed in the Inter-faceController.m file and the connection is opened as soon as the user starts the app on the Apple Watch. Interface Builder in Xcode was used during

Cross-platform Development for Wearable Devices

Institutionen f¨

or datavetenskap

Department of Computer and Information Science

Final thesis

Cross-platform Development for

Wearable Devices

Gustav Beck-Nor´

en

LIU-IDA/LITH-EX-A–15/038–SE

2015-06-10

Final thesis

Cross-platform Development for

Wearable Devices

Gustav Beck-Nor´

en

LIU-IDA/LITH-EX-A–15/038–SE

2015-06-10

Abstract

Preface

Contents

List of Figures

Chapter 1

Introduction

1.1

Motivation

1.1.1

Wearables

1.1.2

Cross-platform

1.1.3

Accedo

1.2

Purpose and Aim

1.3

Research questions

1.4

Scope and Limitations

1.5

Approach

Chapter 2

Theoretical Background

2.1

Smartwatches

2.1.1

Challenges and Limitations

2.1.2

Second screen

2.2

Cross-platform

2.3

Android Wear

2.3.1

Google Play Services

2.3.2

Architecture

2.4

Apple Watch

2.4.1

WatchKit

2.4.2

WatchKit architecture and life cycles

2.5

Accedo VIA Web

2.5.1

MEAN Stack

2.5.2

Architecture

Chapter 3

Method

3.1

Process

3.1.1

Pre-study

3.1.2

Implementation

3.1.3

Evaluation

Chapter 4

Implementation