Evaluating the real-time capabilities of Zephyr on UP Squared - a pilot study with Zephyr and ACRN

(1)

V¨

aster˚

as, Sweden

Thesis for the Degree of Master of Science in Computer Science with

specialisation in Intelligent Embedded Systems 30.0 credits

EVALUATING THE REAL-TIME

CAPABILITIES OF ZEPHYR ON UP

SQUARED - A PILOT STUDY WITH

ZEPHYR AND ACRN

Charlie Kotro

cko14001@student.mdh.se

Jimmy M¨

akkeli

jmi14002@student.mdh.se

Examiner: Thomas Nolte

M¨

alardalen University, V¨

aster˚

as, Sweden

Supervisors: Anna Friebe

M¨

alardalen University, V¨

aster˚

as, Sweden

Shaik Mohammed

M¨

alardalen University, V¨

aster˚

as, Sweden

September 25, 2020

(2)

Abstract

In this study we have investigated if using Zephyr (User VM) and ACRN (Hypervisor) on the UP Squared (N3350) board are suitable for use in real-time scenarios. This was achieved by conducting a series of tests on Zephyr’s scheduling API functions such as busy-wait and sleep under different system configurations. The tests involved running a task in isolation and taking timestamps of the current system cycle count before and after each function. To do this, we used the Common Trace Format functionality in Zephyr to collect execution traces, which were then analyzed using Babeltrace to convert the binary trace into human-readable form. The difference between the before and after timestamp were compared to the configured clock frequency to see if the system behaved as expected. Our findings in this report suggest that Zephyr and ACRN on the UP Squared board are not suitable for use in real-time scenarios due to large differences in system cycle count depending on the configuration.

(3)

List of Figures

1 Overwiew of different hypervisor types . . . 4

2 High level overview of ACRN . . . 5

3 Device Passthrough . . . 5

4 Trace results of Sleep compared to the baseline in a logarithmic scale (x-axis). . . . 18

5 Trace results of Sleep compared to the baseline. . . 18

6 Trace results of Busy-wait compared to the baseline. . . 19

7 Trace results of Busy-wait compared to the baseline. . . 19

8 Trace results of Loop compared to the baseline. . . 20

(5)

Acknowledgements

We want to thank our supervisors Anna Friebe and Shaik Mohammed, for all their help during this thesis. They listened to us whenever we had problems during every phase of the work, they also provided us with a lot of feedback as well as help on how to proceed forward. We would also like to thank our families for always providing support and giving us that nudge forward.

(6)

1. Introduction

Open-source software (OSS) is software that is freely available to anyone, and you are allowed to make changes to the software. This is different compared to closed-source software, which is usually bought, and you are not allowed to freely change the software. Both types of software come with pros and cons. This is also true when it comes to the development of both types of software. Open-source software development is different from traditional development, and an example where these two approaches to development differ is when it comes to the documentation. The documentation for closed-source software tends to be more extensive than OSS documentation. Many studies have shown that OSS development does not use many of the best practices, but does so without loss in quality [1]. But OSS development does keep some of the best practices such as central ownership, system testing, and planning and strategy meetings, among others [1]. It is not easy to select an OSS for your system since there are no set standards for determining the quality of the software [2]. Open-source software, have several benefits such as lowered development and maintenance costs, and the ability to customize the software [3].

Virtual Machine Monitors (VMM), also called Hypervisors, sit between the hardware and the operating system and manage resource distribution. This allows the system to run multiple other operating systems on the same hardware without them affecting each other [4]. ACRN is an open-source lightweight type-1 hypervisor, which means that it runs directly on the hardware as opposed to running on an operating system. It was built for development for the Internet of Things (IoT) [5]. ACRN was developed as collaborative work between the Linux foundation and Intel because of Intel’s expertise in virtualization [6]. Zephyr OS [7] is an open-source operating system for resource-constrained systems. It has a small memory footprint and supports a lot of features, such as multiple scheduling algorithms, among others. Another project from the Linux foundation is Akraino-edge stack, which has an integration project called Time-Critical Edge Compute [8], which uses both Zephyr and ACRN. The three main reasons for opting to use Zephyr and ACRN are: both systems are considered lightweight, made for IoT development, and there is no previous research of using them together.

Edge-computing is a paradigm in which the resources are brought to the edges of the computational networks. By shifting where the computation and storage take place, it becomes possible to lower latency and decrease the bandwidth usage [9]. Some examples of edge-computing stacks are Akraino Edge Stack, Apache Edgent, and Azure IoT Edge [9]. Because of the increasing demands for more responsive applications and less usage of bandwidth, the real-time capabilities of edge computing are suitable to facilitate these types of applications or services [10]. Some examples of applications that would require real-time performance are multiplayer games, cloud games, and streaming services [10].

As mentioned earlier in this section, both ACRN and Zephyr are made for IoT development. But according to some researchers, most existing hypervisors have a hard time fulfilling real-time requirements [5]. Therefore it becomes interesting to investigate if these two systems can be used for real-time systems.

1.1. Problem formulation

This study will focus on finding out if Zephyr and ACRN are suitable for use in real-time systems when ran on UP Squared. It is not known how suitable these systems are for use in real-time systems, and to our knowledge, there is no previous research that has used these two systems together in real-time systems. Therefore, this study will focus on the aspects related to common functionalities that are used when scheduling tasks in the system, such as sleep functions and busy-waits. By using a hypervisor, extra overhead is added to the system because of this extra layer that each request has to go through. But this extra amount of overhead is not known, therefore this needs to be studied to be able to determine if the systems can be used in real-time scenarios.

(7)

1.1..1 Research questions

This is our research question that we are going to look into:

• RQ: What is the performance of common scheduling API methods when run on Zephyr standalone and when Zephyr is run via ACRN?

Our research question was chosen because we are trying to determine if ACRN and Zephyr can be used for real-time systems, and therefore it is important to investigate how these systems perform when scheduling tasks, which is an important part of a real-time system. As real-time systems need to be deterministic, it is important to know by how much these functions can differ from the specifications.

(8)

2. Background

In this section, we will talk about topics that are related to this work, such as hypervisors and quality assessment for open-source software. Hypervisors are relevant as we are using a hypervisor ourselves, and we are evaluating whether it is ready to be used in real-time scenarios. But to be able to use the hypervisor, we need to have an operating system, which the hypervisor can run, and therefore we opted to use Zephyr, as this is also a system which has not been used in this type of work before. By assessing the quality of these two systems, we can determine if they can be used for real-time systems. A common method for gathering data related to scheduling is to use various tracing tools such as Babeltrace and Common Trace Format. Both the hypervisor and the RTOS we have chosen are open-source systems.

2.1. Hypervisors

Hypervisors, or Virtual machine monitors, can be divided into two categories: type-1 (bare-metal) or type-2 (hosted hypervisor). The type-1 hypervisor interfaces directly with the hardware and does not need an operating system, unlike the type-2 versions [11]. Figure 1 shows the differences between type-1 and 2 hypervisors. Some examples of type-1 hypervisors are XenServer, KVM, and VMware ESXi [12]. And some examples of type2 hypervisors are VMware workstation, -fusion, and VirtualBox [13]. Hypervisors allow multiple different virtual operating systems to run at the same time while still having access to the same hardware. The hypervisor manages and monitors the system resources while also handling requests for these resources. An advantage of using hypervisors is the isolation they give, in the sense that if one of the guest operating systems is having problems, these problems do not affect the other guest systems [11].

(9)

2.2. ACRN

This section contains information about ACRN from the paper ACRN: A Big Little Hypervisor for IoT Development [5] and the ACRN documentation [15].

When it comes to IoT usages, it is not easy for existing solutions to handle real-time constraints. Figure 2 shows a high-level overview of ACRN and the different operating systems it can run. ACRN is developed from scratch to be a lightweight type-1 hypervisor. This means that ACRN has a small code size and impact on performance, which is critical for industrial usage. Being a type-1 hypervisor, ACRN does not have to rely on RTOS services. The developers argue that this makes porting it to other hardware architectures easier since there is no need to port an RTOS first, which can be a problem for other hypervisors since it would require extensive source code modification. ACRN also has the benefit of not binding to a specific RTOS, which means that it works with various operating systems on demand. Another benefit is it being open source and free. [5]

Figure 2: High level overview of ACRN [15].

ACRN uses x86 as a starting point, compared to other solutions which do not have mature support for x86 embedded systems [5]. Some of the features that ACRN supports are rich I/O virtualization, device pass-through and, mediated pass-through. According to ACRN documentation, it is possible to allocate a physical device exclusively to a guest operating system, which allows almost near-native performance [15]. In order to avoid complex CPU-emulation most CPU features such as CPUID, MSR, etc. are passed through. It also uses Intel VT-x to help with CPU virtualization. [5]

Figure 3: Device Passthrough [15].

The ACRN hypervisor supports three types of VMs: Service VM, User VM, and Real-Time VM, each able to run an OS. The User VM and RTVM are isolated in order to guarantee that the performance of the application is running in real-time. By using dedicated pass-through, which can be seen in Figure 3, the RTVM minimizes the use of emulated devices at boot time. The

(10)

hypervisor passes through access to hardware for the RTVM to optimize real-time performance and to avoid additional latency from the Service OS. [5]

To achieve isolation, the CPU core and memory partitioning are spatially isolated. This means that the hardware can not be accessed by a VM without it being assigned to it. Therefore, other VMs have no access to process this data. [5]

ACRN can be run using four different scenarios: SDC (Software Defined Cockpit) with 2 VMs, IU (Industrial Usage) without Safety VM, IU with Safety VM, and Logical Partition. However, for our UP Squared board, the only verified scenario is SDC with 2 VMs.

2.3. Zephyr

Zephyr is a small-footprint kernel intended for use in embedded systems that are resource con-strained. These systems include sensors that are wearable, smart-watches, and IoT applications, among others. It supports a lot of different architectures and boards. Zephyr supports several dif-ferent scheduling algorithms and kernel services. According to the documentation, the system also features memory protection in the form of stack-overflow protection, kernel and object permission tracking, and thread-level memory protection. [7]

2.4. Open-source

Open-source software is software that is available to anyone, and you are allowed to change, in-spect and redistribute the modified software [16]. This type of software is often supported by a community, which helps out with tasks such as debugging, testing, and even developing new fea-tures. A few of the advantages for companies using OSS is that it can lower the development costs, maintenance costs, and the company can change the software to suit their needs. But this does not mean that adopting OSS is risk-free, as studies have shown that most open-source projects end up in failure [3]. Keeping most open-source projects alive can be quite hard since the group of core developers is usually small and without financial backing [17]. But they also need the help of the community to debug, test, and maintain the software. Maintaining this community can be hard if the contributors do not feel that they can learn something new or take on challenges. Recognition was also a motivating factor for contributors as studies have shown that most programmers are looking for social status [1].

2.4..1 Quality assessment of open-source software

Models and metrics are suitable methods to assess the quality of software [2]. To assess the quality of software means to look at the different parts that make up the software, e.g. the documentation and the source code. But as mentioned before in the introduction, the lack of any standards makes this quite challenging [18]. Models and metrics are not equally useful for everybody, e.g. models are more useful to system-designers than the end-user [2]. An advantage that metrics have is that they can be used by anybody as they are just measurements. Some studies have shown that over time as the software is undergoing development, the quality tends to decline due to increased complexity. This means that it becomes harder to maintain the software. A lot of different factors contribute to this loss of quality, such as lines of code, high coupling, or lower cohesion in methods [19].

2.5. Software tracing

Software tracing is a method of logging and recording execution data for a program. Some examples of tracing software are Common Trace Format, SEGGER SystemView, and DTrace. Software traces are useful for debugging as they contain valuable information about the execution of the system. One of the challenges with software tracing is getting valuable system information without affecting the system in question. When a purely software-based tracing approach is used, it affects the execution of the software as the original source code is extended with tracing calls. This is

(11)

an important factor to take into consideration for real-time systems, as the system behavior is changed. This is not the case with hardware-implemented tracing methods. [20]

2.5..1 Common Trace Format - CTF

Common Trace Format is a binary tracing format resembling C. It uses a declarative language called Trace Stream Description Language (TSDL) and it allows for different stream layouts. A packet contains:

• A header

• A context (which is optional)

• Zero or more events which call all contain a header, stream specific context (optional), an event specific context (optional), and a payload.

The different data types that are supported in CTF are integers, null-terminated strings of bytes, and floats, among others. All of the mentioned information is taken from the homepage [21].

2.5..2 SEGGER SystemView

SEGGER SystemView is a program that can record and visualize what happens in the system during run-time. SystemView can record various events that happen in the system, such as Inter-rupt Service Routines, timers, and user-defined events, among others. According to the developer’s homepage, the system limits the amount of overhead introduced in the system by only recording a small amount of data for each event. The developers also argue that when the system runs 10000 events per second the overhead from SystemView is less than 1%. [22]

2.5..3 DTrace

DTrace is another tracing software that allows the user to trace all software according to their website. It uses the language D for writing scripts. DTrace stands for Dynamic Tracing and is referring to that it can run and change tracing scripts during program execution. Another feature that DTrace supports is static tracing, where the user defines trace-points before deployment. By providing filters that can filter the data in the kernel before the data is sent to the user, DTrace can reduce the overhead of presenting and gathering data. [23]

2.6. Babeltrace 2

Babeltrace 2 is the reference implementation of CTF [21, 24]. It is an open-source program that converts or processes traces. Babeltrace 2 is a completely rewritten version of the old library that comes with a lot of changes, some of these changes are full plugin support, Babeltrace 2 can run on any major operating system, and expanded certain features of the old library.

(12)

3. Related work

3.1. Hypervisors

J. Hwang et al. [25] compared four different hypervisors to each other. The researchers compared the different hypervisors with regards to CPU, memory, disk I/O, and network I/O. The results from the study showed that there was no clear indication of which hypervisor that was the best, rather that it depended on the workload. This was also the case in another study made by S. Shirinbab et al. [12] They compared three different type-1 hypervisors with each other when it comes to CPU-utilization and Disc-utilization during different scenarios. The results showed that the three different hypervisors performed differently in each case and that there was no conclusive evidence that one hypervisor was better than the others. This was not the results in the study by S. Pawar and S. Singh [26], in which they compared VMware ESXi 6.0 and Citrix XenServer 6.5. In their study, they compared the two hypervisors based on CPU, memory, disk, and system up-time, on the guest operating system Microsoft Windows Server 2008 R2 (64-bit). The results showed that the ESXi hypervisor was significantly better than XenServer.

J. Ruh and W. Steiner [27] compared the system latencies of the hypervisors: KVM, ACRN and, Xen RTDS. They evaluated each of the hypervisors based on four requirements, and they determined that none of the hypervisors available fulfilled all of the requirements. Based on this, the authors state that none of the hypervisors qualified as deterministic virtualization. Experiments using the RTEval tool was conducted to measure the system latency of each hypervisor. Each hypervisor was tested twice, the first test involved running one VM, and the second test involved running two VMs at the same time. Their results showed that KVM and ACRN had low average system latencies, but could have peaks up to 1.968ms. On the other hand, Xen had higher average system latency, but a maximum latency of 267µs. From these results, they drew the conclusion that KVM and ACRN are not suitable for acting as control VMs because of the peaks in latency, almost reaching two milliseconds.

3.2. Quality assessment for open-source projects

Jospeh Tassone et al. [2] reviewed the body of research to find out which characteristics that affect the quality of source projects the most. There are four main characteristics in which open-source projects (OSPs) differ from closed-open-source projects (CSPs): different levels of programmers, frequent beta releases, platform independence, and support for many run- and compile-time con-figurations. OSPs also differ from CSPs in regard to the development process as they do not use more traditional processes such as the Waterfall model and that the development process changes over time as the project grows. An advantage OSPs have over CSPs is that they are available to anyone, which means that the larger the community around the project becomes, the more people can help out. Two different approaches are used to measure quality: models and metrics. But these approaches are useful for different people in the project. Metrics is a useful tool to determine if there are problems within the project and can also be used to analyze errors. Models are used to bring together different attributes related to quality to get a final result of the quality of the software. An advantage of models is the utilization of metrics to establish a larger picture over the process. But a disadvantage is that they can not be used by anyone, as they need to have all the information needed to establish a working model. The authors conclude that both metrics and models are suitable for determining the quality of the software. Therefore, it is preferable to concentrate effort on the idea of maintainability. Care should be taken when gathering metrics, as there are a lot of tools that are not comparable with each other, and there are no standards set yet.

Other researchers have also looked at the differences between OSPs and CSPs but from another point of view. Mark Aberdour [1] wrote an article about what lessons can be learned and how the development process is different between OSPs and CSPs. Some of the highlighted differences between OSPs and CSPs were: effort put into planning, level of documentation, the methodology regarding testing, peer review, and people management. Peer review was something that con-tributed a great deal to the quality of the code if the comments and the code got quick responses,

(13)

which helped keep the contributors interested. When it comes to managing the project, OSPs have a different mindset. In OSPs, the development process is more people-focused, which means people volunteer to work on certain parts of the system. This results in a culture that fosters creativity and innovation compared to more traditional development processes. The community is highlighted as a key element for success in OSPs, as it is the community that builds new features, test and debug the system. But to sustain the community, the project must attract volunteering programmers and motivate people to work by giving opportunities to take on challenges, learn new technology or tools. The author concludes that the study raised many questions but also found some guidelines for open-source software, some of which were: a large sustainable community, code modularity, good documentation, rapid release cycles, and that the culture and the environment are equally important as the system design.

3.3. Quality models for open-source projects

Normi Sham Awang Abu Bakar and Nadiah Arsat [16] studied which factors affected the quality of open-source systems the most. In the study, McCall’s Quality Factor Model was used to determine the quality of two different open-source systems used by the Malaysian government: MyMeeting and MyTaskManager. The researchers also used six object-oriented measurements by Chidamber and Kemerer to measure quality. The PHP Depend tool was used to visualize the different statistics of the two different systems. One result from the study was that there is no correlation between the number of lines of code and the depth of the inheritance tree. Another result was that the weighted methods per class increased as the lines of code increased, which suggested to the researchers that the lines of code are a good indicator of complexity and coupling. This meant that the factors that affected the quality the most are: correctness, maintainability, efficiency, and usability.

Won Jun Sung et al. [18] proposed in their work a quality model for selecting open-source software (OSS) components. This model determines the quality of an OSS component by looking at the different characteristics of the component. The model has four main categories: functionality, reusability, usability and portability, and ten sub-categories. The researchers conclude that the model provides basic measurements for the quality and that it is expected to be used to gather practical data.

3.4. Risk management for open-source projects

Xavier Franch et al. [3] propose a framework for risk management in OSS adoption processes. This proposed framework will provide the companies with tools and methods that are centered around a community-based development approach. Their framework builds upon the concepts of business and software ecosystems. These different ecosystems handle different aspects of the development, the business ecosystem handles all things related to business such as production, distribution, and costs, while the software ecosystem handles all matters about software such as licensing problems and release schedules. The platform provided by the framework is divided into two tiers: the decisional and the technological. The technological tier provides the necessary observations needed for the decisional tier to be able to make any decisions. One important duty of the decisional tier is risk management, by utilizing the data provided from the technological tier and the business, it can detect potential risks related to the ecosystems. One of the most common mistakes companies make when adopting OSS is having insufficient risk management, which can lead to costly failures. A scenario-based approach is planned to validate the framework.

Nguyen Duc Linh et al. [28] also studied the risks that come with using open-source software. The researchers studied different methods for assessing the risk involved with using OSS components in a system. Two of the different methods were: Qualitative and Quantitative analysis. In qualitative risk assessment, each risk is evaluated based on the rate that they would occur and the severity. Some examples of tools that can help with qualitative risk management are the risk matrix and risk breakdown structure. In quantitative risk assessment, a risk matrix is also used, which the project manager uses. Risk assessment needs to be re-evaluated every time a major risk occurs, after phase end-reviews, and when the project needs to be re-planned. When a component has

(14)

been selected, different methods can be used to integrate it into the project, some examples that the researchers list are: having an expert as the supervisor, using a version control system, and backup systems periodically. A case study was conducted in order to learn what difficulties there are when integrating OSS components. The results from the case study showed that the community for Broadleaf was not strong, and Broadleaf’s products were not diverse. They also found that the cause for selecting unsuitable components came down to three different causes. The first cause was that the team had not fully defined their requirements. The second cause was that the team had only checked the homepage for Broadleaf and not checked the reviews of the system. The third and last cause was that the team had not compared Broadleaf to any similar products.

3.5. Summary

All of these different topics are connected to our work in at least one way. As mentioned earlier in the introduction text in the background section, both systems we have chosen to work with are open-source. There are various risks involved when selecting OSS and also the quality can be varying as well. These risks can also be higher depending on the quality of the proposed software e.g. if the documentation is not understandable, then risks for failure might increase and therefore the company might opt to select another software instead. One of the ways to assess the quality of a software is to use metrics and models. These can be used to evaluate different parts of the system as well as different aspects such as portability, quality of documentation, and how easy the system is to use among others.

(15)

4. Method

We evaluated if Zephyr and ACRN can be used for real-time systems by conducting a series of tests on different configurations, e.g. different service VMs, clock frequencies, and tick-rates, which can be seen in Table 1. We ran the tests using two different clock frequency and tick-rate baseline configurations, these were 19.2 MHz with a tick-rate of 10000 ticks/s, and 25 MHz with a tick-rate of 100 ticks/s. These are the default settings when configuring Zephyr stand-alone and Zephyr for ACRN during the build process. We then used these clock frequencies to calculate the difference between expected clock cycles taken and actual clock cycles taken. We used clock cycles as these gave us the most accurate results, but also because many system functionalities like deadlines, and busy-waits, etc. depend on clock cycles. As we have no expected value for the loop, we chose to use the Zephyr binary that was closest to having the correct cycle count (for sleep and busy-wait) as a baseline value for the loop. Strip means that the compiled executable does not contain debug information.

UART was used for serial communication and Common Trace Format (CTF) for creating traces. We opted to use CTF because it was readily available in Zephyr, and we did not have to buy any additional resources. We had planned to use SEGGER SystemView as it could present the results very nicely. However, we could not find any information regarding if it was possible to use it on our board or not.

Configuration Clock frequency (MHz) Tick-rate (Ticks/s)

ACRN Debian 19.2 10000

ACRN Debian Strip 19.2 10000

ACRN Clearlinux 19.2 10000

ACRN Clearlinux Strip 19.2 10000

ACRN Ubuntu 19.2 10000

ACRN Ubuntu Strip 19.2 10000

Zephyr non-tickless 19.2 10000

Zephyr non-tickless Strip 19.2 10000

Zephyr tickless 19.2 10000

Zephyr tickless Strip (Baseline for loop) 19.2 10000

ACRN Debian 25 100

ACRN Debian Strip 25 100

ACRN Clearlinux 25 100

ACRN Clearlinux Strip 25 100

ACRN Ubuntu 25 100

ACRN Ubuntu Strip 25 100

Zephyr non-tickless 25 100

Zephyr non-tickless Strip 25 100

Zephyr tickless 25 100

Zephyr tickless Strip (Baseline for loop) 25 100

Table 1: Table showing the different configurations that we ran tests on.

The board we used is an UP Squared with an Apollo Lake Intel N3350 processor, 2 GB DDR4 RAM, 32 GB eMMC. The versions of Zephyr and ACRN we have used are Zephyr v2.3.0 and ACRN v2.0 with SDC as the scenario configuration. We created a test program which ran a single thread containing three test functions. The functions that we tested were sleep, busy-wait, and a loop which ran for 100000 iterations. The sleep and busy-wait were configured so that they would run for one second. To reduce the risk for errors, we ran the different functions four times each and averaged the results. This can be seen in Table 2, which shows an example using the baseline configuration running the sleep function. During each test, we ran the test program for about a minute before stopping the execution. While the test program was executing, we collected the tracing data in a text file. When we ran Zephyr directly on the UP Squared, we were able to receive the CTF trace via the UART connection. However, when we ran ACRN, we could not

(16)

use the UART interface to capture the data, so we had to in the service VM pipe the CTF trace from a terminal to a text file. Because Zephyr printed out some extra text when it booted up the kernel, we had to remove it before we could read it with Babeltrace. To do this we used the sed command.

We created a set of timestamp functions in CTF to be able to see when a specific function had been executed. These functions had a name that corresponded to the executed function e.g. ctf before sleep. We had to configure Zephyr to enable CTF, and we did so by going into a configurations menu when building the RTOS. In this menu, we had to go down to the option Tracing and enable CTF. We changed the size of the CTF buffer to 16 KB because we saw that in some cases there were some gaps in the trace. We also changed the tracing thread waiting period to zero as this lead to a more responsive tracing experience. CTF produces a binary output, and we used Babeltrace to read it. The default format for each timestamp is hh:mm:ss:ns, and because of this, we used a flag that converted them into clock cycles instead. Then to calculate the time taken for each function, we calculated the difference between the before and after timestamps, and this result was divided by the expected cycle count for each tested function, which can be seen in Table 3. This results in a number that describes how many times longer it took to execute each function on every configuration compared to the baseline.

Before sleep After sleep Difference 269221246 288424130 19202884 288424317 307627332 19203015 307627512 326829871 19202359 326830052 346032877 19202825

Table 2: Example showing how the data was collected. This is the sleep function running on Zephyr Strip tickless.

Configuration Sleep duration (cycles) More cycles taken relative to expected value

Zephyr tickless Strip 19202770,75 1,00014431

ACRN Debian 7553942456 393,4345029

ACRN Clearlinux 5961251437 310,4818457

(17)

5. Ethical considerations

In this work, there are no ethical considerations needed to be taken into account because the data or the results will not be related to any company or person.

(18)

6. Implementation and problems

In this section, we will talk about the installation of Zephyr and ACRN, the various problems we had during the installation phase. We will also talk about the solutions that we found to certain problems and also how some problems are still unresolved.

6.1. Zephyr problems

This section will describe the problems we had with Zephyr.

• The sleep function does not sleep for the specified amount of time. • The busy-wait function does not work with every configuration. • Installing Zephyr when following the guide from the documentation. • UART support for earlier versions for tracing data.

When compiling the system, it is possible to specify whether the system should use a tickless kernel and idle. Tickless idle disables periodic system clock interrupts when the kernel is idle. Tickless kernel disables periodic clock system generation and turns the kernel into an event-driven kernel. We built the binaries with both tickless enabled and disabled and measured the time it took to perform functions such as sleep and busy-wait.

One of the problems we had was timing related when using the sleep and busy-wait functions. In Zephyr v2.2.99 (Tickless), the sleep was shorter than specified. This problem seems to be fixed in v2.3 (Tickless). However, when tickless is not used, v2.2.99 would sleep for a shorter time and v2.3 for a longer time than specified. The busy-wait function seems to work with tickless enabled and disabled for v2.3, but only with tickless enabled for v2.2.99. When disabled, the thread would stop responding, and after a while, the system enters the idle state without finishing the remaining instructions in the thread.

We also had problems installing Zephyr SDK v2.3 on another computer, following the guide in the documentation. The problem was related to an environment variable that we had to set, but this was missing from the guide. The solution to this problem was to go back to the guide for version 2.2, and in that guide, that step was explained.

Due to UART support being introduced in v2.2, we have not been able to test these functions on earlier versions of Zephyr. This is because we rely on UART to get our tracing data. This would not have been a problem if we had an alternative way of getting these data.

6.2. ACRN problems

This section will describe the problem we had using ACRN. • Efibootmgr not saving the boot order correctly.

• Huge time differences in functions compared to running only Zephyr. • Documentation and configuration files.

One step in the installation process of the hypervisor is to create a custom entry efibootmgr in Linux. The entry for the hypervisor is required to be the first entry, or the hypervisor will not be initialized correctly during boot. The problem we had is that changes such as removing entries or changing the order of them would not persist when rebooting. Because of this, we had to spend time changing the boot order and rebooting until the board managed to save it. This was not documented when we started our work but was later found by another person that posted a pull request and updated the documentation to include this. Later we would find a solution to this problem that worked on our board (N3350). In the BIOS for the board, there is a setting called

(19)

UEFI harddisk drive BBS priorities that allowed us to force a change to the boot order. This allowed the hypervisor to initialize correctly on the first try when installing the hypervisor. When running our test program on Zephyr using ACRN as the hypervisor, we noticed huge time differences in the tested functions compared to running Zephyr standalone. With one setup, the sleep function took ≈310.44 times as many cycles, the busy-wait ≈1.11 times and our simple loop ≈27.35 times. Then with another setup, the sleep took ≈2.97 times as many cycles, while the busy-wait and loop function took ≈1.56 times and ≈27.77 times. We measured the current clock cycle before and after each function and calculated the difference. This difference was then divided by the cycles expected for the sleep and busy-wait functions, while using Zephyr v2.3 tickless as our baseline for the loop. We chose this as our baseline since the cycles reported for each function closely matched the cycles we expected it to take.

When we noticed this cycle difference, we posted an issue on the ACRN GitHub. We received an answer from one of the developers asking us about which ACRN and Zephyr version we were using. We were running acrn-2019w35.1-140000p with Zephyr 2.2.99. This was the only version of ACRN that we had managed to install on our board. The developer then told us that they had only verified Zephyr v2.1 on ACRN v2.0 and asked us if we could test it again using these versions. However, because of an update to ACRN, our board (N3350) was no longer the default board when building, because it had been changed to the newer (N4200). This led to a problem because the configuration files needed to build the hypervisor on our board for our scenario did not exist. After discussing this, we received a configuration file which made it possible for us to build v2.0 for the N3350 board. But we could still not get measurements for Zephyr v2.1 since it lacked the UART support we relied on to get our trace measurements. Because of this, we had to use newer versions of Zephyr. Since the hypervisor was up and running, we wanted to get back to solving our original problem, posted on July 2nd, 2020, about the cycle differences. But as of writing this, we have yet to receive a response.

When installing ACRN, we had problems following some of the documentation. The guide used for our board (N3350) referenced the installation guide for another board (nuc7i7dnb) and noted changes that had to be made in the installation process. This could make it difficult to keep track of what changes had to be made and when to make these changes. There was also no mention of the flag that had to be used in order to build the hypervisor for the UP Squared board (Both N3350 and N4200). We had to check the makefiles in order to find this flag. As mentioned before, we were told that Zephyr v2.1 had only been verified for ACRN v2.0. According to the developer answering us in the GitHub issue, it was also only verified to run with the logical partition scenario. However, the documentation states that the only scenario that has been verified for Up Squared (both N3350 and N4200) is SDC with 2 VMs. The N3350 board did have scenario configuration files for logical partition mode but not SDC, while N4200 had configuration files for both.

(20)

6.3. Test program implementation

The pseudo-code for the program used to take all our measurements can be seen in Listing 1. The actual code can be found in Appendix B, Listing 2. This program consists of two loops that test the API functions sleep and busy-wait. There is also a loop doing a simple calculation. At the start and end of each iteration, we measure the current system clock cycle by taking a timestamp. These timestamps are then used in order to calculate the number of cycles each function took so we can compare that to the cycles they are expected to take given the system configuration.

1 v o i d f u n c ( ) 2 { 3 i n t i = 4 ; 4 5 // S l e e p 6 w h i l e ( i −−) { 7 t i m e s t a m p c u r r e n t c y c l e ( ) ; 8 s l e e p ( 1SECOND ) ; 9 t i m e s t a m p c u r r e n t c y c l e ( ) ; 10 } 11 12 i = 4 ; 13 // Busy−w a i t 14 w h i l e ( i −−) { 15 t i m e s t a m p c u r r e n t c y c l e ( ) ; 16 b u s y w a i t ( 1SECOND ) ; 17 t i m e s t a m p c u r r e n t c y c l e ( ) ; 18 } 19 20 i = 4 ; 21 // Loop 22 w h i l e ( i −−) { 23 t i m e s t a m p c u r r e n t c y c l e ( ) ; 24 f o r ( i n t j = 0 ; j < 1 0 0 0 0 0 ; j ++) { 25 v o l a t i l e i n t a = 2 ; 26 a = a ∗ a ; 27 } 28 t i m e s t a m p c u r r e n t c y c l e ( ) ; 29 } 30 }

(21)

7. Results

From our results, we can see that depending on the configuration, especially with regards to clock frequency and tick-rate, the results changed when running ACRN. In Table 4, we can see that the sleep function took between 310 to 465 times more cycles than the expected number of cycles, while the busy-wait took barely any extra cycles at all. Finally, the loop took 27 times more cycles than the baseline. These results were taken when the system was running with a clock frequency of 19.2 MHz and a tick rate of 10000 ticks/s.

Configuration Sleep Busy-Wait Loop

ACRN Debian 393,4345029 1,070640299 27,33158789 ACRN Debian Strip 402,8886401 1,111302643 27,32981885 ACRN Clearlinux 310,4818457 1,107642656 27,34719077 ACRN Clearlinux Strip 307,2372667 1,000026185 27,08388763 ACRN Ubuntu 447,1061502 1,076953177 27,21614067 ACRN Ubuntu Strip 465,5921071 1,101838633 27,33703651

Table 4: Difference from the baseline when running the test program on ACRN with a clock frequency of 19.2 MHz and a tick rate of 10000 ticks/s.

Interestingly enough, when the clock frequency was changed to 25 MHz and the tick rate to 100 ticks/s, the results were dramatically different from previous results, which can be seen in Table 5. When ran under this setup, we saw that the sleep function took around 2.25 to 3.98 times more cycles, the busy-wait did not change much (1.18 as maximum compared to 1.11 as the maximum for 19.2 MHz), and the loop took around 41 times more cycles than the baseline.

ACRN Debian 3,92751806 1,08976253 40,62336083 ACRN Debian Strip 3,65007764 1,0000202 41,04018613 ACRN Clearlinux 2,28071853 1,19650823 41,05499154 ACRN Clearlinux Strip 2,24608165 1,08700284 40,80430415 ACRN Ubuntu 2,66058113 1,18279499 40,67364636 ACRN Ubuntu Strip 3,98172415 1,0462964 40,62801396

Table 5: Difference from the baseline when running the test program on ACRN with a clock frequency of 25 MHz and a tick rate of 100 ticks/s.

When running Zephyr directly on the board using a clock frequency of 19.2 MHz and a tick-rate of 10000 ticks/s, the cycle count for each test function did not differ that much from the expected values, with the exception of the sleep function. When running Zephyr in non-tickless mode, the sleep function slept for about 40% more cycles than expected, but this did not happen when using the tickless mode. This can be seen in Table 6. But when the clock frequency and tick-rate were changed to 25 MHz and 100 ticks/s, the results were a little different. The sleep function actually woke up around 30% cycles earlier than expected, which can be seen in Table 7. The cycles taken for the busy-wait and loop did not differ much between different configurations and not even between the different clock frequencies and tick-rates.

Zephyr non-tickless 1,405909128 1,000016745 1,061562412 Zephyr non-tickless Strip 1,405874466 1,000018867 1,060005661 Zephyr tickless 1,000152891 1,000017135 0,999823097 Zephyr tickless Strip 1,00014431 1,000017526 1

Table 6: Difference from the baseline when running the test program on Zephyr directly on the board with a clock frequency of 19.2 MHz and a tick rate of 10000 ticks/s.

(22)

Configuration Sleep Busy-Wait Loop Zephyr non-tickless 0,69324044 1,00000043 1,000105753 Zephyr non-tickless Strip 0,6932399 1,00000043 1,000052876

Zephyr tickless 0,69326735 1,00000042 1 Zephyr tickless Strip 0,69326683 1,00000044 1

Table 7: Difference from the baseline when running the test program on Zephyr directly on the board with a clock frequency of 25 MHz and a tick rate of 100 ticks/s.

It is easier to see the differences in cycles taken when looking at a bar chart. We have separated each function and system configuration into its own charts, which can be seen below. Since the cycle difference was so large in the Sleep Difference 19.2 MHz chart, we had to use a logarithmic scale for the X-axis. This is not the case for the other charts.

1 10 100 1000 ACRN Debian

ACRN Debian Strip ACRN Clearlinux ACRN Clearlinux Strip ACRN Ubuntu ACRN Ubuntu Strip Zephyr Zephyr Strip Zephyr Tickless Zephyr Tickless Strip

Sleep Difference (19200000)

Figure 4: Trace results of Sleep compared to the baseline in a logarithmic scale (x-axis).

0 0,5 1 1,5 2 2,5 3 3,5 4 4,5 ACRN Debian

Sleep Difference (25000000)

(23)

0,94 0,96 0,98 1 1,02 1,04 1,06 1,08 1,1 1,12 ACRN Debian

Busy Wait Difference (19200000)

Figure 6: Trace results of Busy-wait compared to the baseline.

0,9 0,95 1 1,05 1,1 1,15 1,2 1,25 ACRN Debian

Busy Wait Difference (25000000)

(24)

0 5 10 15 20 25 30 ACRN Debian

Loop Difference (19200000)

Figure 8: Trace results of Loop compared to the baseline.

0 5 10 15 20 25 30 35 40 45 ACRN Debian

Loop Difference (25000000)

(25)

8. Discussion

Many of the problems encountered during the installation of these systems stem from the docu-mentation being somewhat unclear at times. This lead to it being hard to figure out what could have gone wrong when the system did not function correctly. Also, the biggest hurdle was finding out how to consistently be able to install or re-install the hypervisor while also managing to get it to boot correctly every time. Most of the problems related to the documentation are easily fixed by just adding the missing information to the existing pages. This would not only make it easier to use the systems but also increase the quality of the documentation.

The differences in cycles taken between running Zephyr on the board directly and when run via ACRN are unexpectedly large. This suggests to us that there is a problem either with running Zephyr via ACRN or them running on our board. We did not have access to the other supported boards for ACRN to make similar tests. Another problem for us was that Zephyr standalone was not supported on the NUC-line of boards that ACRN supported. Because of this, we could not perform any comparisons between the boards even if we had them.

When measuring the sleep duration on one ACRN configuration, the cycle count was very large compared to another configuration (the highest being ≈465 times more cycles than expected). For the busy-wait function, the difference to the baseline was not that large, with ≈1.19 times more cycles than expected being the highest. The basic loop behaved differently depending on which clock frequency and tick-rate we used. when running under ACRN with a clock frequency of 19.2 MHz and a tick-rate of 10000 ticks/s, the loop took 27 times more clock cycles than expected on all configurations. But when the clock frequency was changed to 25 MHz and the tick-rate to 100 ticks/s, the loop took around 41 times more clock cycles than expected on all configurations. This can lead to problems since the system does not seem to behave as one would expect. The different functions do not seem to exhibit the same behavior when it comes to the increases and decreases in cycles taken to perform them. That is, the cycles taken for the sleep, busy-wait, and loop do not increase for all of them on one configuration. Rather sometimes one performs better on one configuration and worse on another independently from each other.

We know that software-based tracing methods affect the execution on the hardware by adding tracing functions in the source code. However, we do not know how much CTF affects our test program when it is run on the systems. That is why we think it would be good to investigate how other tracing methods affect the results.

(26)

9. Conclusions

We have answered our research question about the performance of common scheduling API methods when run on Zephyr and ACRN by performing a series of tests. According to our results, the current versions of Zephyr and ACRN are not ready to be used for real-time systems, at least not on the UP Squared board. This is because depending on the configuration, the time taken to perform each function varied greatly. When ACRN ran Zephyr using a 19.2 MHz clock frequency and a tick-rate of 10000 ticks/s, the highest results for the sleep function was 465 times more cycles than the baseline, while the lowest result was 307 times more cycles, which is a substantial disparity. While the busy-wait ran for almost the specified time, producing results that only varied a little bit, with the lowest being 1.000026 times more cycles and the highest being 1.11 times more cycles. The loop took around 27 times more cycles than expected, which is a very unexpected outcome here. However, when the clock frequency was changed to 25 MHz and a tick rate of 100 ticks/s, the results became closer to the expected amount of time, but still off by a large margin. The sleep function on this configuration took around two to four times more cycles than the baseline, and the busy-wait produced similar when running these settings (1.000020 - 1.19 times more cycles than the baseline). The time it took for the loop did not change much between the different configurations, with 40.623 times more cycles being the lowest and 41.05 times more cycles being the highest. From these results, we can clearly see that the system does not behave in a deterministic manner, which is one of the more important attributes for a real-time system.

When we ran Zephyr directly on the board, the results for the busy-wait and the loop did not change much between the different configurations, and clock frequency, and tick-rate. The sleep function, however, changed behavior completely between the different setups. When running Zephyr non-tickless with a clock of 19.2 MHz and 10000 ticks/s, the sleep function slept for too long, around 40%, but this did not happen in tickless mode. When the clock frequency was changed to 25 MHz and tick-rate to 100 ticks/s, the sleep function slept around 30% less than expected. This is also something that is considered non-deterministic behavior.

We would argue that based on the problems we encountered during the installation of each system that the documentation for both systems needs more work in order to make them clearer and understandable. For both systems, the documentation is still usable but can be hard to follow or use. However, some sections are missing some crucial information that needs to be added.

(27)

References

[1] M. Aberdour, “Achieving quality in open-source software,” IEEE Software, vol. 24, no. 1, pp. 58–64, 2007.

[2] J. Tassone, S. Xu, C. Wang, J. Chen, and W. Du, “Quality assessment of open source software: A review,” in 2018 IEEE/ACIS 17th International Conference on Computer and Information Science (ICIS), 2018, pp. 411–416.

[3] J. Franch Guti´errez, A. Susi, M. C. Annosi, C. P. Ayala Mart´ınez, R. Glott, D. Gross, R. Kenett, F. Mancinelli, P. Ramsany, C. Thomas et al., “Managing risk in open source software adoption,” in ICSOFT 2013: Proceedings of the 8th International Joint Conference on Software Technologies, 2013, pp. 258–264.

[4] O. Osanaiye, S. Chen, Z. Yan, R. Lu, K. R. Choo, and M. Dlodlo, “From cloud to fog computing: A review and a conceptual live vm migration framework,” IEEE Access, vol. 5, pp. 8284–8300, 2017.

[5] H. Li, X. Xu, J. Ren, and Y. Dong, “Acrn: A big little hypervisor for iot development,” in Proceedings of the 15th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments, ser. VEE 2019. New York, NY, USA: Association for Computing Machinery, 2019, p. 31–44. [Online]. Available:

https://doi-org.ep.bib.mdh.se/10.1145/3313808.3313816

[6] Intel testimonial:R Introducing acrnTM: A big little hypervisor for iot development.

[Online]. Available: https://projectacrn.org/intel-testimonial/

[7] Zephyr project: Introduction. [Online]. Available: https://docs.zephyrproject.org/latest/ introduction/index.html

[8] Time-critical edge compute. [Online]. Available: https://wiki.akraino.org/display/AK/ Time-Critical+Edge+Compute

[9] J. Liang, F. Liu, S. Li, and Z. Cai, “A comparative research on open source edge computing systems,” in Artificial Intelligence and Security, X. Sun, Z. Pan, and E. Bertino, Eds. Cham: Springer International Publishing, 2019, pp. 157–170.

[10] K. Bilal and A. Erbad, “Edge computing for interactive media and video streaming,” in 2017 Second International Conference on Fog and Mobile Edge Computing (FMEC), 2017, pp. 68– 73.

[11] N. Alam, “Survey on hypervisors,” Indiana University, Bloomington, School of Informatics and Computing, 2009.

[12] S. Shirinbab, L. Lundberg, and D. Ilie, “Performance comparison of kvm, vmware and xenserver using a large telecommunication application,” in Cloud Computing. IARIA XPS Press, 2014.

[13] V. Timcenko, B. Djordjevic, S. B. Rakas, and N. Davidovic, “Performance examination of type-2 hypervisors: Case of particular database application in a virtual environment,” in Proceedings of the International Conference on Information Systems and Design of Communication, ser. ISDOC ’14. New York, NY, USA: Association for Computing Machinery, 2014, p. 122–126. [Online]. Available: https://doi-org.ep.bib.mdh.se/10.1145/ 2618168.2618187

[14] Wikipedia: Hypervisor. [Online]. Available: https://en.wikipedia.org/wiki/Hypervisor

[15] Introduction to project acrn. [Online]. Available: https://projectacrn.github.io/latest/ introduction/index.html

(28)

[16] N. S. Awang Abu Bakar and N. Arsat, “Investigating the factors that influence the quality of open source systems,” in The 5th International Conference on Information and Communica-tion Technology for The Muslim World (ICT4M), 2014, pp. 1–6.

[17] G. Avelino, E. Constantinou, M. T. Valente, and A. Serebrenik, “On the abandonment and survival of open source projects: An empirical investigation,” in 2019 ACM/IEEE Interna-tional Symposium on Empirical Software Engineering and Measurement (ESEM), 2019, pp. 1–12.

[18] W. J. Sung, J. H. Kim, and S. Y. Rhew, “A quality model for open source software selection,” in Sixth International Conference on Advanced Language Processing and Web Information Technology (ALPIT 2007), 2007, pp. 515–519.

[19] G. Singh, “Metrics for measuring the quality of object-oriented software,” SIGSOFT Softw. Eng. Notes, vol. 38, no. 5, p. 1–5, Aug. 2013. [Online]. Available: https: //doi-org.ep.bib.mdh.se/10.1145/2507288.2507311

[20] C. Koehler, A. Mayer, and M. Wurm, “Combined hardware and software tracing of real and virtual embedded system parts,” in Proceedings of the 19th International Conference Mixed Design of Integrated Circuits and Systems - MIXDES 2012, 2012, pp. 340–345.

[21] The - common trace format. [Online]. Available: https://diamon.org/ctf/#specification

[22] Systemview. [Online]. Available: https://www.segger.com/products/development-tools/ systemview/

[23] About dtrace. [Online]. Available: http://dtrace.org/blogs/about/

[24] Babeltrace - a trace manipulation toolkit. [Online]. Available: https://babeltrace.org/ #bt2-get

[25] J. Hwang, S. Zeng, F. y. Wu, and T. Wood, “A component-based performance comparison of four hypervisors,” in 2013 IFIP/IEEE International Symposium on Integrated Network Management (IM 2013), May 2013, pp. 269–276.

[26] S. Pawar and S. Singh, “Performance comparison of vmware and xen hypervisor on guest os,” IJICSE, vol. 2, pp. 56–60, 2015.

[27] J. Ruh and W. Steiner, “The need for deterministic virtualization in the industrial internet of things,” in Proceedings of the Workshop on Fog Computing and the IoT, ser. IoT-Fog ’19. New York, NY, USA: Association for Computing Machinery, 2019, p. 26–30. [Online]. Available: https://doi.org/10.1145/3313150.3313222

[28] N. D. Linh, P. D. Hung, V. T. Diep, and T. D. Tung, “Risk management in projects based on open-source software,” in Proceedings of the 2019 8th International Conference on Software and Computer Applications, ser. ICSCA ’19. New York, NY, USA: Association for Computing Machinery, 2019, p. 178–183. [Online]. Available:

(29)

A

Appendix A: Raw cycle count data

Here is the raw data containing all the cycle counts measure before and after the function as well as the difference between them. Each section contains all the measurements taken for each configuration. Because of the size of these tables we could not fit them on this page. You can find them on the next pages.

(30)

1.1. A

CRN

Debian

Before sleep After sleep Difference Before Lo op After Lo op Difference Before Lo op After Lo op Difference 2331837746 9956203327 7624365581 32547609462 32566809949 19200487 32629845052 32630042612 197560 9956203765 17633990298 7677786533 32566812503 32586066888 19254385 32630042858 32630233364 190506 17633990769 25173830035 7539839266 32586069522 32607954920 21885398 32630233531 32630424102 190571 25173830529 32547608971 7373778442 32607957508 32629842413 21884905 32630424257 32630618120 193863 T able 8: Configuration: 1000 tic ks, 19200000Hz Before slee p After sleep Difference Before Lo op After Lo op Difference Before Lo op After Lo op Difference 2723257699 2843503700 120246001 3116011401 3141011896 25000495 3224998210 3225188869 190659 2843504132 2940926060 97421928 3141014485 3174989260 33974775 3225189064 3225382166 193102 2940926549 3024058765 83132216 3174991937 3199992416 25000479 3225382392 3225576416 194024 3024059247 3116010908 91951661 3199994999 3224995503 25000504 3225576581 3225767065 190484 T able 9: Configuration: 100 tic ks, 25000000Hz

1.2. A

CRN

Debian

strip

Before sleep After sleep Difference Before Lo op After Lo op Difference Before Lo op After Lo op Difference 2787381016 10393517079 7606136063 33729230327 33748922642 19692315 33814588521 33814786059 197538 10393517549 18146230790 7752713241 33748925191 33770810604 21885413 33814786311 33814976818 190507 18146231088 26157270444 8011039356 33770813142 33792697796 21884654 33814976984 33815167546 190562 26157270947 33729229849 7571958902 33792700334 33814585995 21885661 33815167702 33815361545 193843 T able 10: Configuration: 1000 tic ks, 19200000Hz

(31)

1.3. A

CRN

Clearlin

ux

Before sleep After sleep Difference Before Lo op After Lo op Difference Before Lo op After Lo op Difference 4052643619 10067686159 6015042540 27897651281 27917242022 19590741 27982728720 27982922850 194130 10067686648 16017405535 5949718887 27917244650 27939125798 21881148 27982923118 27983120959 197841 16017405991 21975320019 5957914028 27939128384 27960815847 21687463 27983121132 27983311618 190486 21975320497 27897650789 5922330292 27960818427 27982726031 21907604 27983311852 27983502336 190484 T able 12: Configuration: 1000 tic ks, 19200000Hz Before sleep After sleep Difference Before Lo op After Lo op Difference Before Lo op After Lo op Difference 2358617710 2420890477 62272767 2586691212 2611691699 25000487 2706352232 2706550260 198028 2420890959 2476175571 55284612 2611694205 2647492426 35798221 2706550511 2706744563 194052 2476176058 2531448463 55272405 2647495038 2672495590 25000552 2706744822 2706935309 190487 2531448882 2586690951 55242069 2672498010 2706349573 33851563 2706935472 2707129337 193865 T able 13: Configuration: 100 tic ks, 25000000Hz

(32)

1.4. A

CRN

Clearlin

ux

strip

1.5. A

CRN

Ubun

tu

(33)

1.6. A

CRN

Ubun

tu

strip

Before sleep After sleep Difference Before Lo op After Lo op Difference Before Lo op After Lo op Difference 699063043 9519869695 8820806652 36456538850 36475739278 19200428 36541170036 36541367689 197653 9519870198 18795659017 9275788819 36475741713 36497392692 21650979 36541367936 36541558525 190589 18795659522 27527326353 8731666831 36497395253 36519280279 21885026 36541558682 36541749184 190502 27527326842 36456538368 8929211526 36519282795 36541167569 21884774 36541749335 36541943245 193910 T able 18: Configuration: 1000 tic ks, 19200000Hz Before slee p After sleep Difference Before Lo op After Lo op Difference Before Lo op After Lo op Difference 2340489948 2447473160 106983212 2738664406 2763665038 25000632 2843304546 2843497813 193267 2447473624 2555272179 107798555 2763667636 2788668139 25000503 2843498055 2843692183 194128 2555272724 2645560543 90287819 2788670706 2818298701 29627995 2843692384 2843882866 190482 2645561083 2738663912 93102829 2818301374 2843301884 25000510 2843883025 2844073505 190480 T able 19: Configuration: 100 tic ks, 25000000Hz

(34)

1.7. Zeph

yr

1.8. Zeph

yr

strip

(35)

1.9. Zeph

yr

tic

kless

(36)

1.10. Zeph

yr

tic

kless

strip

(37)

B

Appendix B: Implementation source code

This is the full version of the implemented function that we use to test sleep, busy-wait and our simple loop. The program consists of a single thread running three loops. Each loop will perform a a test and a timestamp is collected before and after the function. The ctf-functions are custom CTF messages that contains the current system clock cycle. This makes it easier to see which timestamps that belong to which functions, as well as if the timestamp is before or after it. These CTF messages are captured over UART and stored in a binary trace file that can be translated into a human-readable format by Babeltrace.

1 v o i d f u n c ( v o i d ∗ p1 , v o i d ∗ p2 , v o i d ∗ p3 ) 2 { 3 i n t i = 4 ; 4 5 // S l e e p 6 w h i l e ( i −−) { 7 c t f b e f o r e s l e e p ( k c y c l e g e t 3 2 ( ) ) ; 8 k s l e e p (K MSEC ( 1 0 0 0 ) ) ; 9 c t f a f t e r s l e e p ( k c y c l e g e t 3 2 ( ) ) ; 10 } 11 12 i = 4 ; 13 // Busy−w a i t ( 1 s e c o n d ) 14 w h i l e ( i −−) { 15 c t f b e f o r e b u s y w a i t ( k c y c l e g e t 3 2 ( ) ) ; 16 k b u s y w a i t ( 1 0 0 0 0 0 0 ) ; 17 c t f a f t e r b u s y w a i t ( k c y c l e g e t 3 2 ( ) ) ; 18 } 19 20 i = 4 ; 21 // Loop 22 w h i l e ( i −−) { 23 c t f b e f o r e l o o p ( k c y c l e g e t 3 2 ( ) ) ; 24 25 f o r ( i n t j = 0 ; j < 1 0 0 0 0 0 ; j ++) 26 { 27 v o l a t i l e i n t a = 2 ; 28 a = a ∗ a ; 29 } 30 c t f a f t e r l o o p ( k c y c l e g e t 3 2 ( ) ) ; 31 } 32 }

Listing 2: The implementation of the test program.

The k sleep function takes K MSEC as a parameter, K MSEC converts the number of milliseconds to the correct format in the k timeout t data structure, which is required for the k sleep function.

In this case, the k sleep function will sleep for one second. The k busy wait function takes mi-croseconds as input and will also run for one second. However, there are certain instances where the k busy wait function will run for more time than specified by the input. We did not measure the time taken for one iteration of the loop in order to set the number of iterations to a number that would also take one second.

Evaluating the real-time capabilities of Zephyr on UP Squared - a pilot study with Zephyr and ACRN

V¨

aster˚

as, Sweden

Thesis for the Degree of Master of Science in Computer Science with

specialisation in Intelligent Embedded Systems 30.0 credits

EVALUATING THE REAL-TIME

CAPABILITIES OF ZEPHYR ON UP

SQUARED - A PILOT STUDY WITH

ZEPHYR AND ACRN

Charlie Kotro

cko14001@student.mdh.se

Jimmy M¨

akkeli

jmi14002@student.mdh.se

Examiner: Thomas Nolte

M¨

alardalen University, V¨

aster˚

as, Sweden

Supervisors: Anna Friebe

M¨

alardalen University, V¨

aster˚

as, Sweden

Shaik Mohammed

M¨

alardalen University, V¨

aster˚

as, Sweden

September 25, 2020

Table of Contents

List of Figures

Acknowledgements

1.

Introduction

1.1.

Problem formulation

2.

Background

2.1.

Hypervisors

2.2.

ACRN

2.3.

Zephyr

2.4.

Open-source

2.5.

Software tracing

2.6.

Babeltrace 2

3.

Related work

3.1.

Hypervisors

3.2.

Quality assessment for open-source projects

3.3.

Quality models for open-source projects

3.4.

Risk management for open-source projects

3.5.

Summary

4.

Method

5.

Ethical considerations

6.

Implementation and problems

6.1.

Zephyr problems

6.2.

ACRN problems

6.3.

Test program implementation

7.

Results

Sleep Difference (19200000)

Sleep Difference (25000000)