Degree Project at Bachelor Level Evaluation of WireGuard and OpenVPN VPN solutions

(1)

Author: Ahmad Anbarje Author: Mohammed Sabbagh Supervisor: Diego Perez Palacin

Degree Project at Bachelor Level

Evaluation of WireGuard and

OpenVPN VPN solutions

(2)

Abstract

This project evaluates two types of VPN solutions–WireGuard and OpenVPN.

The evaluation is done by four different experiments to measure the maximum throughput of each of the VPN solutions, and a theoretical study on the encryption techniques that each VPN solution use. Experiments were conducted using a network performance measurement tool called IPerf. By applying static parameters that control the measurements, such as transferring regular and zip file types, measuring throughput when the compression is enabled on the VPN channel and when it is disabled, also, applying measurements with the conditions of the same parameters but with different operating systems when using both VPN solutions.

one of the measurement results shows that the throughput is about 93 Mbps for WireGuard comparing it with 56 Mbps and 59Mbps for OpenVPN when sending regular and compressed files respectively.

In conclusion, throughput can vary slightly depending on the measurement tool and the version of the software used. Also, the buffer of the measurement tool has an important role when measuring the throughput, it gives different throughput and packet loss results which depends on the size of that buffer. Moreover, According to a theoretical study, throughput can also vary depending on the speed of the encryption algorithm used. Both VPN solutions use many encryption algorithms, such as AES in OpenVPN, while WireGuard uses ChaCha20 with Poly1305.

(3)

Preface

We would like to thank our supervisor Diego Perez Palacin and our reader Ola Flygt who guided us step by step to finish this thesis project. Without thier support and valuable knowledge in this field, we would not be able to complete this study.

(4)

1 Introduction

With the advancement of technology and a significant increase in working remotely, many organizations and commercial companies need to transfer data between locations. Still, they prefer not to share their information over the Internet. The exchange of information via the Internet can be a risk because sensitive information can be exposed. That is where the benefit of a VPN comes in. VPN is an abbreviation for “Virtual Private Network.”

VPN secures communication, so professionals can work at any time and any place without having to be in the office. WireGuard and OpenVPN are software that implement (VPN) technologies to create a secure connection when exchanging data or while surfing the Internet. WireGuard is a very recent application, while OpenVPN is a traditional one.

WireGuard has been designed to overcome some of the OpenVPN weaknesses, such as its complexity. OpenVPN has a massive code of 400,000-600,000 lines, which makes it hard to find errors, while WireGuard has only 4000 lines [7, 8]. This project analyzes and compares these two VPN technologies in terms of their performance.

1.1 Definition of a VPN

Before investigating VPN technology, network concepts will be presented. Each network adapter in the computer has a physical address (its value stored on a chip inside it), known as a MAC address, used to identify the equipment on the local network. It has another logical IP address, used to determine the network where the computer located. The user needs an IP address to connect to any computer. In the case of a local network, there is a device called Switch that will handle the connection between multiple devices. The Switch has multiple ports, and each port is connected to a device on its network adapter using a network cable. The Switch, in this case, will handle the request because it knows the MAC address of this IP; otherwise, it will send it to the router to direct the packets to the correct path to reach the other network that this address refers. Some IP addresses are public addresses that any computer connected to the Internet can access, while others are private addresses that can be contacted through the same physical network [1]. VPN is a fictitious network that the user creates to send and receive data between his device and the Internet or another device that guarantees its safe and encrypted passage and is not viewed by anyone else. For example, if someone enters a website on the network and downloads specific files or even verifies an email, the VPN encrypts the connections and sends them to a server that is powered by the VPN provider itself, which in turn, sends it to the destination server [6].

(7)

1.1.1 Types of VPN connection

• Site to site: This is about connecting two networks in two locations, which are a two-way or (multi-to-multi) connections. It allows any device from the first branch to communicate with the other, and vice versa. It is set up at the level of network devices to achieve flexibility as it allows any number of devices in the network to communicate directly with the other network without additional steps on the systems Personal. The example in figure 1.1 demonstrates that.

Figure 1.1: Site-to-Site VPN

• Remote access: It connects individual clients to VPN networks. This should be set up on each standalone clients’ device. Furthermore, this one is the most appropriate to the conditions of continuous mobility, so there is no need to set up a site connection for network devices everywhere to use this type of VPN. An example of a remote access VPN shown in Figure 1.2 below.

Figure 1.2: Remote Access VPN

(8)

1.1.2 Results from Using a VPN

VPN uses are related to several things; most notably the nature of the activity on the Internet, people’s work, and employees’ place of residence. Some people are interested in the protection of their privacy. However, in some countries, it is to access blocked sites and applications. In general, however, VPN uses can be summarized as follows:

• Secure privacy: VPN technology allows us to connect computers to different networks securely. VPN Clients use the public IP address of the VPN server to browse the Internet. Therefore, VPN services are used to disguise users on the Internet (even though its primary goal was to connect VPN clients’ computers to the private network to access its resources securely). Moreover, this will protect VPN Clients’

privacy. However, despite changing the Internet address, the VPN server will still be aware of the real IP address. In the event of legal violations committed on the Internet, the courts will require the VPN service provider to disclose its records. No service provider does not keep this data. This act may expose him to interrogation.

Figure 1.3 presents how the VPN Client access the Internet throws a VPN server.

Figure 1.3: Presents how the VPN connection is encrypted and secures communication.

• Secure Data:The use of encryption in VPN communication leads to the mainte- nance of the privacy of the network. It is also applied in public networks, when using services over the non encrypted protocol, such as HTTP, FTP, and telnet instead of https, SFTP, and SSH. However, the data remains encrypted until it reaches the VPN server on the remote private network [2].

• The secure and flexible work environment: The most important tasks for companies is to protect files and data from outside penetration and to safeguard businesses from piracy attacks that may target their bank accounts and other private information. Employees may also need to access the private network of the company they work for to access specific files or applications without needing to be in the office [28].

• Accessibility: In some countries, certain sites and web applications are prohibited, and the person will not be able to access it except by using a VPN. Accessing international programs, such as American Netflix from any country in the world, is a feature of multiple VPNs [4].

(9)

1.2 Problem

Given the need for VPNs, many solutions work differently. One of these VPNs might provide a higher performance, while another gives lower performance [5]. The main problem is to investigate which VPN’s solutions WireGuard or OpenVPN provides higher throughput when transmitting data with a group of static parameters that control the tests. The investigation will not end here, and further studies will examine the encryption algorithms used by both WireGuard and OpenVPN.

1.3 Objective

The main goal is to benchmark the throughput between OpenVPN and WireGuard transmitting data. Highlighting some important objectives:

Objective ID Description

O1 Implement the WireGuard VPN solution platform on the Ubuntu server.

O2 Implement the OpenVPN solution platform on the Ubuntu server.

O3 Select a set of suitable static parameters that will control the tests.

O4 Run experiments in both environments and compare the result from both VPNs

O5 Analyze why we got these results.

Table 1.1: Project objectives.

1.4 Motivation

Since WireGuard is a new software, there are no studies yet that measure its performance compared to other VPN solutions. Given the increasing use of VPNs, there is always a need to improve their quality in multiple areas, such as speed, security, reliability, costs, and flexibility, to find vulnerabilities or bugs [9]. This thesis will provide information and analysis of the situations and configurations where one of the VPN solutions outperforms the other. As a result, it can help companies choose their best VPN solution given their conditions.

1.5 Limitations

The project deals with experimental measurements of the VPN throughput. The measures are limited to be over VMware machines. Also, all measurements are limited to execute between an Ubuntu platform acting as a server and two clients, a Windows platform, and an Ubuntu platform. Linux was chosen as a server as it is an open-source operating system and free. As for the Client, Ubuntu was chosen for the same reasons, as the server and Windows as a client because it is a universal operating system. The original plan was to make the measurements physically by using an Ethernet cable from the client machine to the VPN server. While we were working on the original project, we faced a problem with the connection (it become interrupted). The reason for the problem may be due to the novelty of the WireGuard software. We found that our connectivity issues were like those found by Mullvad, a commercial virtual private network service. About 100 to 200 of their WireGuard servers were affected on 27 Apr. [10]. That made us deviate

(10)

from the original plan. We chose to do the tests on virtual machines instead, which made the connection stable and gave us a precise result. The experiments were made in a lab environment. Therefore, in real life, the results can differ for many reasons that are hard to control. The reason behind choosing both OpenVPN and WireGuard is that OpenVPN is an open-source secure VPN protocol and widely used, and WireGuard is a new VPN protocol that tries to replace all the old ones.

1.6 Outline

The rest of the document organized as follows:

• Chapter 2 explains the encryption algorithms which both VPNs use with highlighting the differences between symmetric encryption types.

• Chapter 3 describes how we did the configuration for both VPNs on the Ubuntu server.

• Chapter 4 explains the methods that we applied to the measurements, the tools that we used during all these measurements, how we handled the result, and the environment of the experiment.

• Chapter 5 is related to chapter 4. It presents the result of each measurement ex- plained in chapter 4, showing the results through bar graphs and tables.

• Chapter 6 analyzes the results obtained in chapter 5.

• Chapter 7 concludes the report. There, we illustrate recommendations, our reflec- tions, and the validity of the experiments.

(11)

2 Background

This section presents the existing VPN technologies that we experimented with, the measurement tool used in the experiment, and the encryption techniques used in both of the VPN protocols.

2.1 VPN Protocols

There are many VPN protocols for making a VPN connection like PPTP, IP-sec, SSL VPN, OpenVPN, and WireGuard. They have the same usage, but they work differently [3].

2.1.1 OpenVPN

An open-source VPN protocol was developed by James Yonan and got initially released back in 2002. OpenVPN can be configured to use either TCP or UDP and supports up to 256-bit encryption. OpenVPN relies on SSL-TLS for authentication and encryption. OpenVPN supports all the major operating systems, such as Windows, mac-OS, and Linux, as well as mobile platforms like android and IOS. [27].

2.1.2 WireGuard

WireGuard can be defined as a new open-source secure tunneling VPN protocol. It was developed by Jason Donenfeld. It operates on Layer 3 secure network tunnel for IPv4 and IPv6 and uses new cryptography protocols. Furthermore, WireGuard is based on the UDP protocol, and its authentication model is based on SSH’s authenticated keys. WireGuard has less than 4000 lines of code, which makes it much easier to audit. Plus, checking for vulnerability in the platform is much less expensive. WireGuard was initially designed for the Linux Kernel but is now implemented for Android, mac-OS, BSD, IOS, and Windows [7].

2.2 Network measurement tool IPerf

IPerf is a popular tool to measure network performance [14]. It is also a cross-platform, command-line software. It has many features: like in UDP we can specify the bandwidth, which we need to send to the client. Also, we can measure the delay Jitter, packet loss, and much more.

2.3 Encryption

A critical part of the VPN is encryption. The encryption process affects the VPN performance in many aspects: the speed of the encrypting and decryption process, the strength of the encryption algorithm, and the amount of complexity to implement the encryption method into the VPN Application. Different VPN applications use different encryption methods, which makes them both unique and secure.

What is Encryption? It is the science and practice of concealing data by and transferring it from its understood form to an incomprehensible form, so that it becomes impos- sible for unauthorized people to decipher the content. The goal of encryption is to ensure that the privacy of data is preserved and that no one is allowed to tamper with it or view it

(12)

except those who have the authority to do so using a secret key [19]. Every VPN protocol has its unique encryption methods.

OpenVPN application is one of the most popular VPN applications. It has a custom security protocol and uses SSL/TLS for key exchanges. Further, its availability, integrity, and confidentiality are tried and tested [20].

OpenVPNs can use multiple encryption algorithms. The common algorithm is AES (Advanced Encryption Standard), which has proven its security and efficiency with the network during the past. There are three versions of AES ciphers AES-128, AES-192, and AES-256. The difference between those versions is the length of the encryption key and the encryption rounds’ number of steps. A round consists of several processing steps.

These include substitution, transposition, and mixing input plain-text to transform it into the final output of cipher-text. AES-256 has 14 rounds of encryption steps and 256 bits of key length. Longer keys increase the encryption rounds, and more encryption rounds mean higher security than fewer encryption rounds [21].

WireGuard implements ChaCha20 for symmetric encryption and uses different algorithms for key exchanges. Hashing Curve25519 is for Elliptic-curve Diffie–Hellman (ECDH) anonymous key agreement, BLAKE2s for hashing (RFC7693), Sip-Hash24 for hash-table keys, and HKDF for key derivation (RFC5869). ChaCha20 is a 256-bit stream cipher based on the 20-round encryption steps [22] [23].

Both AES and ChaCha20 are symmetric encryption algorithms but different types.

Symmetric encryption is a method where only one key (a secret key) is used for both encryption and decryption. Moreover, there are two types of symmetric encryption techniques, one known as Block Cipher (like AES) and another known as Stream Cipher (like ChaCha20).

A stream cipher is an encryption algorithm that encrypts 1 bit or byte of plain-text at a time, and the typical size of the Block cipher could be 64 to 128 bits. That makes the stream cipher process faster than block cipher, as it has a bigger size and takes more time to convert. Stream cipher uses a different encrypting key for each bit or byte as the application designed, while block cipher uses the same key to encrypt each block. Stream Ciphers has less usage of memory than Block Ciphers because it deals with only small bits at a time [24].

As a result, from the difference between Stream cipher and block cipher, the Wire- Guard has better performance than the OpenVPN as WireGuard uses the stream cipher encryption.

(13)

3 Configuration

This chapter discusses setting up OpenVPN and WireGuard on the Ubuntu server.

3.1 Setting Up an OpenVPN Server on Ubuntu 19.10

OpenVPN is an open-source VPN solution that uses a specific secure protocol by using Secure Socket Layer (SSL). By using this protocol, the connection between the client and the server is encrypted. OpenVPN solution has an extensive configuration. It could route or bridge, which allows creating a secure point-to-point or site-to-site connection. Two computers can make authentication through OpenVPN by using a username/password or pre-shared secret keys or certificates. In this setup, we are going to generate a certificate for each client to let them connect to the OpenVPN server. The client could access the server by using different platforms, for example, Windows, iOS, and Android [13].

3.1.1 Before setting up

- Access the root privileges on the Ubuntu server, which is sudo.

- Configure a password authentication or SSH key pair on the Ubuntu server to not get problems in the future when transferring files.

3.1.2 Installing OpenVPN Server and EasyRSA

Initially using advanced packaging tool apt to update all packages for the server, also installing OpenVPN by using it [13]. Starting with these commands:

SUDO APT UPDATE

SUDO APT INSTALL OPENVPN

Then using EasyRSA to generate trusted certificates for clients to use by creating certificate authority (CA). Downloading EasyRSA tgz file from GitHub and extract it with these commands:

WGET-P HTTPS://GITHUB.COM/OPENVPN/EASY-RSA/RELEASES/

DOWNLOAD/V3.0.4/EASYRSA-3.0.4.TGZ CD

TAR XVFEASYRSA-3.0.4.TGZ

We have done the first step. Next, we are going to configure EasyRSA variables, also creating a directory for CA to create certificates for clients and generating HMAC signature provided in the appendix A.2.

3.2 Set Up WireGuard Server on Ubuntu 19.10

WireGuard is a new type of virtual private network (VPN) technique used to make a secure connection between two machines. It could be two routers by using the data link layer in the open systems interconnection model without relying on any host which connects to the network, the configuration of the connection between these two machines could be routed or bridged. WireGuard is officially now an Object file in the Linux kernel, which means it works as a module. WireGuard uses ChaCha20 as encryption protocol, Poly1305 for authenticating the data, and a lot more [12].

(14)

3.2.1 Installing WireGuard

To install WireGuard, we need first to add the WireGuard repository to the Linux repository list. Then we should update all software and packages on the server. Then we will install WireGuard and remove unnecessary package or dependencies that are not needed anymore by typing these commands:

SUDO ADD-APT-REPOSITORY PPA:WIREGUARD/WIREGUARD SUDO APT-GET UPDATE

SUDO APT INSTALL WIREGUARD SUDO APT AUTOREMOVE

3.2.2 Generating Keys

WireGuard works as the same technique as SSH to let the client connect to the WireGuard server we need to generate public and private keys for both machines. We need to share the public keys between them. After sharing the public keys, the initial handshake will be sent from the client to the server to verify the connection, then the server will send a response handshake to make the connection. We are going to generate public and private keys for the WireGuard server and the client by this command:

SUDO WG GENKEY | TEE /ETC/WIREGUARD/"NAME"_PRIVATE.KEY | WG PUBKEY

> /ETC/WIREGUARD/"NAME"_PUBLIC.KEY

Name here is the name of the machine (server). The next step is to create a WireGuard configuration file, adding peer configuration and the tunnel to the client (provided in the appendix B).

(15)

4 Methods

This section explains our work during the experiments, some essential definitions within this field, and the research method we used. We will provide some screenshots for the experiment, with an explanation of the parameters that we used in each scenario.

4.1 The project plan

To let us understand the idea more broadly, the method that we used to solve the problem was reading different types of articles within this field. In our reading, we focused on:

• How both WireGuard and OpenVPN protocols work.

• How to set up each of these VPN solutions on a Ubuntu server.

• Encryption algorithms used for both VPNs solutions.

• Tools needed to evaluate VPNs solutions.

• Basic terms that have provided us with an understanding of the mechanism of the project and the experiment.

Accordingly, we decided to experiment with measuring the throughput across the channel for each of the virtual private networks. We plan to use IPerf software on both machines to make the measurement. We will talk about this tool in the next section. The plan is to measure the throughput by applying specific parameters such as UDP in the transport layer protocol for both VPNs solutions, as well as measuring when the data transferred is compressed through the tunnels and when it is not. Also, we will be changing some parameters within the application layer by applying different file types.

terms which we need to use in the experiment:

• Throughput: It is the number of packets that reach the destination successfully within a specific time.

• Packet loss: Number of packets that are sent compared with the number of packets that did not reach the destination.

• CPU utilization: rate of work that the CPU handled within the measuring process.

4.2 Ethical considerations

The study is about measuring throughput between two of the most popular VPN solutions.

The ethical aspect is critical here because it is related to security issues. There were no issues related to anonymity, all references come from reliable sources, and there was no bias for any of the software used with the evaluation process. Moreover, we did not seek to cause damage or a violation of privacy to any party of the software used. The results were measured as accurately as possible under the same conditions for both VPN solutions.

(16)

4.3 The validity of the experiment

We were keen to keep the results of the experiment correct and unaffected by any internal or external factors, so we decided to use the same version of IPerf on both machine sides.

We also have monitored some aspects related to CPU, network, disk I/O, and memory utilization. By using "Resource Monitor" software on the Windows machine and a "Sys- tem Activity Report" on Linux while running the measurement for a long time with UDP connection on both VPNs solutions, there are no unique values to mention. The CPU utilization was between 10% and 30%, which depends on the operating system and VPN solution used.

4.4 Research method and literature study

Because we have static parameters in project objective in section 1.3, then we have nu- meric values as a result of measuring throughput. So we decided to do measurement by observing and recording the values which Iperf give us as a result.

we also add literature study to the scientific method, which helps us to understand the process more broadly and analyze the result of the measurement in a better way by reading a different kind of articles and essays.

4.5 Throughput when transferring generated IPerf packets

Measuring network performance is done by running IPerf on the server-and client-side at once. By running IPerf on both machines simultaneously, the client will generate packets to send through the VPN solution tunnel (network interface) to the server-side. (By running IPerf on Ubuntu server-side, as in Figure 4.1).

Figure 4.1: Running IPerf on the Ubuntu server-side.

Now the server is listening for the data which we need to send from the client-side, on port 2020.

-s parameter represents that IPerf is running as a server mode.

-p parameter represents the port, which we need to let the client connect to it. Note that we do not need to specify which protocol connection was used. With IPerf3, we just need to set the UDP on the client-side; then the server-side will automatically know that it is the UDP. On the client-side Windows machine, starting OpenVPN software lets the client

(17)

connect to the Ubuntu server. Then, by running command line prompt with writing this line (as shown in Figure 4.2).

Figure 4.2: Running IPerf on the Windows client-side.

Now, the client-side is sending data to the server-side. It is reporting information related to the bandwidth, transfer rate, and total datagrams per second, calculating the average bandwidth for the process (which remains 10 seconds) and the lost datagrams which we were interested in our measurement.

-c specifying the network interface IP, which OpenVPN is running on.

-u represents the UDP connection.

-p represents the port number, which is IPerf on the server-side listening to it.

-b the network bandwidth, which is Mbits for each second.

4.6 Enable the compression on the OpenVPN tunnel

In this section, we show how to measure the throughput by transferring regular and compressed files, enabling the compression of both tunnels’ VPNs. We are starting with enabling the compression of OpenVPN on the server-side, by navigating to server.conf file, which we already created in the previous section. By adding these two lines, the compression on the tunnel will get enabled:

COMPRESS LZ4-V2

PUSH"COMPRESS LZ4-V2"

Moving to the client-side, by adding this line to the config ovpn file:

COMPRESS LZ4-V2

Now the OpenVPN tunnel is enabled for compression [15].

4.7 Handling the result of the measurement

This section describes how we handled the results of the measurement. In section 4.2, we have experimented with the process on different bandwidth range with OpenVPN and WireGuard. One of the characteristics we focused on in both VPNs is packet loss percent.

The acceptable packet should be less than 1% in UDP connection, to give good streaming

(18)

in audio or video call [16].

We plan to measure the throughput when setting the bandwidth to 100Mbps for both VPNs, which gives a lost packets range of 0% to 1%. The throughput result is calculated by dividing the amount of data that reaches the destination by the time it takes to finish the process [17].

Although it might seem surprising, it is still possible to lose packets with the UDP connection (even on the same host); this could be due to many reasons. For example, the buffer could not handle the number of packets that it sends compared to the number of packets receiving it, depending on the speed of the network card. Alternatively, it might also be the size of the buffer receiver on the server-side. If it gets full when handling the process, packets will be dropped [25]. Another scenario could be that when a UDP connection is established, a socket from the client-side is created to send packets to the server-side. It might be the performance of the CPU cycle is fast compared to the rate when the socket sends the packets, or when the socket is in timeout mode while the CPU cycle sends the packets. This could lead to packet loss [26].

4.8 The environment of the experiment

The experiments have been conducted on a computer which hosts three virtual machines.

These are the physical computer specifications:

• Windows Server 2016 standard x64 based processor.

• CPU: Intel Xeon CPU 3.30GHz with four cores and eight logical processors.

• Memory: 16GB DDR3.

• Ethernet: Intel 82579lm gigabit NIC.

First virtual machine specification (Server-side):

• Ubuntu 19.10.

• CPU: 4 processor cores.

• Memory: 4GB.

• Kernel network driver: e1000, which represents the Intel 82545EM Gigabit Ether- net NIC.

Second virtual machine specification (client-side):

• System: Windows 10 Pro x64 based processor.

• Memory: 8GB.

Third virtual machine specification (client-side):

(19)

• System: Ubuntu 20.04.

• Memory: 8GB.

All the measurements have standard parameters:

• Buffer size for the client-side in UDP to read or write differ 8KB-32KB-50KB- 63KB, which depends on the measurements scenario.

• The protocol used in the transport layer is UDP.

• Datagram size 1470 byte.

• The number of simultaneous connections that connect to the server is 1.

(20)

5 Results of the measurements

In this section, we are going to show bar graphs related to the result of measurements that we have done in the sections UDP measurements, transfer different file types, and transfer files by enabling the compression on both VPNs tunnels.

5.1 Result of throughput between Windows and Linux operating systems

We have done this measurement by running it on the same bandwidth each time for both VPNs solutions, as follows:

• The client-side is Windows 10 Pro, and the server-side is Ubuntu 19.10.

• Each process remains approximately 10 seconds, and then a result reported with the data amount of data transferred successfully.

• The transferred data automatically generated from IPerf.

Starting with OpenVPN measurement details, as shown in table 5.1.

Measurement Number

Bandwidth Packet loss Maximum

CPU utilization

Throughput

1 100 Mbps 0% 34% 54.90 Mbps

2 100 Mbps 0% 36% 56.11 Mbps

3 100 Mbps 0% 35% 56.70 Mbps

Avarage 55.9 Mbps

Table 5.1: Transfer automatically generated packets with OpenVPN.

WireGuard measurement details, as shown in table 5.2.

Measurement Number

CPU utilization

Throughput

1 100 Mbps 0.29% 34% 93.85 Mbps

2 100 Mbps 0.041% 33% 93.83 Mbps

3 100 Mbps 0.37% 35% 93.74 Mbps

Avarage 93.8 Mbps

Table 5.2: Transfer automatically generated packets with WireGuard.

The result of maximum throughput for both VPNs when the loss packets not going over 1% according to Figure 5.1.

(21)

Figure 5.1: Result of the throughput for both VPNs in Mbps.

5.2 Result of throughput between two Linux operating systems

We have done this measurement by running it on the same bandwidth each time for both VPNs solutions, as follows:

• The client-side is Ubuntu 20.04, and the server-side is Ubuntu 19.10.

Starting with OpenVPN measurement details, as shown in table 5.3.

Measurement Number

CPU utilization

Throughput

1 100 Mbps 0.2% 23% 94.6 Mbps

2 100 Mbps 1.4% 22% 94.4 Mbps

3 100 Mbps 1.5% 23% 93.5 Mbps

Avarage 94.1 Mbps

Table 5.3: Transfer automatically generated packets with OpenVPN.

WireGuard measurement details, as shown in table 5.4:

Measurement Number

CPU utilization

Throughput

1 100 Mbps 0% 10% 95.2 Mbps

2 100 Mbps 0% 10% 95.2 Mbps

3 100 Mbps 0% 10% 95.1 Mbps

Avarage 95.1 Mbps

Table 5.4: Transfer automatically generated packets with WireGuard.

(22)

Comparison of result the throughput between WireGuard and OpenVPN when using Linux operating system as in Figure 5.2.

Figure 5.2: Result of the throughput for both VPNs in Mbps.

5.3 Result of throughput when enabling the compression on the tunnels

In this section, we are going to show the result of throughput by transferring a single file, a follows:

• Each process remains 10 seconds, and then a result reported with the data transferred.

• The file type is mp4.

• The file compressed type is zip.

• The operating systems are Windows 10 pro as client-side and Linux Ubuntu 19.10 as server-side.

The entire details for the measurement according to table 5.5.

Application Bandwidth File Compression CPU utaliza- tion

Packet loss

Throughput

OpenVPN 100 Mbps

Regular Enabled 32% 0% 56.9 Mbps

Regular Disabled 32% 0% 56.3 Mbps

Compressed Enabled 32% 0% 56.5 Mbps

Compressed Disabled 31% 0% 59.5 Mbps

WireGuard 100 Mbps Compressed Disabled 34% 0.04% 93.8 Mbps

Regular Disabled 34% 0.16% 93.8 Mbps

Table 5.5: Details for the measurement with enabling and disable compression.

(23)

The result of throughput for both VPNs when the file type is regular and when it compressed, also when the compression on the OpenVPN tunnel is enabled and when it is not, as shown in Figure 5.3.

Figure 5.3: Result of throughput when enabling and disabling the compression on the tunnel.

5.4 Result of packet loss when changing the buffer size

We have done this measurement by running it on the same bandwidth for Wireguard VPN solution, by the following:

• The client-side is Windows 10 Pro, and the server-side is Ubuntu 19.10.

Starting with Wireguard measurement details, as shown in table 5.6.

Measurement number

Bandwidth Packet loss Buffer size Throughput

1 100 Mbps 4.7% 8KB 91.8 Mbps

2 100 Mbps 0.4% 32KB 94.4 Mbps

3 100 Mbps 0% 63KB 94.4 Mbps

Table 5.6: How the buffer size affects the result of packet loss.

(24)

6 Analysis and discssion

This chapter analyses the previous results and how they can affect the throughput of the VPN service according to the change of certain parameters.

6.1 Throughput between Windows and Linux operating systems

According to section 5.1, we made this measurement to evaluate the throughput of both VPNs by using UDP in the transport layer protocol and the data generated by the IPerf software. Moreover, the compression was disabled on both VPN solutions in this measurement, to check how much of the data would be transferred successfully within a specific time. Based on Figure 5.1, WireGuard gives a higher throughput, compared to Open- VPN, where the packet loss was between 0% to 1%.

6.2 Throughput when enabling the compression on OpenVPN tunnel

The reason for doing this measurement is to check if OpenVPN will give a higher throughput than WireGuard when enabling the compression on the tunnel while the compression on the WireGuard tunnel is still disabled.

We experimented with transferring regular and compressed file types such as a zip file, which uses lossless compression of data by using algorithms such as LZSS and Huff- man coding. As shown in Figure 5.3, it makes a difference in the throughput to apply the compression to a file compared to a regular file. One of the variables is the compression mechanism applied over the application layer. In contrast, a low-level compression algorithm could miss duplicating some packets in the network layer. We saw that WireGuard, again, has a higher throughput compared to OpenVPN.

Results show that throughput differs when transferring compressed data versus regular data. When the compression was disabled on the tunnel, OpenVPN gives a higher throughput when transferring a compressed file compared to a regular file. Still, Wire- Guard gives the same results, where the throughput of transferring a regular file is equal to transferring a compressed file. Also, as a result of Figure 5.3, enabling the compression on the OpenVPN tunnel does not make much of a difference regarding the throughput.

Sending compressed files, however, gives a higher throughput regarding OpenVPN, but not for WireGuard.

Regarding the lost packets, WireGuard loses a few more packets when transferring a compressed file than when it transfers a regular file. In comparison, OpenVPN does not lose any packets regarding this measurement scenario. The reason of losing packets to WireGuard is the size buffer of the Iperf tool as it is shown in section 5.4.

In all the scenarios, WireGuard gives a higher throughput than OpenVPN (when the compression on the tunnel is enabled or when transferred files are compressed).

6.3 Throughput between two Linux operating systems

In this section, we have measured the throughput by using a Linux machine as client and server. The data is generated from the Iperf tool. As Figure 5.2 shows, WireGuard gives a higher throughput than OpenVPN. But the difference of throughput between two VPNs close to each other is not as shown in Figure 5.1. This depends on many reasons: buffer size, Iperf versions on Linux are new compared to Windows, and the CPU utilization is not the same as compared to Windows.

(25)

in addition the result of throughput in this scenario is not the maximum, we could get a higher throughput when setting a higher bandwidth, which will lead to a bigger difference between WireGuard and OpenVPN throughput as we can see in section 5.1.

6.4 The lost packets

As it is mentioned earlier, Wireguard operates at layer 3. In other words, it takes the raw IP packet, encrypts it, and finally encapsulates it in a UDP packet to be transmitted to the peer. This means the responsibility for retransmission of lost-packets rests with the layers that are located above layer 3. Wireguard by itself does not retransmit the lost packets. It simply deals with the packets that it received from the above layers [7]. In terms of the connection between the peers, Wireguard will send handshake initiation if no session has been established for 120 seconds, as it mention in Figure 6.1

Figure 6.1: Wireguard timers.

(26)

7 Conclusion

This chapter presents a reflection, recommendation, and experiment validity regarding the measurements.

7.1 Reflection

All the measurements have been done on a physical computer, by running two virtual machines: the client machine with Windows 10 Pro, and the server machine with Ubuntu 19.04 Linux Server. We have used IPerf3 as a tool to make UDP measurements by applying different parameters.

WireGuard gives a higher throughput than OpenVPN, but we also noticed problems with WireGuard connectivity that impeded the project plan and led us to implement Wire- Guard on different Linux kernels. Mullvad also experienced other outages, as mentioned in section 1.5. The novelty of the technology may have caused these problems.

WireGuard gives higher packet loss, comparing it with OpenVPN on a specific bandwidth. The buffer size is one of the reasons that created a different packet loss, which is used in the Iperf tool, as mentioned in section 5.4.

Also, both VPNs use symmetric encryption. However, the Stream cipher mechanism, which is used in WireGuard, is more secure than a block cipher mechanism in OpenVPN.

The reason for that is based on two main factors: the use of keys and the encryption round.

Furthermore, Stream cipher in WireGuard have lower usage of the memory compared to block cipher in OpenVPN, as mentioned in chapter 2.

In this project, we have learned many things: the mechanism of working with both of these VPN solutions, using the IPerf tool, setting up both VPNs step by step on a server, the difference between the encryption algorithms which both VPNs used such as a block cipher and stream cipher, and much more.

7.2 Recommendation

This report benchmarks two of the most prominent VPN solutions used, namely Wire- Guard and OpenVPN. However, there are VPN solutions that are more widely used to do an even more extensive study when evaluating VPN solutions such as IPsec and SSH port forwarding.

All experiments in this study are based on client and server. Windows 10 Pro and Linux 20.04 were used as clients in different scenarios. In contrast, the server was Ubuntu Linux, and we recommend to do experiments on different operating systems such as a Windows server and trying a different kind of Linux distribution. Also, we recommend doing the measurements with the Windows operating system on the server-side. This will give us a broader view of the results while testing more different types of operating systems.

(27)

References

[1] How-To Geek. 2020. How To Find Any Device’S IP Address, MAC Address, And Other Network Connection Details. [online] Available at: <https://www.howtogeek.com/2 36838/how-to-find-any-devices-ip-address-mac-address-and-other-network-connection-d etails/> [Accessed 20 May 2020].

[2] Vpnmentor.com. 2020. VPN Use And Data Privacy Stats For 2020. [online] Available at: <https://www.vpnmentor.com/blog/vpn-use-data-privacy-stats/> [Accessed 20 May 2020].

[3] GeeksforGeeks. 2020. Types Of Virtual Private Network (VPN) And Its Protocols - Geeksforgeeks. [online] Available at: <https://www.geeksforgeeks.org/types-of-virtual- private-network-vpn-and-its-protocols/> [Accessed 20 May 2020].

[4] How-To Geek. 2020. What Is A VPN, And Why Would I Need One? [online] Avail- able at: <https://www.howtogeek.com/133680/htg-explains-what-is-a-vpn/> [Accessed 20 May 2020].

[5] Pena, C. J. C., and J. Evans. “Performance Evaluation of Software Virtual Private Networks (VPN).” Performance Evaluation of Software Virtual Private Networks (VPN)

| Proceedings of the 25th Annual IEEE Conference on Local Computer Networks, 1 Nov.

2000, https://dl.acm.org/doi/10.5555/788015.788519.

[6] A. Walker and G. Paulsen, “Virtual private network system and method,” Google Patents. [Online]. Available: https://patents.google.com/patent/US6055575A/en.

[7] J. Donenfeld, WireGuard: Next Generation Kernel Network Tunnel. 2018 [Online].

Available: https://www.wireguard.com/papers/wireguard.pdf. [Accessed: 13- Mar- 2020]

[8] "The New Cloudflare VPN: What It Is Isn’t | OpenVPN", OpenVPN, 2020. [Online].

Available: https://openvpn.net/what-is-cloudflare-vpn/. [Accessed: 21- Mar- 2020]

[9] Kotuliak, L., Rybár, P., Trúchly, P. (n.d.). Performance comparison of IPsec and TLS based VPN technologies. Retrieved from https://ieeexplore.ieee.org/abstract/document/61 12567.

[10] "Post-mortem - WireGuard server connectivity issues - Blog | Mullvad VPN", Mull- vad VPN, 2020. [Online]. Available: https://mullvad.net/en/blog/2020/4/29/post-mortem- wireguard-server-connectivity-issues/. [Accessed: 21- May- 2020]

[11] OpenVPN. n.d. Setting Up Your Own Certificate Authority (CA) | Openvpn. [online]

Available at: <https://openvpn.net/community-resources/setting-up-your-own-certificate- authority-ca/> [Accessed 8 April 2020].

[12] Donenfeld, J., 2020. Protocol Cryptography - Wireguard. [online] Wireguard.com.

Available at: <https://www.wireguard.com/protocol/primitives> [Accessed 10 April 2020].

[13] Drake, M., 2018. How To Set Up An Openvpn Server On Ubuntu 18.04 | Digitalo-

(28)

cean. [online] Digitalocean.com. Available at: <https://www.digitalocean.com/communit y/tutorials/how-to-set-up-an-openvpn-server-on-ubuntu-18-04> [Accessed 8 April 2020].

[14] V. GUEANT, "iPerf - iPerf3 and iPerf2 user documentation", Iperf.fr, 2020. [On- line]. Available: https://iperf.fr/iperf-doc.php. [Accessed: 21- May- 2020]

[15] "VORACLE – OpenVPN Community", Community.openvpn.net, 2020. [Online].

Available: https://community.openvpn.net/openvpn/wiki/VORACLE. [Accessed: 21- Apr- 2020]

[16] "QoS - Quality of Service - in VOIP implementations.", VoIP-Info, 2020. [Online].

Available: https://www.voip-info.org/qos/. [Accessed: 22- May- 2020]

[17] "Bandwidth and Throughput in Networking: Guide and Tools - DNSstuff", DNSstuff, 2020. [Online]. Available: https://www.dnsstuff.com/network-throughput-bandwidth.

[Accessed: 05- Apr- 2020]

[18] "/etc/network/interfaces Ubuntu Linux networking example - nixCraft", nixCraft, 2020. [Online]. Available: https://www.cyberciti.biz/faq/setting-up-an-network-interfaces- file/. [Accessed: 22- May- 2020]

[19] H. Garden, HowStuffWorks, Tech, Computer, Hardware and Networking, "How a VPN (Virtual Private Network) Works", HowStuffWorks, 2020. [Online]. Available:

https://computer.howstuffworks.com/vpn7.htm. [Accessed: 14- May- 2020].

[20]"Why OpenVPN? | OpenVPN", OpenVPN, 2020. [Online]. Available: https://openvp n.net/faq/why-openvpn/. [Accessed: 15- May- 2020].

[21]"What is AES Encryption and How Does it Work?", SearchSecurity, 2020. [On- line]. Available: https://searchsecurity.techtarget.com/definition/Advanced-Encryption- Standard. [Accessed: 15- May- 2020].

[22]J. Donenfeld, "WireGuard: fast, modern, secure VPN tunnel", Wireguard.com, 2020.

[Online]. Available: https://www.wireguard.com/conceptual-overview. [Accessed: 16- May- 2020].

[23]"The ChaCha family of stream ciphers", Cr.yp.to, 2018. [Online]. Available: http://cr.y p.to/chacha.html. [Accessed: 16- May- 2020].

[24]"Difference between Block Cipher and Stream Cipher - GeeksforGeeks", Geeks- forGeeks, 2020. [Online]. Available: https://www.geeksforgeeks.org/difference-between- block-cipher-and-stream-cipher/. [Accessed: 17- May- 2020].

[25] J. Evans, "Why do UDP packets get dropped?", Julia Evans, 2020. [Online]. Avail- able: https://jvns.ca/blog/2016/08/24/find-out-where-youre-dropping-packets/. [Accessed:

24- May- 2020]

[26] "UDP - Packet loss, client and server on same computer", GameDev.net, 2020. [On- line]. Available: https://www.gamedev.net/forums/topic/622874-udp-packet-loss-client-

(29)

and-server-on-same-computer/. [Accessed: 25- May- 2020]

[27]"What is OpenVPN How to Use It?", Purevpn.com, 2020. [Online]. Available:

https://www.purevpn.com/what-is-vpn/protocols/openvpn. [Accessed: 26- May- 2020]

[28]J. Iannarelli and M. O’Shaughnessy, Information governance and security. Kidling- ton, Oxford, UK: Butterworth Heinemann is an imprint of Elsevier, 2015, p. 103

(30)

Appendices

A OpenVPN configuration

In this section, we present the configuration of CA, EasyRSA, and Encryption for Open- VPN configuration.

A.1 Setting up the CA, EasyRSA variables, and Encryption configuration

EasyRSA contains a default configuration file called vars.example, updating this file depends on personal details. It also contains a script file that was used to create many tasks with building the CA and sign certificates which clients need. Writing these commands to build the CA

./EASYRSA INIT-PKI

./EASYRSA BUILD-CA NOPASS

The output is two files ca.crt, ca.key. These files are responsible for generating certificates for clients. In more detail, ca.crt is used to verify the secure connection between the client and the server, while ca.key is responsible for signing certificates and keys for clients. Now with these configurations, we can sign certificates which we are going to generate for clients to use.

A.2 Creating a certificate for the server, generate an HMAC signature

Now we are going to generate a certificate for the server. To understand why we need to generate this certificate, we need to know how OpenVPN is establishing the connection.

OpenVPN works as bidirectional authentication, which means that before establishing the connection between the server and the client, a checking point is established from the user side to authenticate the server certificate [11]. At the same time the server is checking if the user certificate is valid and signed from the CA server side. Navigate to EasyRSA, then write this command

./EASYRSA GEN-REQ SERVER NOPASS

This will generate a certificate request file called server.req and a private key. Now we need to copy this key to the private folder with this command

SUDO CP /EASYRSA-3.0.4/PKI/PRIVATE/SERVER.KEY /ETC/OPENVPN/

Now it is time to sign the server certificate by calling a sign-req method with passing two parameters. The first parameter is the server, which is the type of the certificate we need to sign. The second parameter is the server request file server.req which we have generated in the previous step but without adding the file extension

./EASYRSA SIGN-REQ SERVER SERVER

To confirm request details, we just need to type ’yes’ and hit enter. Now it is time to configure Diffie-Hellman, which is used to exchange the keys between the client and the server privately by using TLS protocol, by adding these commands

(31)

SUDO CP/TMP/SERVER.CRT,CA.CRT /ETC/OPENVPN/

CDEASYRSA-3.0.4/

./EASYRSA GEN-DH

OPENVPN–GENKEY –SECRET TA.KEY

After that we just need to copy the new files to this directory:

SUDO CP /EASYRSA-3.0.4/TA.KEY /ETC/OPENVPN/

SUDO CP /EASYRSA-3.0.4/PKI/DH.PEM /ETC/OPENVPN/ With this step, the certificate of the server-side is ready.

A.3 Creating a certificate for the client

As we did in the previous step to generate the server certificate, we will do the same here;

but first, we will make a good folder structure for our keys and certificates:

MKDIR-P /CLIENT-CONFIGS/KEYS

Now generate a request certificate file after navigation to EasyRSA folder for the client by this command:

./EASYRSA GEN-REQ THECLIENT NOPASS

copying the generated files to this directory with these commands:

CP/TMP/CLIENT1.CRT /CLIENT-CONFIGS/KEYS/

CP /EASYRSA-3.0.4/TA.KEY /CLIENT-CONFIGS/KEYS/

SUDO CP/ETC/OPENVPN/CA.CRT /CLIENT-CONFIGS/KEYS/ The client certificate is ready now.

A.4 Adding configuration to the OpenVPN server

In these commands, we are going to copy a sample of the configuration file which called server.conf:

SUDO CP/USR/SHARE/DOC/OPENVPN/EXAMPLES/

SAMPLE-CONFIG-FILES/SERVER.CONF.GZ /ETC/OPENVPN/

SUDO GZIP-D /ETC/OPENVPN/SERVER.CONF.GZ

After that need to open it with text editor Nano to specify the authentication by using TLS protocol, the advanced encryption standard, Diffie-Hellman method, and the user and group configurations, by un-commenting line according to these:

SUDO NANO /ETC/OPENVPN/SERVER.CONF TLS-AUTH TA.KEY 0 THIS FILE IS SE-

CRET CIPHER AES-256-CBC AUTH SHA256 DH DH.PEM USER NOBODY GROUP NOGROUP PROTO UDP PORT60020

(32)

A.5 Adding configurations to the client, running OpenVPN service

To start OpenVPN service, we need to use the service manager by Linux, which is sys- temd, also a tool called systemctl, which is controlling the interface to start our service.

The server here is server.conf file, which we have edited previously by applying this command:

SUDO SYSTEMCTL ENABLE OPENVPN@SERVER

Creating configuration to the client-side, we are not going to create that for each client.

Otherwise, we are going to use a script to generate a configuration for each certificate.

First, we need to make a folder structure to store client configurations, then specify the IP address for the server, the protocol type, key direction, and TLS authentication, while commenting some lines like updating DNS information (if the client does not work on a Linux platform) by adding these commands:

MKDIR-P /CLIENT-CONFIGS/FILES CP/USR/SHARE/DOC/OPENVPN/EXAMPLES/

SAMPLE-CONFIG-FILES/CLIENT.CONF /CLIENT-CONFIGS/BASE.CONF NANO

/CLIENT-CONFIGS/BASE.CONF«BASE.CONF»REMOTE YOUR SERVER IP1194

PROTO UDP USER NOBODY GROUP NOGROUP CA CA.CRT

CERT CLIENT.CRT KEY CLIENT.KEY TLS-AUTH TA.KEY1

CIPHERAES-256-CBC

AUTHSHA256

KEY-DIRECTION 1

SCRIPT-SECURITY 2

UP/ETC/OPENVPN/UPDATE-RESOLV-CONF DOWN/ETC/OPENVPN/UPDATE-RESOLV-CONF

Now it is time to create a script file and make it executable. This script is taking key- and certificate files which we have generated in “step 4”, adding to them the base.conf file which we have created in this step by typing these commands:

(33)

NANO /CLIENT-CONFIGS/MAKE_CONFIG.SH

«MAKE _COFIG.SH»

!/BIN/BASH

FIRST ARGUMENT: CLIENT IDENTIFIER

KEY_DIR= /CLIENT-CONFIGS/KEYS

OUTPUT_DIR= /CLIENT-CONFIGS/FILES

BASE_CONFIG= /CLIENT-CONFIGS/BASE.CONF CATBASE_CONFIG

<(ECHO -E ’<CA>’) KEY_DIR/CA.CRT

<(ECHO -E ’</CA><CERT>’) KEY_DIR/1.crt

< (echo − e⁰ < /cert >< key >⁰) KEY _DIR/1.KEY

<(ECHO -E ’</KEY><TLS-AUTH>’) KEY_DIR/TA.KEY

<(ECHO -E ’</TLS-AUTH>’)

> OUTPUT_DIR/1.OVPN

CHMOD700 /CLIENT-CONFIGS/MAKECONFIG.SH

A.6 Send the certificate to the client:

By running the script file we will generate the client certificate, and the output is a file called theClient.ovpn. All the client needs it to get that file and to install the OpenVPN software and import this certificate to it to access the server. By typing these commands:

CD /CLIENT-CONFIGS

SUDO./MAKE_CONFIG.SH CLIENT1

LS /CLIENT-CONFIGS/FILES

B WireGuard configuration

In this section, we presented the configuration of WireGuard server configuration file, client tunnel configuration, and ran the service.

B.1 Create WireGuard configuration file

Now it is time to create a configuration file for WireGuard VPN, which will include the IP Addresses for clients and servers, as well as the private key for the server and public keys for the client. For now, we are creating the file with these commands:

SUDO NANO/ETC/WIREGUARD/WG0.CONF

Adding these lines to the file, add the private server key and the virtual private network IP address.

(34)

[INTERFACE]

ADDRESS = 192.168.10.1

PRIVATEKEY= “SERVER _PRIVATE _KEY” LISTENPORT= 51200

We have specified the port in which the VPN will listen to it.

B.2 Adding Peer to the configuration file

Now it is time to let the client give us the public key. Then we are opening the configuration file again to add the client public key with this command:

SUDO NANO/ETC/WIREGUARD/WG0.CONF

Adding the IP for the client, which should be allowed for the VPN network, also the client public key by adding these lines to the configuration file:

[PEER]

PUBLICKEY= “CLIENT_PUBLIC _KEY” ALLOWEDIPS= 192.168.10.2/32

Now the server configuration file is done.

B.3 Run the WireGuard VPN service

To run the WireGuard VPN service by writing these commands:

WG-QUICK UP WG0

SYSTEMCTL ENABLE WG-QUICK@WG0.SERVICE

wg0 is the interface name of the network. Then we apply the systemctl command, which is used by Linux to manage the services of the system.

B.4 Install WireGuard VPN software on the client machine

Now we are going to install WireGuard software to Windows 10. Adding a new tunnel will generate a public and private key. The public key is the same as in the previous configuration file. Editing the tunnel to add these lines:

[INTERFACE]

PRIVATEKEY= “CLIENT_PRIVATE_KEY” ADDRESS = 192.168.10.2/32 (ALLOWEDIPS).

PEER

PUBLICKEY= “SERVER_PUBLIC_KEY” ALLOWEDIPS= 192.168.10.0/24

ENDPOINT = “SERVER_PUBLIC_IP”:”PORT_NUMBER” PERSISTENTKEEPALIVE = 25

EndPoint: The server public IP address with the port, which we specified before 51200.

AllowedIPS: The virtual network IP which the server specified.

(35)

Address: The allowed IP address, which we specified in the configuration file to the client.

PublicKey: The server public key, which we generated in step 2.

Save the tunnel and connect [13, 11].

Degree Project at Bachelor Level Evaluation of WireGuard and OpenVPN VPN solutions

Author: Ahmad Anbarje Author: Mohammed Sabbagh Supervisor: Diego Perez Palacin