A study of the eﬀects of diﬀerent contextual variables on sorting algorithms

(1)

A study of the eﬀects of diﬀerent contextual variables on sorting

algorithms

Casper Björk

24-08-2020

Faculty of Computing

Blekinge Institute of Technology SE–371 79 Karlskrona, Sweden

(2)

This thesis is submitted to the Faculty of Computing at Blekinge Institute of Technology in partial fulﬁlment of the requirements for the bachelor’s degree in software engineering.

The thesis is equivalent to 10 weeks of full-time studies.

Contact Information:

Author(s):

Casper Björk

E-mail: cabk17@student.bth.se

University advisor:

Dr. Ahmad Nauman Ghazi

Department of Software Engineering

Faculty of Computing Internet : www.bth.se

Blekinge Institute of Technology Phone : +46 455 38 50 00 SE–371 79 Karlskrona, Sweden Fax : +46 455 38 50 57

(3)

Abstract

Background: Computers use sorting algorithms to prepare data for search or insert operations, these operations can be a bottleneck for performance.

Objectives: To evaluate sorting algorithms performances when existing in different implementation environments like different languages, sorting in different data types and in different sizes.

Methods: By performing an experiment with Java, C++, Python and Javascript on three diﬀerent sizes and three diﬀerent data types performance of sorting algorithms will be evaluated. The sorting algorithms used in this study are Insertion sort, Selection sort, Quick sort and Merge sort. They were all tested on each size on each language three times.

Results: In the end 432 tests were performed and the results found that Java had best execution time on all four algorithms with quick sort as the best algorithm.

The best data type could not be pinpointed down since they acted diﬀerently on diﬀerent algorithms. Quicksort was determined to be the fastest algorithm in the literature review which focused fastest algorithm

Conclusions. Together with the results of the experiment and literature review quicksort is the fastest sorting algorithm. The best performing implementation language was Java. For data types one type could not be determined the best only the worse could be decided, ﬂoats performed the worse of all three types.

Keywords: Sorting algorithms, Performance, Implementation languages, Data types.

(4)

Acknowledgements

I would ﬁrstly like to thank my supervisor Dr. Ahmad Nauman Ghazi for his help and support during this thesis. Without his help this thesis would never had been ﬁnished. I would also like to thank my family, girlfriend and friends for their support and encouragement throughout this thesis. One special thanks to my beloved friend Joel whom without his ideas and discussions this thesis would not be the same.

(5)

List of Tables

2.1 Time complexity sorting algorithms . . . 13

2.2 Implementation language speciﬁcs . . . 17

2.3 Experiment table . . . 18

4.1 Java result table . . . 23

4.2 C++ result table . . . 23

4.3 Python result table . . . 24

4.4 Javascript result table . . . 24

(7)

List of Figures

4.1 Slower algorithms performance for each language . . . 25

4.2 Faster algorithms performance for each language . . . 26

4.3 Data type performance for each language . . . 27

4.4 Data type performance with slower algorithms . . . 28

4.5 Data type performance with faster algorithms . . . 29

4.6 Slow sorting algorithms performance on diﬀerent sizes . . . 30

4.7 Fast sorting algorithms performance on diﬀerent sizes . . . 31

(8)

Chapter 1 Introduction

In a world full of computers, time is everything. A computer is fast, it can calculate, remember and draw up beautiful pictures. But there do exists a limit for how fast a computer can execute these operations and the limit may vary on how the program running on the computer is made. One major operation a computer does regularly that can aﬀect how fast a computer performs is the operation of sorting data. This can be a table of a employees where it needs to sort all employees by the last name alphabetically. It can be when a competition records the time of a lap around a circuit and wants to summarise a list with the fastest lap on top. After waiting 10 seconds for the computer to ﬁnish a user will want to perform an other task [1], this is why a fast performance is important.

What will be investigated in this study is how sorting algorithms performance can be aﬀected by other contextual factors rather than how the sorting algorithms themselves performs theoretically. There are already many studies of the performance between diﬀerent sorting algorithms in which they are compared to each other.

What is investigated in this study instead is the performance of a sorting algorithm when used with different implementation languages and if the data type also can affect the performance. The results of this study can then be used for decisions in which implementation language to be used or if different sorting algorithms should be used for different data types.

The examined languages are Java, C++, Python and JavaScript. These languages are very well known and is used in many programming projects to full ﬁll a projects task. Java and C++ are seen as the faster languages in the programming world while Python and JavaScript are seen as languages made not for the quickest performance but the ease of writing code and get something delivered faster. All the languages are already fast and have their own pros and cons, the best language to use is something that has to be decided based on several factors.

Based on the results of the experiment a discussion will be had about the different performances. The focus will be mainly about the programming languages and their differences but also about data types such as, integers (natural numbers), floats (decimal numbers) and chars (characters). As a result of the study, more knowledge of how a sorting algorithm performs is gained but also about the different programming languages.

(9)

1.1 Background

1.1.1 Sorting algorithms

To be able to define "Sorting algorithms" one must first understand what an algorithm is. In a MIT published book authored by T. H. Cormen et la [2] the definition for algorithm is explained. Algorithm can be defined as a set of carefully thought out sequences of steps a computer goes through to alter values that were sent into the algorithm. The algorithm alters the values and turn them into a new form which then will be sent forward in the system [2]. Sorting data structures is an example of a type of algorithm [2].

There is a need for sorted data structures in the perspective of a computer. A sorted data structure has an advantage compared to an unsorted data structure when it comes to performing a search [3]. This is true for both humans and computers.

An example of a search algorithm that are very quick and requires a sorted data structure is the binary search [4].

1.1.2 Programming languages

A programming language is a language system that is designed to write computer programs [5]. There are several languages in the world, example of the common languages are C++, Java and Python. Each of these languages has its own unique grammar or syntax. The syntax for a programming language describes how to form combinations of symbols to from a correct program [6]. Each programming language also have diﬀerent performances when processed through diﬀerent problems and solutions [7] which is interesting to evaluate.

1.1.3 Importance of software performance

Software performance is a broad and complex subject that changes depending on the requirement for the project. Performance is usually measured with time where it is grouped into throughput, response time and utilisation. For this study we care about the throughput, the amount of work that we can perform during a certain time [8].

If an interaction with a software takes more the 10 seconds a user will lose its patience or flow. A user feels this when a computer does not respond to the operation or action the user has started [1]. A big calculation like simulations are why supercomputers were created and being fast is sometimes a life critical task so both the hardware and software need to be efficient and quick [9]. While hardware is not being discussed in this study the software is and since both parts needs to be fast we will focus on how a engineer can use different sorting algorithms, data types and programming languages to achieve the best performance.

(10)

1.2 Related work

There is very much research done on various sorting algorithms, how to improve them, the creation of new ones, etc. However there is not much research that has been performed that compares multiple languages with multiple sorting algorithms and data sets. The research is most often performed on a single or two languages with one or two sorting algorithms. However, that does not mean that there is no previous research in this ﬁeld.

In this article by Lutz Prechelt [7] the focus is on which programming language is the most eﬀective language comparing implementations made by out of study programmers where they solve the same problem with diﬀerent programming languages.

Prechelt ﬁnds that generally Java is more favourable of the top languages, C and C++. However, C and C++ do have a better execution time than Java. Other ﬁndings which are not relevant to this study talk a lot for the scripting languages which were examined but when they are compared to the top languages they were not as good performance wise. This study is quite old (2000) and since this study all languages have gone through major updates.

There are also a few different types of research fields that have a correlation for this study. One type is about green computing and sorting algorithms, in this field a study from Chandra et al. [10] was made to study which sorting algorithm consumes the least amount of energy. To make this research they made an experiment with four sorting algorithms (bubble-, selection-, insertion- and quick2 sort) and three programming languages (Java, Visual Basic 6.0 and C#.Net). In their results they saw that different languages consumed different amount of energy and that different sorting algorithms consumed different amounts of energy. Java was the most power efficient language and quick2 sort was the most efficient sorting algorithm.

Another study in the same field by Chandra et al. [11] focused on the energy efficiency on a set of sorting algorithms. This study also made an experiment with 5 sorting algorithms (bubble-, insertion-, selection-, quick- and quick2 sort) and two data sets, one with integers and one with doubles. The results showed that the quick2 sort algorithm was the most energy- and time efficient algorithm. The result also showed that integer data type required less energy than the double data type.

There are other types of research that handle sorting algorithms such as an article from D.R Aremu et al. [12] that aims to see which sorting algorithm is the most eﬃ- cient. The article presented a comparative study of three sorting algorithms (heap-, quick and median sort). The study evaluated the performance of the algorithms with the CPU time and memory spaced used by the algorithms. The result was that the heap sort algorithm was faster and required less space than the other algorithms.

(11)

Chapter 2 Research methodology

2.1 Research questions

The research questions that were chosen for this study are:

RQ1: Which of the programming languages has the best average execution time for a given sorting algorithm?

Implementation languages have diﬀerent performances [13]. The article written by L. Prechelt [13] was published in 2000, that is 20 years ago from when this study is conducted. Since 2000 the used implementation languages have been updated so the performances are unknown. Sorting algorithms do themselves have their performance diﬀerences [14]–[16]. The question explores a combination of the two variables of implementation languages performance and sorting algorithms performances to see if there is any correlation to performances between a chosen sorting algorithm and implementation language.

RQ2: To which extent does the execution time of a given sorting algorithm change for the given data sets?

Sorting algorithms performance has a theoretical performance which is calculated with Big-O notations [3], [17]–[19]. The theoretical performance determines the efficiency of the sorting algorithms. The efficiency of A sorting algorithm affects the execution time for the sorting algorithm, the larger the data set the longer the execution time. This question is searching for the best sorting algorithm by looking at which of the sorting algorithms grow the least when increasing the data sets.

RQ3: Which data type has the best average performance for a sorting operation?

Data types are diﬀerent between diﬀerent languages. For integers, C++ has at least 16-bit integers [20] while Java has by default 32-bit [21]. JavaScript integer or number as it is called has a “double-precision 64-bit binary format IEEE 754” [22]

but in Python it is not speciﬁed as it is documented to have “unlimited precision”

[23]. The diﬀerences go on for ﬂoats and chars as well between the languages. This question will try to bring forward the best data type for a sorting operation.

(12)

2.2 Method

To conduct this study and answer the research questions data needs to be gathered.

The data that is needed is about implementation languages, sorting algorithms and data types. Firstly, a literature review of the current study’s on this subject will be performed. In addition to a literature review, an experiment was selected for the preferable way of gathering precise data from a controlled environment. A survey was considered an option but was dismissed. The dedicated time for this project is limited. For a survey to work, a big sample size is needed to make it viable [24].

The sample was needed to be gathered with the help of experts in this subject, an undertaking that could take an uncertain amount of time.

The selected languages to be examined in this experiment are as follows:

• Java

• C++

• Python

• Javascript (NodeJS)

These languages were selected for their popularity and wide use out in the world.

Three of the languages, Python, Java and Javascript is in the top 3 list of the most searched implementation languages on google [25]. An index made by Tiobe, an index that rates the popularity of programming languages based on the numbers of skilled engineers, courses and third-party vendors [26]. This index has Java, Python, C++ in the top 4 with JavaScript in the 7th position. The languages chosen are popular in both indexes which makes them a good choice for this study.

The selected sorting algorithms are:

• Insertionsort

• Selectionsort

• Quicksort

• Mergesort

The algorithms chosen for this study are a mix of theoretical faster and slower algorithms. See ﬁgure 2.1, the table shows the Best, Worse and average case for the selected sorting algorithms in a Big-O format [3], [17]–[19]. The variety of speeds of the algorithms gives a bigger variety of results.

(13)

Table 2.1: Time complexity sorting algorithms

Algorithm name Best case Average case Worst case Insertion sort [3],[27],[28] O(n) O(n²) O(n²) Selection sort [3],[27],[28] O(n²) O(n²) O(n²) Quick sort [3],[27],[28] O(n log n) O(n log n) O(n²) Merge sort [27],[28] O(n log n) O(n log n) O(n log n)

The data types that will be used are:

• Integer

• Float

• Char

These are the main data types in programming. As mentioned in the description of research question 3, the data type is diﬀerent between each language and thus can the overall performance for a data type can be evaluated by performing the sorting algorithms in diﬀerent languages.

For each datatype there exists three data sets in the sizes of small, medium and large. In these data sets one entry is one value in the data set. The values are for integers between 0-100, ﬂoats 0.00 - 9.99 and chars a-z. The sizes that are decided to be used are for small 10 000 entries, medium 100 000 entries and large 1 000 000 entries. These were generated in a Java program written for this subject and used the java class “Random” and the inbuilt functions “nextInt” and “nextFloat”. For char, the “nextInt” function is used with the range of 0 - 25 and the value returned is then added to a single char “a” which will give the results of a randomised char value between a-z.

Each language will test all of the sorting algorithms with different data sizes and data types. To make sure the data is as representative to the actual run time of a sorting algorithm each data size for the specified data type and the sorting algorithm will be run three times. These three runs will then be combined into one average performance for that specific data type, sorting algorithm, data size and implementation language. That makes the total test to be executed 4 ×3 × 4 × 9 = 432tests.

2.3 Hypotheses

The hypotheses presented in this chapter are the hypotheses I had before starting the experiment. Since the purpose of this experiment is to evaluate the sorting algorithms and their performance when performing in different languages, different data types and different data sizes. The structure used to present the hypotheses in this section follow the guidelines expressed in “Experimentation in Software Engi- neering” [29]. Each hypothesis will be presented with a sub hypothesis that lifts the different possible outcomes of the main hypothesis. The structure starts with the

(14)

null hypothesis (H0) that gives a negative hypothesis. The main measurement that will be considered is how long the execution time is for a certain sorting algorithm.

The hypotheses that I have for this study are as follows:

2.3.1 Hypothesis 1.

There is no notable diﬀerence between the sorting algorithms execution time when executed on diﬀerent implementation languages.

H0 : x1 = x2 = x3 = x4

There is a notable change between the sorting algorithms execution time when executed on diﬀerent implementation languages.

H1 : x1 = x2 = x3 = x4

x is the average time a sorting algorithm takes to perform its sorting. The num- ber is the diﬀerent implementation language the sorting algorithm is performed in.

Since diﬀerent implementation languages do exist there most exist a diﬀerence between the languages other than the syntax. I do believe that H1 will occur.

2.3.2 Hypothesis 2.

There will be a diﬀerence between the lower-level implementation languages than the higher-level ones. The higher-level implementation languages are NodeJS(Javascript) and Python. While Java and C++ are still high-level languages they are lower than the JavaScript and Python.

H0: H = L

Higher and lower languages have the same execution time when executing the sorting algorithms.

H1: H < L

Lower languages have better execution times than the higher languages.

H2: H > L

Higher languages have better execution times than the lower languages.

H and L stands for Higher respectively Lower level languages. NodeJS(JavaScript) and Python are considered to be a higher level than Java and C++ and thus they are in the H group while Java and C++ are in the L group.

I believe the lower languages will outperform the higher-level languages (H1). The ﬂexibility of writing in high-level languages comes often to the cost of performance and thus the lower level languages which usually are more complex to write has

(15)

better performance.

2.3.3 Hypothesis 3.

There will be a difference in execution times between the data types that are used by the sorting algorithms. Mainly integers and floats could have different execution times since they are similar while floats are longer.

H0: I = F = C

Integers, Floats and Chars have roughly the same average execution times.

H1: I = F = C

Integers, Floats and Chars do not have the same average execution times.

I = Average execution time for integers, F = Average execution time for ﬂoats and C = Average execution time for chars.

The diﬀerence in Floats and Integers between the languages are mainly in the amount of memory each type take-up in the computer. I believe that a diﬀer- ence will be seen between the types across this experiment.

2.4 Implementation methods

All projects are created and developed through Jetbrains development tools. Which are:

• For Java: IntelliJ IDEA Ultimate.

• For C++: CLion.

• For Javascript: Webstorm.

• For Python: Pycharm.

While the tool does not generally matter when developing code (it is usually up to personal preferences) it does matters for this experiment. For Java and C++, the built-in compiler tool is used. Which compiler version that was used for each implementation language will be mentioned in “Performing experiment” section.

To keep the code safe and to make sure it is organised each implementation has its own Gitlab project. They are all saved under the same group which groups them up on one page making it easier to ﬁnd. All the projects for each language can be found here: “https://gitlab.com/bachelor_experiment”

(16)

2.4.1 Sorting algorithms

Implementation of each sorting algorithm will be checked against the references that are listed below. This is to make sure that the implementations are correct. To make the experiment as valid as possible the implementation must be next to identical between all different languages even though that can be difficult. The references chosen are from books that are meant to learn the reader more about algorithms and to make sure that the implementations are correct not one single reference will be used but four from different authors.

• Grokking Algorithms: An illustrated guide for programmers and other curious people [17]

• A Common-Sense Guide to Data Structures and Algorithms [3]

• JavaScript Data Structures and Algorithms: An Introduction to Understand- ing and Implementing Core Data Structure and Algorithm Fundamentals [19]

• Python Data Structures and Algorithms [18]

2.4.2 Implementation tests

Performance will be calculated with the time it took to ﬁnish the sorting algorithm.

Each implementation language has to provide a reliable way to take the run time of a sorting algorithm. Listed below are the functions or frameworks built into the implementation languages which will used to record the run time of each speciﬁc sorting algorithm for that speciﬁed language:

• Java: nanoTime() - returns time from the JVM High resolution time source.

[30]

• C++: high_resolution_clock.now() - comes from the chrono header package and returns the time from the high resolution clock. [31]

• Python: process_time() - returns the time the process took when it was on the CPU excluding the time while it was in sleep. [32]

• Javascript: process.hrtime() - Returns the current High resolution time [33]

The testing will then be performed in the main file. The written code is simple, during the testing phase the only things that need to be altered between tests are the algorithm and test data. To keep a general flow between the implementation languages the code that was going to be changed have comments around them specifying which line that is allowed to change. To make it fair and easier to conduct the test all classes need for all the different sorting algorithms are imported from the beginning in the main test file. For each language this means that each class containing a sorting algorithm will be imported to the main test file for each test for all the sorting algorithms.

(17)

2.5 Performing experiment

2.5.1 Testing environment

The decided test environment is a laptop. This choice was made to capture a ordi- nary working computers perspective. While this computer is 3 years old it is still a high-end computer with good speciﬁcations. The decision for using this particular computer was also that it had a Linux operating system. Having an terminal based operative system opens up the ability to conﬁgure the execution of each test and handle the clearing of ram between each run much easier.

The computer that is used is an “HP Pavilion Power - 15-cb008no” laptop with the speciﬁcations:

• Intel i7-7700HQ 2.8GHz 6MB cache 4 cores

• 8 GB DDR4-2400 SDRAM

• NVIDIA GeForceR GTX 1050 (2 GB GDDR5 dedikerat)R

The system is running Pop!_OS 20.04 as its operative system which is a Ubuntu- based system created by System76.

Between each test a bash script will be executed to make sure ram caches and swap memory is cleared out giving each run the maximum amount of memory available.

Functions that are used in the script are, “3 > /proc/sys/vm/drop_caches” which clear the page cache, dentries and inodes. The other command is “swapoﬀ -a” and

“swapon -a” which clears the swap memory

Each language has its way to compile and run its code. Below is the listing for the version of each language’s compiler that will be used during the experiment.

Table 2.2: Implementation language speciﬁcs

Language Compiler

Java JDK 11.0.5

C++ gcc 9.3.0

NodeJS (Javascript) Node version 12.16.1

Python Python 3.8.2.

2.5.1.1 Running the experiment

To make sure each language gets the same prepared system which is made to make sure the result is viable a checklist for the setup of the lab environment shall be followed. After the lab setup checklist, the next checklist that shall be followed is one that is focused on the actual test run.

(18)

Environment setup checklist 1. Restart test computer.

2. Open up two terminals.

3. Navigate to the output folder for the selected language with one of the terminals.

4. Navigate to the “cleanRamAndSwapCache.sh” script location.

5. Proceed with the “Running sorting algorithm test” checklist for selected language. Each sorting algorithm shall be tested 3 times for each data size.

This checklist is made to be used any time the computer is started or when the switch between implementation languages are occurring. Below is the checklist used when running the tests.

Testing sorting algorithm checklist

1. Open the associated IDE for the language in question. IDEs are listed below [Table 2.3]

2. Change the code within the two comments that say “# CHANGE HERE ONLY #” to the sorting algorithm in question and the required data set.

3. If the language is Java OR C++ run the build command in the IDE.

4. Close the IDE.

5. Run “cleanRamAndSwapCache.sh”. Continue when the script is complete.

6. Have result document open on a separate device.

7. Use the terminal that is placed in the output folder and run use the language- speciﬁc command to run the code. The diﬀerent commands are listed below [Table 2.3]

8. The sorting algorithm is ﬁnished when the time is displayed and “Sorting ﬁnished” is displayed. Take the time and enter it in the result document.

9. Return to step 5 and repeat until there are three results for the same size and algorithm. When the third result is ﬁnished continue to step 1 or change language.

Table 2.3: Experiment table

Language IDE Launch command

Java IntelliJ IDEA Ulti-

mate

$ sudo java -jar Java.jar

C++ Clion $ sudo .\Cpp

NodeJS (Javascript)

Webstorm $ node main.js

Python Pycharm $ python3 main.py

(19)

This is the checklist and information used to perform the experiment. Another rule that was followed to save time was to stop testing of a data size when the time to execute the sorting algorithm took over one hour. More of this decision will be discussed in the discussion.

(20)

Chapter 3 Literature review

The goal of this review is to be able to claim which sorting algorithm has the best theoretical performance by looking at already ﬁnished studies on this subject. The articles that were chosen for the review were articles that were published after the year 1990, research was about existing algorithms and the study is not using an interview or survey as its research option.

An experiment about performance in smaller data sizes were performed [14]. The work was about several sorting algorithms like mergesort and selectionsort and compares them against each other. To make sure the implementations of the sorting algorithms were correct and eﬃcient all code was examined after the code had been compiled down to assembly code. This made sure the code did not have any performance errors [14]. Each sorting algorithm was tested on a wide range of array sizes going from an array size of 7 to an array size of 1000. What was also important to D. Mittermair and P. Puschner was the worst-case and best-case scenario for the algorithms. Each algorithm had a worst-case data set in which they would perform much worse than during normal circumstances.

Increasing the sizes of the arrays this experiment is similar to the article mentioned above. The experiment was made in C++ and featured well-known algorithms like quicksort and insertions sort [15]. The major diﬀerence with this article other than it using bigger sizes for the array sizes the above-mentioned one was published in 1997 while this article was published in 2018 [15].

The previous experiments have focused on one implementation language and the sorting algorithms performances. However, comparing two languages performance is something performed in this experiment [16]. The data sizes have also increased in this experiment. The focus was on which sorting algorithm was the better for Java or C. The experiment was conducted on the same computer in a Hotspot JVM and the array sizes were around 100 000 to 2 million. Apart from examining the performances of the sorting algorithms in the implementation languages the speciﬁc implementation languages were also examined for which handled the data sets the best [16].

Comparing all the results, D. Mittermair and P. Puschner [14] found that for lower sizes (size less than 25 entries) insertions sort was the fastest while for medium sizes

(21)

(size greater than 25 up to 100) quicksort was the better one and for large sizes (sizes greater than 100 up to 1000) distribution counting was the best. The unique data from this article are results for the worst-case scenarios which concluded that merge sort was the best performing algorithm in the medium to large sizes while for the smaller sizes the exponential growing algorithms (insertion sort/selection sort) performed the best. When the sizes of the array start to grow A. Frak et al [15]

found that quick sorts start to outperform all the other algorithms. In T. Fagbola and S. Thakur [16], results are split into two diﬀerent parts, one for Java and one for C. In the Java implementation quicksort was the best performing algorithm while in the C implementation heap sort was the better one. Note that for T. Fagbola and S. Thakurs [16] experiment was performed in the sizes of 100 000 to 2 million.

While quicksort is certainly the fastest it has its downsides, D. Mittermair and P.

Puschners [14] shows in their results that quicksort is not the best algorithm while handling a worst-case scenario. The sorting algorithm that is the best for handling those is Merge sort. Another point that should be taking into account is that in A. Fraks et al [15] article distribution counting was not in their list of sorting algorithms which when compared to D. Mittermair and P. Puschners [14] results could be a contender for the best algorithm for the medium to large sizes. Lastly, T.

Fagbola and S. Thakurs [16] in their experiment while comparing C and Java are running their experiment in a Java Virtual Machine. Hotspot (the JVM they are using) is created by Oracle, an organisation providing Java compilers for develop- ers. While C is a backbone language in the computer world one can believe that there could be some diﬀerent performances for the language (Java) that is running in an environment created for this speciﬁc language. When comparing it to another language in the same environment. However, since C is not apart of this study’s experiment the performance of C is irrelevant.

In conclusion, for the lower sizes of data (size less than 25) insertion sort is the best and for the rest, it can be concluded that quicksort is the best according to the articles that were chosen for this review. But since quicksort has bad performance during the worst case scenarios merge sort should be mentioned as well.

(22)

Chapter 4 Results and Analysis

The fastest and best performing implementation language during this experiment was Java. Together with Java, Quick sort was the best algorithm where it had a 30 millisecond diﬀerence to the second fastest algorithm Merge sort. For the best data type it is much closer but in the end char was the better one with a close second being integer.

Python was the worst performing language in this experiment. It can be seen in the following result tables. Several of the sorting algorithms are performing very bad at larger sizes of data and for python this was very clear. When 3 600 000 milliseconds is displayed in a table it means that a sorting algorithm was stopped before reaching its end since it took more than one hour to ﬁnish. Some result entries are above one hour and those are the sorting algorithm that ﬁnished before the execution was stopped and thus their times were entered.

The tables below show the execution times in milliseconds for each implementation language in all the data sizes for each data type. The data displayed is the three executions times combined into one value. Thus the value shown is the product of the average taken of the three executions for each sorting algorithm, data size, data type and implementation language.

(23)

Table 4.1: Java result table

Java Small Results Integer Float Char

Insertionsort 16,67ms 21,33ms 13,67ms

Selectionsort 61,33ms 45,00ms 62,33ms

Quicksort 2,33ms 2,67ms 2,00ms

Mergesort 4,00ms 4,67ms 4,00ms

Java Medium Results Integer Float Char

Insertionsort 1002,33ms 1179,67ms 773,67ms Selectionsort 5322,00ms 4055,67ms 5364,33ms

Java Large Results Integer Float Char

Table 4.2: C++ result table

C++ Small Results Integer Float Char

C++ Medium Results Integer Float Char

C++ Large Results Integer Float Char

(24)

Table 4.3: Python result table

Python Small Results Integer Float Char

Python Medium Results Integer Float Char

Python Large Results Integer Float Char

Table 4.4: Javascript result table

Javascript Small Results Integer Float Char

Javascript Medium Results Integer Float Char

Javascript Large Results Integer Float Char

(25)

4.1 Analysis

Performance evaluation for the implementation languages and their performance on sorting algorithms were made with combining of all the executions for a speciﬁc language, sorting algorithm and data type. This means that all recorded 432 executions for each sorting algorithm in each implementation language and data type is combined and the average is taken. This will create a performance value for each language that has the average performance for the sorting algorithm on the speciﬁc language. The graph is divided into two graphs to make the faster algorithms visible in the graphs.

In these two graphs the X-axle is representative in milliseconds and the Y-axle sorts the two sorting algorithms into two sections with the implementation languages as the blocks rising up.

Figure 4.1: Slower algorithms performance for each language

32922,30 0,00

200000,00 400000,00 600000,00 800000,00 1000000,00 1200000,00 1400000,00

Insertionsort Selectionsort

milliseconds Combined performance: Slower algorithms

Java C++ Python Javascript

Java is the best performing for these two algorithms. The best of the worse are Insertion sort but only when comparing the Java, C++ and Javascript implementation. Python has almost the same result when viewing the graph but looking at the numbers it is visible that selection sort is 4000 milliseconds faster.

(26)

Figure 4.2: Faster algorithms performance for each language

35,41 64,11 67,74

433,96 268,07

468,74

0,00 500,00 1000,00 1500,00 2000,00 2500,00

Quick sort Mergesort

milliseconds

Combined performance: Faster algorithms

The best performing implementation language was again Java and for the algorithm Quicksort, however, C++ is closer than in Figure 4.1. However C++ increases in its execution times by 7 and does not follow the trend Java is following by only increasing by 2. C++ performed only a little better than JavaScript in Mergesort.

JavaScript is a language that also follows the increase of 2 like Java. What is also interesting to notice is the python performance. Quicksort is the slowest algorithm for Python and Mergesort is a better algorithm for Python.

Overall it can be concluded that Quicksort is the fastest algorithm when performed in Java. Python is performing against the general trend when compared to other languages. Python increases the performance for the sorting algorithms that the other languages decreases their performance in. C++ has a noticeable anomaly when performing the Mergesort algorithm and does not follow the general decease in performance that Java and JavaScript follow but rather performs even worse.

To evaluate if the diﬀerent data types had performance diﬀerences between each other or the implementation languages an average performance was calculated. The average performance was calculated by adding all the execution times for all the sorting algorithms performed in one data type and implementation language and take the average of that addition.

(27)

Figure 4.3: Data type performance for each language

341879,58 347764,83

0,00 200000,00 400000,00 600000,00 800000,00 1000000,00 1200000,00

Integer Float Char

milliseconds Data type combined for each language

No data type can be said to be the fastest overall the languages in this graph. All languages except JavaScript follow a trend between each data type where integer and char a more or less equal and ﬂoat has a worse performance. The anomaly is the JavaScript performance when handling decimal numbers and the small noticeable diﬀerence between the trend of integer and char where JavaScript handles chars better than integers.

Another view on the data type performance is to look at the effect the different sorting algorithms have on the data types performance. A performance average of the execution times for each sorting algorithm in a specific data type is calculated.

Each recorded execution time for a sorting algorithm not caring for the implementation languages will give a general performance value for that sorting algorithm.

The graph for this calculation is split into two parts to make it more visible since there is a big gap between the slower algorithms and faster ones. The ﬁrst one is of the slower algorithms and the second is with the faster algorithms.

(28)

Figure 4.4: Data type performance with slower algorithms

0 200000 400000 600000 800000 1000000 1200000

Integer Float Char

milliseconds Data type performance slower algorithms

The best performing data type in the slower sorting algorithms are char and its when being executed in insertion sort and selection sort. An interesting result is the performance of insertion sort when using ﬂoats where it performs worse than Selection sort.

(29)

Figure 4.5: Data type performance with faster algorithms

0 100 200 300 400 500 600 700 800

Integer Float Char

milliseconds Data type performance faster algorithms

Quicksort Mergesort

Integer is the best performing data type when performed for in quicksort. Other- wise is the performance quite linear between the diﬀerent data types, integer has a little faster execution time in quick sort but between char and there is only a gap of approximately 20 milliseconds. Both algorithms see an increase of time when executing ﬂoats.

The diﬀerent sizes had some impact on the performance of the algorithms. The overall performance was calculated by adding every execution time for an algorithm in one size. This means that all the execution times for the small data set in insertion sort was summed up together and then an average was taken from that value.

This gave an overall performance value for each sorting algorithm. Two graphs were created again to help with the visibility of the fastest algorithms.

(30)

Figure 4.6: Slow sorting algorithms performance on diﬀerent sizes

882,56 957,83 0,00

500000,00 1000000,00 1500000,00 2000000,00 2500000,00 3000000,00

Small Medium Large

milliseconds Slower Algorithms performance on size

The slow algorithms in this graph act just like expected, they grow exponentially for each increase in size. Insertion sort performs the fastest when comparing them both. Both sorting algorithms follow their theoretical performance of n² and when the data sizes increases the execution times grow exponentially as well. These execution times are not even close to the other algorithms that are shown in ﬁgure 4.7.

(31)

Figure 4.7: Fast sorting algorithms performance on diﬀerent sizes

33,75 51,75

0 200 400 600 800 1000 1200 1400 1600 1800

Small Medium Large

milliseconds Faster Algorithms performance on size

Quicksort Mergesort

Notice the diﬀerence in the X-axle when comparing the to ﬁgure 4.6. The best performing in this graph and the experiment is Quicksort. Mergesort is still fast but when the size grows the performance decreases.

(32)

Chapter 5 Discussion

The results for this study shows that Java has performed the best overall. This result follows with the results found in the literature review however, the other language’s performances are interesting as well. Python is the worst performing language in this experiment, performing worse than Javascript. The long execution times could be because of three possible reasons.

Firstly, it can be that the implementation of the python code is not optimal and could with some changes become faster. It could be that the functionality to record the run-time for Python to finish a sorting operation was working differently compared to the other languages. Comparing the documentation between the different languages function for displaying run-time the python documentation [32] never mention the high resolution clock that Java [33], C++ [31] and NodeJS(JavaScript) [33] does. This can give an unjustified execution time as the answer for each execution in Python.

Data type performance showed that integer and char were the faster data types from the selection of data types and the performance of ﬂoats was worse because of the length of each entry. For Chars each entry was only one character like “a”, “f”

or “o” while for Integers it was in the range of 1 digit to 3 digits. That means the integer has at maximum 3 characters and lowest 1 character. This could impact the performance difference for floats which always had 3 digits. Char had a performance increase in figure 4.4 which could be because of the digit difference. The difference would not have done a major difference in the smaller sizes however when it came up to the larger sizes, the one million entries it would start to affect. Since integer was sometimes three characters and sometimes one character the performance difference was minimal but this could be why floats be showed worse performance. Another notable reason could be the difference in memory size between the data types. The small differences could effected the execution times when the program had to load in the data

Due to the time limit of this experiment a decision was made to stop algorithms if they exceeded the one-hour time limit. If an algorithm would execute slow and it took one hour the ﬁrst time it was executed the test of that algorithm with that size did not continue. This gave a somewhat incorrect result. The motivation for this decision is that the goal of the experiment was not to ﬁnd the algorithm or lan-

(33)

guage with the worse performance. How bad they were is not one hundred percent certain. This decision saved a lot of time in the experiment. If the experiment did not have this rule a sorting algorithm that had an average of one and a half-hour should have been run three times. That is four and a half-hour’s of a days work which would have been spent on one sorting algorithm. That amount of time did not exist.

It is also interesting to look at the results of C++ and Javascript. The Javascript implementation was executed and compiled using NodeJS. NodeJS is using the Google V8 engine, written in C++ [34]. It raises the question if there is any performance diﬀerence between C++ and Javascript. The performance of the NodeJS compiler could write better than the C++ written for the experiment. For example, the NodeJS could have had a better way of using lists and arrays and would thus have a better performance than the implementation used in the native C++ program.

It could have been like that but in the end, with the results at hand, it was not the case. Looking at figure 4.1 it can be seen that there exists a big difference in the performance for C++ when using the slower algorithms but when looking at figure 4.2 it can be seen that C++ has the same performance as JavaScript when executing mergesort. The similar results could be because of the V8 engine [34] which handles the memory allocation automatically. The V8 engine is written in C++ [34]

which is why the same technique of memory allocation could aﬀect the results on memory heavy sorting algorithms. Looking at ﬁgure 4.3 which has calculated the average performance of each language based on data types the similarities of C++

and JavaScript are visible again.

5.1 Research Questions

In this section, the research questions will be presented more thoroughly and the result of the study and experiment will be discussed with the research questions in mind.

5.1.1 Research Question 1.

“Which of the programming languages has the best average execution time for a given sorting algorithm?”

This question explores an interesting part of this study, the question of different performances for implementation languages. Will they have a significant difference between languages that could result in a decision in the early stages of a project?

This was the question I asked myself when the idea of this study took form. It was what started the idea for the study and the reason for the large experiment.

The experiment had to be large for this question to make sense in my mind. What follows with a broader experiment is that other angels can be explored as well which is how the other questions were created.

The results show that not one single language was only good at one speciﬁc algorithm but showed that one single language was faster compared to the others on

(34)

all algorithms. That language is Java which means that Java is the programming language with the best average performance for all the sorting algorithms. For some algorithms, C++ is close to being the fastest with only 30 milliseconds behind Java in Quicksort but that is not consistent over all the algorithms. Java always performs the best and thus would be the choice according to the results in this study.

5.1.2 Research Question 2.

“To which extent does the execution time of a given sorting algorithm change for the given data sets?”

The data sets in this study were created to perform a good experiment, an experiment without different sizes and different data types creates a very shallow experiment. Small (10 thousand), Medium (100 thousand) and Large (1 million) is a good variety of sizes for a computer. A computer will have no problem at all with sorting sizes of 10 or even 100 so no major differences would have been seen between the algorithms. As reviewed in the article written by D. Mittermair and P. Puschner [14] the results for the smaller sizes are very fast being toward the 1.2 nanoseconds area.

Looking at the figures 4.6 and 4.7 the different performance for the different sizes are visible. The slower algorithms with the theoretical performance of n² (reference table 2.1) increase their execution times exponentially when the sizes increases.

For the faster algorithms, Quicksort with nlog(n) (reference table 2.1) theoretical performance is the best overall performing algorithm, this is the same result gathered from the literature review for the larger sizes. The second best is Mergesort with the same theoretical performance of nlog(n) (reference table 2.1), the performance is not equal to Quicksort in this experiment. Mergesort is also considered in the literature review but in this study, the best and worse case of an algorithm is not explored which if it were explored then maybe Mergesort would have been better.

5.1.3 Research Question 3.

“Which data type has the best average performance for a sorting operation?”

This questions is interesting since during this study no other studies have shown up that take on this question. Since the main goal was to figure out the contextual performances for sorting algorithms the data type had to be explored as well. Even though the differences would be minimal between the types there was no clear answer when searching on the topic in other studies. Either it is common knowledge that there is no difference or people have not tested it since there is not enough interest.

Integer is the best performing data type for the fastest sorting algorithms as seen in ﬁgure 4.5. However, looking at the slower algorithms, chars is the best performing when executed on insertion sort. This could be because of the problem with the data sets and the length of characters since a diﬀerence between integer and chars

(35)

is seen in selection sort as well. This was discussed before in this chapter. An integer can have between 1-3 characters in this experiment while char only has 1 character on every entry. Again the different types take up a different amount of memory space which is a possible reason for why a difference occurs for the slower algorithms. Since the theoretical n² algorithms grow exponentially when met with increasing sizes the time difference of loading a smaller type (memory space-wise) could make a difference in the final execution times but the difference would not be that noticeable when looking at the faster algorithms.

5.2 Validity threats

Human errors

This was a big experiment and it created a big data set. A lot of data is always good, it gives many opportunities to look at many diﬀerent angels. But it also gives a bigger room for errors or missed angles. An error that can easily occur is the logging of incorrect data when you are alone and do not have an automatic system for recording the data, human errors can occur easily. This could have happened in this experiment and it would be diﬃcult to actually check since the responses from the sorting executions are already gone and only the logging done by me is remaining.

Background processes

The computer these experiments were run on did not always have the same performance even though the process of clearing out the ram was performed, the computer could have had background processes that were unknown and had not any control over. The long-running algorithms could have been aﬀected by these processes which could have aﬀected the execution time. The best lab environment would have no unnecessary background processes and would only focus on the sorting algorithm.

Ineﬃciently written software

As discussed in this chapter the implementation of all the sorting algorithms in the diﬀerent implementation languages are not the best implementations for each language. An expert from each language would be able to use more out of the language and make a diﬀerence in the performance. The implementations written for this study has been done by a bachelor’s degree student and room for improvement was certainly there.

Time limit

The sorting algorithms had a time limit put onto them. That was to make sure that one slow sorting algorithm did not have to take up an entire day. However, this can cause a threat to the validity, they perform badly but let’s say that insertion sort was taking two hours to complete and selection sort only takes one hour five minutes but they are ranked equally. That would be wrong and insertion sort would not deserve to be ranked next to selection sort. While that could happen it will not be known since the limit was in place. What was also affected by the time limit was the number of runs required by each test. For this study, only 3 runs on each specific test were performed which is too small to catch the spread of times a computer can perform. With more time each test should have been run at least 100 times with a selection process of the top 10 fastest time and the average is taken on those 10 times.

(36)

Data set character lengths

In the discussion, the performance of float is examined. The focus of the performance difference is the uneven amount of digits for each data type. Chars are formed with the use of single letters which means they are only one character long. Integer has between one to three digits in their entries. Floats always has three digits. This could be the reason for the poor performance from floats and is something that could be a threat for the results of this experiment. What should have been done was to make the data sets fair in character length. However, each data type has a different memory size which could also be the reason for the different performances.

(37)

Chapter 6 Conclusion

The goal of this study was to examine sorting algorithms performance when compared in diﬀerent contexts. The study conducted a smaller literature review and an extensive experiment. The experiment covered the diﬀerent contexts like implementation languages, data types and data sizes around sorting algorithms. The sorting algorithms used were two theoretically slower algorithms, Insertion sort and Selection sort and two theoretically faster algorithms, Quicksort and Merge sort.

Four implementation languages were selected by looking at two diﬀerent popularity indexes covering the most popular implementation languages [25], [26]. Java, C++, Python and JavaScript(NodeJS) were chosen. Sizes were decided to be 10 000, 100 000 and 1 000 000. These sizes were then created in three data types integer, ﬂoat and char. Each test were performed three times and the three values recorded was than calculated into one average performance value in milliseconds. Totally 432 tests were performed across all sorting algorithms in all implementation languages together with all data types and data sizes.

In the end, it was concluded that Java was the best implementation language for all the sorting algorithms. The worse performing language was Python having an continued slower trend in all sorting algorithms except merge sort, the best algorithm for Python. The best sorting algorithm overall was Quick sort which performed best compared to all other algorithms in all sizes. The Quick sort result goes together with the results from the literature review which also concluded that Quicksort was the best algorithm. For data types, the best data type could not be concluded.

Both char and integer outperformed each other on diﬀerent sorting algorithms, in insertion sort and selection sort char was the better data type and for quick sort and merge sort integer was the better data type. Float performed worse at all times during the experiment compared to the other data types.

6.1 Future work

There are deep dives that can be taken in the many different angels that exist in this subject. Looking at sorting algorithms exact performance in higher-level languages when they are not compared to lower-level languages like Java and C++, which of the easier to get into languages is the fastest. Improving the implementation for a specific language could make the difference in a sorting algorithms performance.

The sorting algorithms in this study were written by a Software engineer student

(38)

that is not a professional on any of the listed languages and had not time to optimise the sorting algorithms for each language.

Data types are interesting as well, data types like strings are a big subject of their own. Sorting algorithms are not only made for sorting numbers and could be im- proved to sort strings in the best way. Another point of view can be to look at when performance will be eﬀected if the data type starts growing for example, the number of digits/characters.

The never-ending examination of performance is important to evaluate if languages start handling data better or worse and since they get updated yearly it can be interesting to see if progress is being made or lost performance-wise. Sorting algorithms do have a big important role in computer science and will for a long time to come and thus needs to be evaluated together with the progress of implementation languages.

(39)

References

[1] (1993). Response time limits, nnGroup, [Online]. Available: https : / / www . nngroup . com / articles / response - times - 3 - important - limits/. (accessed: 18-07-2020).

[2] T. H.Cormen, C. E. Leiserson, R. L. Rivest, and C. Stein, Introduction to Algorithms, Second Edition. The MIT Press, 2001, isbn: 9780262032933.

[3] J. Wengrow, A Common-Sense Guide to Data Structures and Algorithms.

Pragmatic Bookshelf, 2017, isbn: 9781680502442.

[4] R. Sedgewick, Introduction to Algorithms, Second Edition. Addison-Wesley Professional, 1997, isbn: 9780768685312.

[5] C. Dictionaries, Collins English Dictionary, 12th. Collins UK, 2014, isbn:

0007522746.

[6] N. Ari and N. Mamatnazarova, “Programming languages,” in 2014 11th Inter- national Conference on Electronics, Computer and Computation (ICECCO), Sep. 2014, pp. 1–8. doi: 10.1109/ICECCO.2014.6997548.

[7] L. Prechelt, “An empirical comparison of seven programming languages,” Com- puter, vol. 33, no. 10, pp. 23–29, 2000.

[8] V. Cortellessa, A. D. Marco, and P. Inverardi, Model-Based Software Perfor- mance Analysis. Springer, 2011, isbn: 9783642136207.

[9] (2020). Why do supercomputers matter for your every day life, European Commission, [Online]. Available: https://ec.europa.eu/digital-single- market/en/blog/why-do-supercomputers-matter-your-everyday-life.

(accessed: 18-07-2020).

[10] T. B. Chandra, P. Verma, and A. K. Dwivedi, “Impact of programming lan- guages on energy consumption for sorting algorithms,” in Software Engineer- ing, M. N. Hoda, N. Chauhan, S. M. K. Quadri, and P. R. Srivastava, Eds., Singapore: Springer Singapore, 2019, pp. 93–101, isbn: 978-981-10-8848-3.

[11] T. B. Chandra, V. Patle, and S. Kumar, “New horizon of energy eﬃciency in sorting algorithms : Green computing,” Oct. 2013.

[12] D. Aremu, O.OAdesina, O. Makinde, O.Ajibola, and O. Agbo-Ajala, “A com- parative study of sorting algorithms,” African Journal of Computing & ICT, vol. 6, no. 5, pp. 199–206, Dec. 2013, issn: 2006-1781.

[13] L. Prechelt, “An empirical comparison of seven programming languages,” Com- puter, vol. 33, no. 10, pp. 23–29, Oct. 2000, issn: 1558-0814. doi: 10.1109/

2.876288.

(40)

[14] D. Mittermair and P. Puschner, “Which sorting algorithms to choose for hard real-time applications,” Proceedings Ninth Euromicro Workshop on Real Time Systems, pp. 250–259, 1997.

[15] A. Frak, M. Saringat, Y. Prasetyo, A. Mustapha, H. Aman, and N. Ibrahim,

“Comparison study of sorting techniques in static data structure,” Data Infor- mation Engineering, Vol. 10 No. 6, pp. 106–112, 2018.

[16] T. Fagbola and S. Thakur, “Investigating the eﬀect of implementation languages and large problem sizes on the tractability and eﬃciency of sorting algorithms,” International Journal of Engineering Research and Technology, pp. 196–203, 2019.

[17] A. Y. Bhargava, Grokking Algorithms: An illustrated guide for programmers and other curious people, eng. Manning Publications, 2016, isbn: 9781617292231.

[18] B. Baka, Python Data Structures and Algorithms, eng. Packt Publishing, 2017, isbn: 9781786467355.

[19] S. Bae, JavaScript Data Structures and Algorithms: An Introduction to Under- standing and Implementing Core Data Structure and Algorithm Fundamentals, eng. Apress, 2019, isbn: 9781484239889.

[20] (2020). Cpp: Fundamental types, Cppreference, [Online]. Available: https : //en.cppreference.com/w/cpp/language/types. (accessed: 16-09-2020).

[21] (2020). Java: Primitive data types, Oracle, [Online]. Available: https://docs.

oracle.com/javase/tutorial/java/nutsandbolts/datatypes.html. (accessed: 16-09-2020).

[22] (2020). Javascript: Number, MDN, [Online]. Available: https://developer.

mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/

Number. (accessed: 16-09-2020).

[23] (2020). Python: Built-in types, Python.org, [Online]. Available: https : / / docs.python.org/3.4/library/stdtypes.html. (accessed: 16-09-2020).

[24] A. N. Ghazi, K. Petersen, S. S. V. R. Reddy, and H. Nekkanti, “Survey research in software engineering: Problems and mitigation strategies,” IEEE Access, vol. 7, pp. 24 703–24 718, 2019.

[25] (2020). Pypl index, PYPL, [Online]. Available: http : / / pypl . github . io / PYPL.html. (accessed: 16-09-2020).

[26] (2020). Tiobe index, Tiobe, [Online]. Available: https://www.tiobe.com/

tiobe-index//. (accessed: 16-09-2020).

[27] D. Dharmajee Rao and B. Ramesh, “Experimental based selection of best sorting algorithm,” Computer, vol. 2, no. 4, pp. 2908–2912, Aug. 2012, issn:

2249-6645.

[28] A. Mishra and D. Garg, “Selection of best sorting algorithm,” Computer, vol. 2, no. 2, pp. 363–368, Jul. 2008.

[29] C. Wohlin, P. Runeson, M. Höst, M. C. Ohlsson, B. Regnell, and A. Wesslén, Experimentation in Software Engineering, eng. Springer, 2012, isbn: 978-3- 642-29044-2. doi: 10.1007/978-3-642-29044-2. [Online]. Available: http:

//dx.doi.org/10.1007/978-3-642-29044-2.

(41)

[30] (2020). Oracle nanotime documentation, Oracle, [Online]. Available: https:

//docs.oracle.com/en/java/javase/11/docs/api/java.base/java/

lang/System.html#nanoTime(). (accessed: 15-06-2020).

[31] (2020). C++ documentation for high resolution clock, The C++ Resource Net- work, [Online]. Available: http://www.cplusplus.com/reference/chrono/

high_resolution_clock/now/. (accessed: 15-06-2020).

[32] (2020). Python documentation for process_time(), Python Software Founda- tion, [Online]. Available: https : / / docs . python . org / 3 / library / time . html#time.process_time. (accessed: 15-06-2020).

[33] (2020). Javascript/nodejs documentation for process.hrtime_bigint, OpenJS foun- dation, [Online]. Available: https : / / nodejs . org / api / process . html # process_process_hrtime_bigint. (accessed: 15-06-2020).

[34] (2020). V8 javascript engine, Google, [Online]. Available: https://v8.dev/

docs. (accessed: 23-06-2020).

A study of the eﬀects of diﬀerent contextual variables on sorting algorithms