Linköpings universitet SE–581 83 Linköping
Linköping University | Department of Computer and Information Science
Bachelor’s thesis, 15 ECTS | Computer Science
2021 | LIU-IDA/LITH-EX-G–21/066—SE
Characterization of clients-side
revocation checks and their
security-performance tradeoffs
Karaktärisering av upphävningskontroll av certifikat från
klien-tens sida och deras för- och nackdelar mellan säkerhet och
pre-standa
Christoffer Gärdin
George Shnouda
Supervisor : Niklas Carlsson Examiner : Marcus Bendtsen
Upphovsrätt
Detta dokument hålls tillgängligt på Internet - eller dess framtida ersättare - under 25 år från publicer-ingsdatum under förutsättning att inga extraordinära omständigheter uppstår.
Tillgång till dokumentet innebär tillstånd för var och en att läsa, ladda ner, skriva ut enstaka ko-pior för enskilt bruk och att använda det oförändrat för ickekommersiell forskning och för undervis-ning. Överföring av upphovsrätten vid en senare tidpunkt kan inte upphäva detta tillstånd. All annan användning av dokumentet kräver upphovsmannens medgivande. För att garantera äktheten, säker-heten och tillgängligsäker-heten finns lösningar av teknisk och administrativ art.
Upphovsmannens ideella rätt innefattar rätt att bli nämnd som upphovsman i den omfattning som god sed kräver vid användning av dokumentet på ovan beskrivna sätt samt skydd mot att dokumentet ändras eller presenteras i sådan form eller i sådant sammanhang som är kränkande för upphovsman-nens litterära eller konstnärliga anseende eller egenart.
För ytterligare information om Linköping University Electronic Press se förlagets hemsida http://www.ep.liu.se/.
Copyright
The publishers will keep this document online on the Internet - or its possible replacement - for a period of 25 years starting from the date of publication barring exceptional circumstances.
The online availability of the document implies permanent permission for anyone to read, to down-load, or to print out single copies for his/hers own use and to use it unchanged for non-commercial research and educational purpose. Subsequent transfers of copyright cannot revoke this permission. All other uses of the document are conditional upon the consent of the copyright owner. The publisher has taken technical and administrative measures to assure authenticity, security and accessibility.
According to intellectual property law the author has the right to be mentioned when his/her work is accessed as described above and to be protected against infringement.
For additional information about the Linköping University Electronic Press and its procedures for publication and for assurance of document integrity, please refer to its www home page: http://www.ep.liu.se/.
Abstract
There are several different methods for checking whether certificates on the web have been revoked, timely discovery of revoked certificates are important to ensure security when authentication within the HTTPS protocol is used. These methods have both advan-tages and disadvanadvan-tages as they can contribute to security but at the same time worsen performance on the web browsers. This thesis examines these methods in more detail to identify the pros and cons and whether it is possible to find a good tradeoff between se-curity and performance. This is important as a user is exposed to major sese-curity flaws if the integrity of a domain cannot be verified. Our analysis includes to which extent OCSP and CRL are used, how much OCSP affects the browser Firefox’s performance, and how many web servers implement methods to verify revoked certificates, such as OCSP staple and must-staple. We also compare web browsers own lists of revoked certificates and look for patterns and differences between them. The analysis shows that OCSP and CRL have largely been replaced by other methods of verifying revoked certificates such as OneCRL and CRLSet. It turned out that OneCRL and CRLSet only cover a small fraction of the total number of certificates available. Often, it takes several weeks for certificates to appear in these lists after being revoked. We also found that OCSP’s impact on the web pages per-formance is minimal. We concluded that the existing methods for verifying revocation sta-tuses are inadequately used by CAs, web browsers and web servers, which poses a major security risks for users. Many certificates are not checked at all. However, we believe that it is possible to increase the security without reducing performance if CAs, web browsers and web servers can collaborate in the development of improving and combining current methods for checking revoked certificates.
Acknowledgments
We would like to thank Associate Professor Niklas Carlsson at Linköping University for his guidance and being a resource when writing this thesis.
We also would like to thank Adam Nyberg, Tom Larsson, Sofia Bertmar and Johanna Ger-hardsen for their thoughts and suggestions for improvements during the process of writing this thesis.
Contents
Abstract iii
Acknowledgments iv
Contents v
List of Figures vii
List of Tables viii
1 Introduction 1 2 Background 3 2.1 Certificates . . . 3 2.2 Obtaining a certificate . . . 4 2.3 Certificate revocation . . . 4 3 Related Work 7 4 Method 9 4.1 Adoption of OCSP and CRL . . . 9
4.2 OCSP page load performance . . . 10
4.3 Website certificate information . . . 10
4.4 CRLSet and OneCRL . . . 11
5 OCSP and CRL Results 13 5.1 Adoption of OCSP and CRL . . . 13
5.2 OCSP page load performance . . . 14
5.3 Webserver implementations and OCSP checks for EV certifcates . . . 15
6 OneCRL and CRLSet Results 18 6.1 Size . . . 18 6.2 Update frequency . . . 19 6.3 Revocation reason . . . 19 6.4 Certificate types . . . 20 6.5 Certificate status . . . 20 6.6 Certificate issuers . . . 21
6.7 Date when upload to OneCRL . . . 22
6.8 Certificate intersections . . . 25
6.9 Certificate validation types . . . 26
7 Discussion 27 7.1 Results . . . 27
7.3 The work in a wider context . . . 29
8 Conclusion 31
8.1 Future work . . . 32
List of Figures
2.1 The chain of trust. . . 4
5.1 Use in percentage of OCSP and CRL. . . 14
5.2 Page load time with OCSP on and off. . . 15
5.3 Percentage of OCSP staple and OCSP must-staple. . . 15
5.4 Percentage OCSP checks for EV certificates. . . 16
5.5 Percentage of issuers with EV certificate and no OCSP check. . . 17
6.1 Fraction of revoked certificate reason in OneCRL and CRLSet. . . 20
6.2 Fraction of certificate types in OneCRL and CRLSet. . . 20
6.3 Percentage of valid, unknown and expired certificates in OneCRl and CRLSet. . . . 21
6.4 Top ten issuers in OneCRL and CRLSet. . . 21
6.5 Percentage of certificate uploading status in OneCRL. . . 22
6.6 Number of days from revoked to added in OneCRL. . . 23
6.7 Box and whiskers plot for certificate revocation reason and certificate type. . . 24
6.8 Box and whiskers plot for different certificate issuers. . . 25
6.9 Venn diagram for all unique certificates and issuers in OneCRL and CRLSet. . . 26
List of Tables
4.1 Selected domains from each block. Here, N is the number of entries in the top list. 10
5.1 Usage of OCSP and CRL . . . 13
5.2 Domains checked with CRL. . . 14
5.3 Number of domains offering OCSP staple and OCSP must-staple. Number of do-mains that have an EV certificate. . . 15
5.4 EV domains checked with OCSP. . . 16
6.1 Number of certificates in OneCRL and CRLSet. . . 18
6.2 Update frequency for OneCRL and CRLSet. . . 19
6.3 Number of respective revocation reason in OneCRL and CRLSet. . . 19
6.4 Number of different certificate types in OneCRL and CRLSet. . . 20
6.5 Number of valid and expired certificates in OneCRL and CRLSet. . . 21
6.6 Certificate uploading status in OneCRL. . . 22
6.7 Certificates that were added to OneCRL before being revoked. . . 23
6.8 Number of days between when a certificate was revoked to added in OneCRL. . . 23
1
Introduction
The web continues to grow as more of society is digitalized. Services and shops are getting transferred to the internet through websites. This puts high demands on security, but also user experience. To secure communication between web entities, the internet today is mostly used with the HTTPS protocol, which is an extension of the regular HTTP protocol with TLS. HTTPS includes certificate chains and encryption to provide authentication and integrity. The use of current methods to check whether these certificates have been revoked or not has grown to a large problem. That is because many of these revoked checking methods decrease performance on web browsers but at the same time, it is important to check that a certificate is revoked or not to maintain security by authentication.
HTTPS is built on a Public Key Infrastructure (PKI), which makes it possible to encrypt in-ternet traffic and verify servers’ identities. To verify that a client communicates with a server that belongs to the requested domain, TLS certificates are used. This increases security by preventing spoofing and other attacks. These certificates are issued by trusted CAs (certifi-cate authorities) who sign certifi(certifi-cates with their private key and distribute them to websites, which in turn can be trusted and verified with the websites’ public key. Certificates have an expiration date, but sometimes they need to be revoked by their corresponding CA before they get expired. For example, when a private key has been compromised. Then it is impor-tant that these revoked certificates get verified by the browsers. Otherwise, a client cannot successfully deny the connection from the server, exposing clients to attacks. The problem is that many of the methods that check for revoked certificates today causes delays in web browsers and reveals private information.
Some revocation methods choose only to cover a smaller set of revoked certificates to get some security but at the same time keep a good performance on the web browsers. However, there are methods that address many of these issues but do not prove to be widely used in practice.
This thesis evaluates the methods used for checking revoked certificates and how the methods affects performance and security at the client-side. The work answers the following questions:
• What are the advantages of methods checking for revoked certificates? • What are the disadvantages of methods checking for revoked certificates?
• What are the differences in how web browsers handle revoked certificates?
• Is there any revocation checking method that works well from both a performance and security viewpoint?
Contributions:This work provides an overview of some of the existing methods available
for authenticate revoked certificates. We show how the advantages and disadvantages of these methods affect the security and performance on the web. The work also provides a more transparent certificate landscape where we reveal how methods like OneCRL and CRLSet are used. We also show to what extent newer revocation checking methods such as OCSP staple and OCSP must-staple are applied.
Delimitations: We have chosen to limit ourselves to only investigating different uses of
revocation methods for three types of browsers: Firefox, Chrome and Safari. There was also a limitation by only examining the browsers own revocation lists: Chrome’s CRLSet and Firefox’s OneCRL. Due to our time limits we were only able to visit 3,503 domains.
Outline:The remainder of the thesis is structured as follows. Chapter 2 provides a
back-ground to the work. Chapter 3 includes an overview of other related works. In Chapter 4 we present the method and explanation of the used framework. Chapter 5 shows our results for OCSP and CRL. In Chapter 6 we show the results from OneCRL and CRLSet. In Chapter 7 we analyze the result and discuss the work as a whole. Lastly, in Chapter 8 we present our conclusion and future work.
2
Background
Transport Layer Security (TLS) is the most common technology on the Internet to keep com-munications secure and protect sensitive data sent between two systems [7]. This means that malicious entities cannot efficiently intercept the data. TLS together with a Public Key In-frastructure (PKI) makes it possible for a client (web browser) to verify that it really is the server (website) it wants to connect to [16]. In this way, TLS together with PKI contributes to confidentiality, integrity and authentication for secure online communication [5, 17]. To-gether, they are the cornerstones of Hypertext Transfer Protocol Secure (HTTPS) which is a secure version of the HTTP protocol (HTTPS can be interpreted as HTTP over TLS) [6, 13]. In this chapter, we present a background on TLS and PKI and its internal mechanisms that are interesting for our study.
2.1
Certificates
A certificate serves as proof that a website can be used to show that they are who they claim to be. This works in such a way that the certificate is signed by an issuer who binds a subject to its public key. On the web where HTTPS is used, these issuers are certificate authorities (CAs). CAs act as a trusted third party where both the subject (owner) of the certificate and the entity checking it trusts the CA. The most common form of certificate is the X.509 cer-tificate, this type contains extra information such as a serial number (which is unique within a CA), expiration date, the key’s cipher suite, and information to check if the certificate is revoked [5, 16]. When visiting a website over an HTTPS connection, the initial step is that the browser and the website negotiate about different security features the connection should have, this step is called the TLS handshake [17, 13]. One of the various security features that are transferred during this step is the certificate chain which is sent from the web page to the browser. This is can also be called the chain of trust and can be seen in Figure 2.1. The chain of trust works in such a way that CAs issuing root certificates that are self-signed. Issuing certificates directly to websites can be risky if their private keys gets compromised. That is because trusted root certificates are stored in the browser’s root stores, so it would mean that every certificate issued from a CA even with a compromised private key is automatically trusted. Therefore, you often have one or more intermediate certificates that are signed by the root certificate and thereby indirectly trusted.
2.2. Obtaining a certificate
Figure 2.1: The chain of trust.
These intermediate certificates can then issue a new certificate to an intermediate or leaf certificate (the leaf certificate is at the end of the chain and belonging to a website). So, the browser authenticates the certificate by verifying the signatures in the certificate chain up to the root certificate and checks if it exists within its root store. If the signatures are correct and the root certificate is included in the root store, the browser can conclude that the subject in the leaf certificate is the right owner of its public key and a connection can be initiated [23]. The browser also checks information in the certificate such as expiration date and revocation information [21, 5]. A certificate also contains an extension called Object Identifier (OID) which shows how the certificate has been validated. There is also a type of certificate called precertificate. These differ from the other types in that they serve as proof that a certificate has been logged for embedding Certificate Transparency (CT). Precertificates cannot be used for secure/encrypted connection and webserver authentication [4].
2.2
Obtaining a certificate
For a certificate to be valid, websites first need to apply for one at a CA. The website that needs a certificate generates a private key and sends it to a CA. This is done in a CA signing request (CSR) where the CA verifies that the website is the one it claims to be and if that is the case, it issues a valid certificate. This process has become increasingly automated through the creation of CA Let’s encrypt [9]. A common way to categorize certificates is based on the method that the CA uses to verify the applicant. The three most common methods are [9, 15]: • Domain validation (DV) which verifies that the domain applying for the certificate is indeed the rightful owner. This is the lowest form of verification. No inspection control is done by the company itself if one exists.
• Organization validation (OV) which verifies that the organization or company belongs to the entity applying for the certificate.
• Extended validation (EV) which verifies the organization like OV but has a higher level of control over the validation process. For example, many banks use this certificate to be able to really prove that they are who they claim to be.
2.3
Certificate revocation
When the browser checks the certificate in the TLS handshake mentioned in Section 2.1, it looks at the expiration date of the certificate, among other things. However, it sometimes happens that a certificate needs to be revoked before this date. It may be that a private key has been compromised or that a certificate is no longer valid [5, 11]. It is then important that there exist methods to check whether a certificate has been revoked or not, otherwise a client could visit a website that may have security risks. CA’s then have a responsibility to disseminate this information so that it can be controlled by clients who visit a website via
2.3. Certificate revocation
HTTPS. Some methods used to check for revoked certificates are CRL, OCSP, OCSP staple and OCSP must-staple, but browsers have also started to implement their own controls such as Chrome’s CRLSet and Firefox’s OneCRL [21].
Some common revocation reasons are KeyCompromise (private key has been comprim-ised), CACompromise (CA certificate is compromcomprim-ised), Superseded (the legal name of a user has changed) and CessationOfOperation (an issued certificate is replaced by another) [5].
It has been shown that revocation of certificates has been a major problem and some be-lieve that it has been broken for a long period of time [2, 21]. In this part, these different methods are described in a concise way for a greater understanding of our study.
CRL
A Certificate Revocation List (CRL) is a mechanism that lists all certificates that have been revoked before its expiration date for a specific CA. For each certificate in the list, there is a serial number (which is unique for a certificate in relation to its CA), revocation timestamp and revocation reason. Each CA updates this list at regular intervals and in the TLS’s hand-shake the browsers looks at the certificate if there exists a CRL Distribution Point (CDP) that links to the list and if so, downloads it and check if the current certificate is included. If the certificate is not in the list a safe connection can be established. As HTTPS becomes more common, the size of CRLs increases and downloading these lists causes a lot of network over-head. CRLs have an expiration date and therefore need to be updated at regular intervals, which means that the browsers cannot cache the list for longer periods [5]. Previous work [21] has shown that a CRL can be as large as 76 MB.
OCSP
To avoid downloading the entire list when you are only interested in checking if a specific certificate has been revoked and avoid the large network overhead associated with CRLs, the Online Certificate Status Protocol (OSCP) was developed. Similar to CRL, there is a URL in the certificate of an OCSP responder. An OCSP responder is a HTTP server which most CAs have. A browser can request this server to check if an individual certificate is revoked and then get a response to this request. The response is signed with a private key which proves that it was the CA who confirmed the request [18]. But it has been shown that this method has some performance and security flaws. The method requires an extra HTTP request for each website visited compared to CRLs, this added roundtrip slows down the browsers. If an OCSP server cannot be accessed, many browsers use soft-fail mode, which means that the certificate is considered valid even if the status is unknown. Also, OCSP makes browsers reveal private information to the CAs [21].
OCSP staple and OCSP must-staple
To reduce the shortcomings of OCSP, OCSP staple has been developed. Unlike OCSP, the web server sends a request and gets a response from the OSCP server. This response can be cached on a web server for up to 7 days. The web server then sends with this response in the TLS handshake and in this way the browser can verify whether the certificate has been revoked or not. This does not completely eliminate the performance issue as OCSP staple only applies to leaf certificates [22, 20] .
OCSP must-staple is an extension of OSCP staple that forces the OCSP staple method to be used to verify the certificate. A web browser will be rejected if it visits a web server with a certificate that has this extension without OCSP staple [8].
2.3. Certificate revocation
OneCRL and CRLSet
Due to the shortcomings mentioned above for CRL and OCSP, the browsers themselves have methods for checking revoked certificates. These methods involve collecting lists of revoked certificates from CAs and uploading them directly to the browsers. For example, Google Chrome relies on CRLSet to update a list of revoked certificates in their browser [1]. Like Google, Firefox has a list of revoked certificates called OneCRL. This list contains certificates that have been revoked by CAs in the Mozilla root program [12]. CRLSet and OneCRL have been shown to cover only a small part of the number of existing certificates [21].
3
Related Work
There are several studies in the field of checking revoked certificates. We have selected five different papers because they together contribute to our work with an overall picture of how the HTTPS landscape handles revocation. Each paper provides a new perspective on how the handling of revocation works out in the wild.
Liu et al. [21] presents a study that overviews the handling of revoked certificates in the webs’ PKI. It is based on three parties, web administrators (websites), CAs, and web clients (web browsers) and claims that all of them must contribute for getting revocation checking to function properly. To investigate how the different parties meet their revocation responsibilities they collected a large data set of certificates. They also looked at how web browsers check if a certificate is revoked or not. The results from their empirical study shows that over 8 of the collected certificates are revoked and that web browsers in most cases do not verify the revocation status of a certificate. Their conclusion is that many web browsers prefer performance in front of security and that methods that treats the performance issue is not widely used in practice.
Aas et al. [9] explain how they developed an automated central authority (CA) named Let’s encrypt. It was launched in 2015 and has today grown into the largest HTTPS CA in the world. Automation allows it to issue certificates for free and at the same time, the webserver authorities do not have to perform the actual configuration by themselves. These certificates signed by Let’s encrypt have a lifespan that extends up to 90 days, which they believe increases security as you are not as dependent on revoking certificates when, for example, a private key has been compromised. The paper also shows that Let’s encrypt only covers DV certificates and has in many ways contributed to making it easier for websites to use HTTPS.
Another study by Duremeric et al. [23] focuses on the entire HTTPS ecosystem. By col-lecting a large data set, they were able to investigate several aspects of the ecosystem. Among other things, they looked more at which organizations were involved in the process of issuing certificates and which keys and signatures were used to sign certificates. In this large compre-hensive study, it was found that the distribution of intermediate certificates is very skewed, for example, it turned out that the disclosure of one intermediate certificate would require 26 HTTPS websites to obtain a new certificate. It was also found that that approximately half of all trusted certificates use a 1024-bit RSA key in their trust chain, which according to their conclusions makes entities more vulnerable to attacks.
A study made by Korzhitskii and Carlsson [14] examines the transparency of revoked certificates in the Web PKI. This was done by checking revocation rates on the internet and how revocation status management looks for different CAs. The study showed that many CAs remove revocation very early after the expiration date of the certificate. It also reveals how revocation takes place differently within and among other CAs. It makes them conclude that there should exist a common revocation transparency standard.
A study to analyzing how the patterns for revocation and reissuing of SSL certificates were performed under the heartbleed vulnerability was done by Zhang et al [11]. They found that the majority of the vulnerable certificates had not been reissued and that those domains who reissued certificates did not revoke. This, they say, leads to many websites during the period the study was conducted being vulnerable to attacks when malicious third parties could use stolen keys.
What makes our work unique and different from the five above-mentioned papers is, among other things, that we examine at a deeper level the various advantages and disad-vantages of web browsers own methods for checking revoked certificates with OneCRL and CRLSet. We also compare both of these methods against each other and show patterns, sim-ilarities and differences. Our work also differs from the others in that way we also compare methods such as OCSP, CRL, OCSP staple and OCSP must-staple against each other and re-veal, among other things, properties such as performance of the method OCSP. Where the other works have provided a more holistic view of the certificate landscape, our work fo-cuses on how end-users are affected by the handling of checking for revoked certificates. We also show how transparent OneCRL and CRLSet are by their handling of revoked certificates, which none of the mentioned works have presented.
4
Method
In this chapter, we describe our method that was being used to analyze different approaches to check for revoked certificates. The method is divided up in four parts where we first begin by explaining how we collected data for checking usage of OCSP and CRL, secondly we describe the way we analyzed page load performance for websites using OCSP, thirdly we tell about the way we collected certificate information from the websites, lastly we mention how we collected data from the browser’s own lists of revoked certificates.
4.1
Adoption of OCSP and CRL
In order to collect information to what extent OCSP and CRL is applied for three different browsers, we used a tool that was written in the Python programming language (version 3.9.2) and uses Selenium WebDriver (version 4.0.0)1 to automatize visits on websites. We added Selenium Wire2, which is an extension to Selenium WebDriver that makes it possible to collect underlying HTTP traffic. In that way it was possible to notice if a OCSP or CRL request/response was sent between the client and the web server.
This test was conducted on a laptop (MacBook Pro 2017) with MacOS version 11.2.2 and 2,3 GHz Dual-Core Intel Core i5 processor. Based on five different types of top lists with domains, 3,503 were selected and visited with Selenium WebDriver. These lists were Alexa top 1 M3, Tranco top list4, Umbrella top list5, and Alexa top 50 categories (from May 18 2020, was retired September 21st 2020)6. Together they had many duplicates, so we did a preprocessing phase where we removed domains that occurred more than once. One priority we had was that Alexa top 50 categories should have with all its domains so we also could analyze potential differences in behaviors for different categories. As Alexa top 1M, Tranco top list and Umbrella top list individually contain up to one million domains, we divided these lists into 5 blocks: 1-250, 751-1,000, 9,751-10,000, 99,751-10000, and (N-250)-N, where N is the number of domains in the list. This is shown in Table 4.1.
1https://www.selenium.dev 2https://pypi.org/project/selenium-wire/ 3https://www.alexa.com/topsites 4https://tranco-list.eu 5https://umbrella.cisco.com/blog/cisco-umbrella-1-million 6https://www.alexa.com/topsites/category
4.2. OCSP page load performance
Block 1 2 3 4 5
Sites 1-250 751-1,000 9,751-10,000 99,751-100,000 (N-250)-N
Table 4.1: Selected domains from each block. Here, N is the number of entries in the top list. With this selection, all lists together made up to 6,000 domains, after removing duplicates we ended up with 3,503 domains.
These domains were visited with three types of web browsers using Selenium WebDriver. The browsers used were, Chrome (version 90.0.4430.93), Safari (version 14.0.3), and Firefox (version 88.0) to observe the usage of OCSP and CRL. The test was performed in such a way that the browser was closed down after each visit to a website, in such way the cache was cleared and the order in which we visited the websites did not affect the result. All HTTP requests and responses were saved in a csv-file along with the associated domain and list. In addition, we also saved information in separate csv-files that contained HTTP requests and responses with OCSP or CRL associated with their domain and list.
4.2
OCSP page load performance
To measure the extent to which OCSP affects the page load performance on the web browsers, we did a test where OCSP checks were switched on and off. The same procedure was per-formed as in Section 4.1, except the differences that we used Selenium Web driver without Selenium Wire and only analyzed the Firefox browser. Firefox is the only browser that has the settings to turn on and off OCSP checks. To be able to turn off OCSP checks that are on by default, Selenium WebDriver has a built-in function that allows you to use pre-selected Fire-fox profiles. That is not possible with Selenium Wire, so the csv-file from when we checked the usage of OCSP and CRL with Firefox was adapted in this test. By doing so, we could know which websites were using OCSP. Therefore, a profile was created on Firefox that had OCSP checks disabled and the test was done without Selenium Wire.
To measure the time taken for each website to load, we used a JavaScript Navigation Tim-ing API7together with Selenium. There are several different timestamps of this API, the ones we were interested in were, navigation start, response start and DOM complete. Navigation start is the time it takes for a user-agent to unload a previous website or document. Response start is the time it takes for a user-agent to get the first byte from the webserver. DOM com-plete is the time it takes in our case to load all resources on the website. In this way, we could measure backend performance by calculating response start ´ navigation start and frontend performance by taking DOM complete ´ response start.
Then we run two sessions without Selenium Wire, where one had OCSP checks on and the other off. These two sessions were run individually for two consecutive weekdays. Each session was also divided into five different blocks so that we visited each website at about the same time of the day. This is to reduce several different factors that can affect the loading time, such as varying visitor numbers on the websites during different periods. After these sessions information was logged into csv-files where the domain, list, ranking and the specified times were noted for each website.
4.3
Website certificate information
To get information from the domains visited in Section 4.1 and Section 4.2 a free open-source command-line tool was used (testssl.sh)8. This tool helped retrieve interesting information from the web servers such as certificates, cryptographic flaws and different used protocols. In our case, we were interested in taking a closer look at the certificate information whether they support OCSP staple or OCSP must-staple. We also checked if the certificate of the
7https://www.w3.org/TR/2021/WD-navigation-timing-2-20210308/ 8https://testssl.sh
4.4. CRLSet and OneCRL
webserver was an EV or not. A list of all domains was written into testssl via a bash-script where a restriction was placed to only receive information about the web server’s certificate. To increase the efficiency, we divided the domains into ten parts. We then ran the script concurrently with testssl. When the script was finished the collected data was logged into a csv-file to help us analyze the information gather in Section 4.1 and Section 4.2.
4.4
CRLSet and OneCRL
In this part, we collected data from the browsers’ own lists of revoked certificates. The col-lection was made to analyze how effective, safe and practical these stored revoked certificate lists are. The browsers chosen were amongst the most popular, Google Chrome and Mozilla Firefox with their respective revocation lists CRLSet9and OneCRL10[19]. This collection was made on a laptop with Windows 10 and 3.30 GHz Quad-Core Intel Core i5 6600 processor. The laptop used a virtual machine with the operating system Ubuntu 20.04.
The collection for CRLSet was attained by a software called CRLSet-Tools 11. By using the programming language GO, the software CRLSet-tools could be run at the terminal to download the current list of revoked certificates.
To measure the update frequency, a repeating command was executed into the terminal by a bash script that downloaded the list every other hour in one week. In this way, we could compare the downloaded lists in a Python program and determine if there have been any changes to the list. If a change was noted, we could conclude that an update had been taken place. The output went into text files that stored all the retained data.
The collection for OneCRL was attained by downloading the list from the web page crt.sh12that store information about certificates. To save the data in the lists, the HTTP code was downloaded with the software Curl13. In the same way as with CRLSet, everything was done with a terminal-looped command with the output of the program placed in text files for analysis. The terminal loop had the same execution frequency as the aforementioned CRLSet list every other hour in one week. Unlike CRLSet, OneCRL has a date when it was last updated. By extracting this date, we could find out the update frequency for the list.
Once the data from both lists were gathered it still had to be processed. CRLset and OneCRL had to be refined before data extraction. The main goal was to get as much certifi-cate information as possible from the collected data. From the lists, OneCRL identified each certificate by a unique ID number and in CRLSet a certificate was identified by its serial num-ber. This extracted identifier was then applied for search at crt.sh which further showed the specified certificate’s information. This was done with all unique certificates in both lists. For each certificate, we downloaded all available HTTP data. This was done by making a Python program that printed the code for a bash script. The script was run through the terminal for an extensive Curl downloading session during which all certificate data for our specific cer-tificates was downloaded. From the HTTP data, we then extracted all the possible certificate information such as revocation date, subject, issuer and so on.
However, some certificates in CRLSet had serial numbers which were not unique, mean-ing that they were linked to two or more certificates. Also, it existed serial numbers that were not linked to any certificate. Therefore, we excluded these certificates. In total, the ac-quired certificate data was from 1983 certificates, 712 of which was from CRLSet and 1271 from OneCRL. From the HTTP data, we then extracted all the possible certificate information such as revocation date, subject, issuer and so on.
Furthermore, to get a better scope of what kind of certificate validation types that were in use, we looked for different Object identifier (OID) numbers. The OID number could only
9https://dev.chromium.org/Home/chromium-security/crlsets
10https://blog.mozilla.org/security/2015/03/03/revoking-intermediate-certificates-introducing-onecrl/ 11https://github.com/agl/crlset-tools
12https://crt.sh 13https://curl.se
4.4. CRLSet and OneCRL
be found in leaf certificates and since only OneCRL had leaf certificates that was where we looked for information. After extracting all the OID information, we manually looked up what certificate validation types the OID represented at two different web pages14,15.
14http://oid-info.com/ 15https://oidref.com/
5
OCSP and CRL Results
In this chapter, we present our results from analyzes of the revocation methods OCSP and CRL as well as the website’s certificate information. The results are based on the tests ex-plained in the Sections 4.1, 4.2 and 4.3. The Chapter begins with showing the results for how large the use of OCSP and CRL is from the browsers. Then the result is presented for how much OCSP affects the page loading time on the websites. We also emphasize whether the various websites implement revocation mechanisms such as OCSP staple and must-staple. Finally, we present whether the websites hold an EV certificate and if it is performed more OCSP checks on these.
5.1
Adoption of OCSP and CRL
Table 5.1 shows that Firefox is the only one of the three browsers that use OCSP and that the number of web pages that had certificates checked by OCSP was 2,449 in Firefox. It is also possible to see that there were very few websites that had their certificate checked with CRL, the number for Firefox and Chrome was 4 and 5 websites, respectively. In Table 5.1 it can be noted that Safari did not perform any control at all with either OCSP or CRL on any website.
Web browser Number of domains
Total OCSP CRL
Firefox 3,503 2,449 4
Chrome 3,503 0 5
Safari 3,503 0 0
Table 5.1: Usage of OCSP and CRL
Figure 5.1 shows the fraction of websites where either OCSP or CRL was used. We see that approximately 70% of the total number of websites visited with Firefox had certificates checked with OCSP.Firefox prioritizes the security for its users because of the frequent use of OCSP. It is also possible to see the percentage of websites checked with CRL only was 0.1% for Firefox and 0.2% for Chrome. It is interesting to see how CRL, which was among the first methods developed to check for revoked certificates, is now beginning to phase out and
5.2. OCSP page load performance
almost no longer being used at all. It is also possible to see in Figure 5.1 how OCSP usage has been abandoned. Both Chrome and Safari do not use OCSP to control certificates.
Figure 5.1: Use in percentage of OCSP and CRL.
Table 5.2 shows the domains that had their certificates checked by CRL. It can be noted that there are six unique websites. The domains exploring-greece.org, drupal.org, usm.edu were controlled by both browsers Firefox and Chrome. We also see that tureng.com was only controlled by Firefox and that nejm.org together with digitimes.com were only controlled by Chrome. As we see in Table 5.2, all domains have unique issuers that make it impossible to find any pattern that shows that these few CRL requests are linked to a specific issuer. What is interesting is that the domain drupal.org is a precertificate that cannot be used for webserver authentication. We also see that the other domains are of the certificate type leaf, which is expected as these belong to websites that are at the bottom of the certificate chain.
Web browser Websites with CRL
Domain Issuer Certificate type
Firefox exploring-greece.org Sectigo Limited Leaf
Firefox drupal.org GlobalSign nv-sa Precertificate
Firefox usm.edu Internet2 Leaf
Firefox tureng.com Cloudflare, Inc. Leaf
Chrome exploring-greece.org Sectigo RSA Domain Validation Secure Server CA Leaf Chrome drupal.org Sectigo RSA Domain Validation Secure Server CA Precertificate
Chrome usm.edu Internet2 Leaf
Chrome nejm.org DigiCert Inc Leaf
Chrome digitimes.com Sectigo Limited Leaf
Table 5.2: Domains checked with CRL.
5.2
OCSP page load performance
Figure 5.2 shows that with OCSP turned on, the average time to load the backend of the website took approximately 1.18 seconds. With OCSP turned off, it is possible to note a small decrease in the average loading time for the backend, which took 1.13 seconds. This leads to a time difference of 50 milliseconds with OCSP turned on. Therefore, we can see that the loading time is not affected to any large extent.
Figure 5.2 also shows that the average loading time for the frontend with OCSP turned on took approximately 1.99 seconds. It is also possible to note that with OCSP turned off, the average load time for the frontend took approximately 2.03 seconds. This means that the time difference between OCSP turned on and off for the frontend was approximately 40 milliseconds. So, the average loading time for the frontend with OCSP turned off took a longer time then the one with OCSP turned on.
If we look at the total loading time in Figure 5.2, it can be noted that the total loading time for both frontend and backend took about the same amount of time with 3,17 seconds for OCSP turned on and 3.16 seconds for OCSP turned off. The results show that OCSP
5.3. Webserver implementations and OCSP checks for EV certifcates
in principle has no effect at all on the total average loading time for a website as the time difference is only 10 milliseconds.
Figure 5.2: Page load time with OCSP on and off.
5.3
Webserver implementations and OCSP checks for EV certifcates
Table 5.3 shows that 83 domains use an EV certificate to confirm their validity. We also see that 1,075 of the visited domains offer OCSP staple to check if a certificate has been revoked. But we can see that more domains did not offer OCSP staple as the number was counted to 1,482. In Table 5.3 it can also be noted that none of the domains offered OCSP-must staple. As OCSP must-staple is a very effective method for checking revoked certificates from both a performance and security perspective, it is remarkable that no domain implements this method. For the other 946 domains, it can be deduced that implementation of either OCSP staple, OCSP must-staple and EV certificate is unknown.
Number of domains
Total EV Staple offered Staple not offered Must-staple offered Must-staple not offered Unknown
3,503 83 1,075 1,482 0 2,557 946
Table 5.3: Number of domains offering OCSP staple and OCSP must-staple. Number of domains that have an EV certificate.
In Figure 5.3a, it is possible to see that the fraction of domains that offer OCSP staple is 30.7%, compared to 42.3%, which is the fraction that did not offer OCSP staple. Given the low usage of OCSP from the browsers presented in Section 5.1, it is interesting that more domains do not offer OCSP staple than domains that offer OCSP staple. We can also see in Figure 5.3b that 73% of the domains do not offer OCSP must-staple.
(a) Percentage OCSP staple. (b) Percentage OCSP must-staple. Figure 5.3: Percentage of OCSP staple and OCSP must-staple.
5.3. Webserver implementations and OCSP checks for EV certifcates
The fact that as many as 27% of the domains did not show any information about the implementation of these methods makes it possible that there are more that offer OCSP staple and OCSP must-staple. Regardless, there are a lot of domains that have not yet taken the opportunity to offer these methods.
Table 5.4 shows how many of the domains that had an EV certificate and were checked with OCSP by the Firefox browser. The number of OCSP checks for domains with EV cer-tificates was 54 and the number of domains with EV cercer-tificates for which an OCSP check was not performed was 29. In Figure 5.4 we can see that the fraction of OCSP checks that were performed on EV certificates was 65.1%. From Figure 5.1, we can see that this num-ber is close to 69.9% that was the total numnum-ber of domains that had an OCSP check with the Firefox browser. Therefore,because of the almost same share of OCSP checks, we can see that there is no major correlation with performing more OCSP checks for domains that have an EV certificate.
Web browser Number of domains
Total OCSP checks for EV No OCSP checks for EV
Firefox 83 54 29
Table 5.4: EV domains checked with OCSP.
Figure 5.4: Percentage OCSP checks for EV certificates.
Figure 5.5 shows the fraction of issuers who had issued an EV certificate for the 29 do-mains shown in Table 5.4 for which an OCSP check had not been performed. We can see that a large majority of these certificates had the issuer Digicert Inc which had a proportion of almost 60%. It can also be seen that Entrust Inc and Sectigo Limited have an equal share of 13.8% and that GlobalSign nv-sa has a share of 6.9%. In addition to this, it can be noted that Cybertrust Japan Co. Ltd and COMODO CA limited have a smaller fraction of 3.5%.
5.3. Webserver implementations and OCSP checks for EV certifcates
6
OneCRL and CRLSet Results
In this chapter, we present our results from analyzing Firefox and Chrome’s own lists of revoked certificates, OneCRL and CRLSet. The results are based on the tests explained in Section 4.4.
The Chapter begins with showing the results from how large the both lists are. Then we present the update frequency which shows how often these lists are updated. Sections 6.3 - 6.6 presents the results of the characteristics of OneCRL and CRLSet, such as the reason for the revocation of certificates, certificate types, certificate status and certificate issuers. Section 6.7 presents the results for how long it takes for a revoked certificate to be uploaded to OneCRL and whether the time varies depending on the type of certificate, the reason for revocation and the issuer of the certificate. Section 6.8 presents the certificate intersections from OneCRL and CRLSet. Lastly, the certificate validation types are shown in Section 6.9.
6.1
Size
Table 6.1 shows the maximum, minimum, average and median number of certificates in the two revocation lists OneCRL and CRLSet during a period of one week. It gives an indication of the size of both lists. By comparing these metrics for certificates from both OneCRL and CRLSet, we can see that during the one-week period the measurement was conducted that OneCRL had a constant size of 1,294 certificates. We can also see that CRLSet had a maximum of 961 certificates and a minimum of 953 certificates. It is also possible to notice that the size of the OneCRL list is larger than the CRLSet list size in the number of certificates. The median value for CRLSet is 953 which is the same as the minimum value, it reveals that during most of the time the list was downloaded it had a constant value at 953 certificates which reflects the same behavior as OneCRL.
Revocation list Number of certificates
Max Min Average Median
OneCRL 1,294 1,294 1,294 1,294
CRLSet 961 953 955.3 953
6.2. Update frequency
6.2
Update frequency
How often the lists of OneCRL and CRLSet were updated is shown in Table 6.2 in the value of hours represented in maximum, minimum, average, and medium time. During the one-week measurement, it shows the difference between the two lists in update frequency. It is very clear that OneCRL has a higher update frequency than CRLSet. OneCRL has an average update frequency of 2 hours compared to CRLSet that has an average update frequency of 12 hours. We can also see that CRLSet has some periods when the list is not updated for 14 hours compared to OneCRL that has a maximum when not updated of 5 hours. The lowest time between two updates for CRLSet is 10 hours compared to OneCRL that has a minimum of 2 hours. It reveals that OneCRL keeps their list more up to date than CRLSet does.
Revocation list Number of hours
Max Min Average Median
OneCRL 5 2 2 2
CRLSet 14 10 12 12
Table 6.2: Update frequency for OneCRL and CRLSet.
6.3
Revocation reason
In Table 6.3 we can see what type of revocation reason the certificates have in OneCRL and CRLSet. As shown in Figure 6.1a we can see that the majority of certificates in OneCRL have a revocation reason set as cessationOfOperation. We can also see that about a third of the certificates in OneCRL has a revocation reason set as unknown and that around a fifth of the certificates has a revocation reason set as superseded. In addition, it can be noted that the revocation reasons affiliationChanged, keyCompromise, certificateHold and CAcompromise individually constitute of less than 1% of all certificates in OneCRL. It is interesting to see that 1.65% of the certificates have not been revoked by their issuer and that OneCRL has chosen to include these in the list anyway. We can also see that 1.8% of the certificates have the reason for revocation set as expired which is an odd behavior as these certificates still will be denied anyway with the use of TLS.
Figure 6.1b shows that almost half of the certificates in CRLSet have a revocation reason set as cessationOfOperation. We can also notice that almost a third of the certificates have a revocation reason set as superseded and around a sixth of the certificates has a revocation reason set as unknown. In addition to cessationOfOperation and superseeded, CRLSet has only one more revocation reason which is certificateHold and makes up only 0.3% of all certificates.
By comparing OneCRL and CRLSet, it is possible to see differences in the number of cer-tificates that are set to unknown. More cercer-tificates that do not have a revocation reason set as unknown increases the transparency. As these reasons are set by the CA of the certificate, that responsibility is not on OneCRL and CRLSet. One more difference we can see is that OneCRL has significantly more revocation reasons than CRLSet. Reasons such as CAcompromise and keyCompomise are serious reasons and it is strange why CRLSet has not chosen to include more certificates with these kinds of revocation reasons.
Revocation list Number of certificates
Total Superseded CessationOfOperation Unknown affilationChanged notRevoked keyComprimise Expired certificateHold CAcomprimise
OneCRL 1,271 288 582 348 3 21 3 23 1 2
CRLSet 712 234 363 113 0 0 0 0 2 0
6.4. Certificate types
(a) Percentage revocation reason in OneCRL. (b) Percentage revocation reason in CRLSet. Figure 6.1: Fraction of revoked certificate reason in OneCRL and CRLSet.
6.4
Certificate types
Table 6.4 shows the number of different certificate types that occur in OneCRL and CRLSet. By looking at Figure 6.2a, it shows how large a fraction of the total number of certificates in OneCRL that are intermediate. It also possible to notice that 0.7% are root certificates, 1.7% are precertificates and 5.4% are leaf certificates. Figure 6.2b shows that all certificates in CRLSet are intermediate certificates. Both OneCRL and CRLSet have a large fraction of intermediate certificates. OneCRL has for a long period of time since it was launched only added intermediate certificates, we can now see that it has extended this policy and added more certificate types as leaf, root and precertificates.
Revocation list Number of certificates
Total Intermediate Leaf Root Precertificate
OneCRL 1,271 1,171 69 9 22
CRLSet 712 712 0 0 0
Table 6.4: Number of different certificate types in OneCRL and CRLSet.
(a) Certificate types in OneCRL. (b) Certificate types in CRLSet. Figure 6.2: Fraction of certificate types in OneCRL and CRLSet.
6.5
Certificate status
Table 6.5 shows the number of certificates that have expired, still are valid and miss infor-mation on validity dates. It can be seen that the number of expired certificates in OneCRL is
6.6. Certificate issuers
447 compared to the CRLSet which only has 3 expired certificates. At the same time, it shows that the number of valid certificates is about the same with a little more in OneCRL where the number is 713 certificates compared to CRLSet’s 707 valid certificates. Only 2 certificates have an unknown expiration date in the CRLSet compared to 111 in the OneCRL. In Figure 6.3a, it can be seen that more than half of all certificates in OneCRL still are valid, but also that as much as 35.2 % of the certificates have expired. Such a large number of certificates that have expired in OneCRL is strange as checks do not need to be made when they have already expired. In CRLSet, Figure 6.3b shows that only 0.4 % of the certificates have expired and 99.3 % are still valid. Unlike OneCRL, CRLSet has a list that is more useful due to the large proportion of still valid certificates.
Revocation list Number of certificates
Total Expired Valid Unknown
OneCRL 1,271 447 713 111
CRLSet 712 3 707 2
Table 6.5: Number of valid and expired certificates in OneCRL and CRLSet.
(a) Certificate status in OneCRL. (b) Certificate status in CRLSet. Figure 6.3: Percentage of valid, unknown and expired certificates in OneCRl and CRLSet.
6.6
Certificate issuers
Figure 6.4a and Figure 6.4b, respectively, present results for the top 10 CAs that are linked to a certificate in OneCRL and CRLSet. Figure 6.4a shows that Verisign.Inc, Baltimore, GlobalSignnv-sa and QuoVadistLimited are the ones that have most certificates in OneCRL.
(a) Number of certificates per issuer in OneCRL. (b) Number of certificates per issuer in CRLSet. Figure 6.4: Top ten issuers in OneCRL and CRLSet.
6.7. Date when upload to OneCRL
It can also be seen that a large proportion have unknown issuers. Figure 6.4b shows that SymantecCorpration, GlobalSignnv-sa and HellenicAcademic.Authority have the vast ma-jority of the certificates in CRLSet. Unlike OneCRL, most certificates in CRLSet have a known issuer.
6.7
Date when upload to OneCRL
Table 6.6 shows how many certificates that both have a revocation date and an uploading date to OneCRL, these ones are labeled as known, certificates that only have a revocation date or an uploading date to OneCRL is labeled as unknown. It also shows the number of certificates that have been added to OneCRL before being revoked. In Figure 6.5 it is possible to see that the number of certificates with both a known revocation date and uploading date to OneCRL is more than half of the total number of certificates. Whereas almost half of the certificates miss either the revocation or uploading date makes the list less transparent.
Revocation list Number of certificates
Total Known Unknown Added to OneCRL before revoked
OneCRL 1,271 726 545 20
Table 6.6: Certificate uploading status in OneCRL.
Figure 6.5: Percentage of certificate uploading status in OneCRL.
In Table 6.7 we can see the 20 certificates that were added to OneCRL before it had been revoked and were shown in Table 6.6. We can observe that certificate 4, 7, 8, 14, 18 and 20 are self-signed intermediate certificates. It can be noted that all certificates in Table 6.7 are of the type intermediate. We can also see that there are no serious revocation reasons where the reason cessationOfOperation dominates.
6.7. Date when upload to OneCRL
Certificate Added to OneCRL before revoked
Issuer Subject Certificate type Revocation reason
1 Cybertrust,Inc Unknown Intermediate certificate cessationOfOperation 2 Certinomis StartComCA Intermediate certificate unknown 3 Cybertrust,Inc Unknown Intermediate certificate cessationOfOperation 4 AgenciaCatalanadeCertificacio AgenciaCatalanadeCertificacio Intermediate certificate cessationOfOperation 5 Baltimore Siemens Intermediate certificate cessationOfOperation 6 Baltimore Bancad’Italia Intermediate certificate cessationOfOperation 7 DeutscheTelekomAG DeutscheTelekomAG Intermediate certificate superseded 8 AgenciaCatalanadeCertificacio AgenciaCatalanadeCertificacio Intermediate certificate cessationOfOperation 9 Unknown FirmaprofesionalS.A. Intermediate certificate unknown 10 Certinomis StartComCA Intermediate certificate unknown 11 Unknown FirmaprofesionalS.A. Intermediate certificate unknown 12 Baltimore Siemens Intermediate certificate cessationOfOperation 13 Baltimore TelecomItaliaTrustTechnologiesS.r.l Intermediate certificate cessationOfOperation 14 AgenciaCatalanadeCertificacio AgenciaCatalanadeCertificacio Intermediate certificate cessationOfOperation 15 Unknown BancoSantanderS.A. Intermediate certificate unknown 16 IdenTrust U.S.Government Intermediate certificate cessationOfOperation 17 Baltimore TelecomItaliaTrustTechnologiesS.r.l. Intermediate certificate cessationOfOperation 18 AgenciaCatalanadeCertificacio AgenciaCatalanadeCertificacio Intermediate certificate Expired 19 Cybertrust,Inc Unknown Intermediate certificate cessationOfOperation 20 JapanCertificationServices,Inc. JapanCertificationServices,Inc. Intermediate certificate cessationOfOperation
Table 6.7: Certificates that were added to OneCRL before being revoked.
With the known dates from Table 6.6 it is possible to see in Table 6.8 the number of days between when a certificate was revoked until it was uploaded to OneCRL. It is possible to see that the maximum number of days from when a certificate was revoked until it was uploaded at OneCRL is 3,709 days, the minimum number of days is 0 days, the average number of days is 346 and the median number of days is 56. The number of 3,709 days stands out when it is a long time after the certificate was revoked until it was uploaded on OneCRL which makes end-users vulnerable to attacks.
Revocation list Number of days to added in OneCRL
Max Min Average Median
OneCRL 3,709 0 346 56
Table 6.8: Number of days between when a certificate was revoked to added in OneCRL. Figure 6.6a shows the empirical distribution function for the number of days it takes for certificates to be added to OneCRL after being revoked. It is possible to notice that approx-imately 60% of the certificates have a value under 100 days from revoked to uploaded on OneCRL. It is critical that certificates that has been revoked uploads as fast as possible to OneCRL so that users do not enter websites with security risks. Here we can see that this is not always the case which creates severe vulnerabilities to end-users.
Figure 6.6b shows a complementary cumulative distribution function (CCDF) for the number of days it takes for certificates to be added to OneCRL after being revoked.
(a) CDF for number of days. (b) CCDF for number of days. Figure 6.6: Number of days from revoked to added in OneCRL.
6.7. Date when upload to OneCRL
We can see that small and medium-sized values take up a large part of the distribution. At the same time, it is observable that the CCDF is heavy-tailed as there are very few large values.
Figure 6.7 shows a box and whiskers plot for the number of days it takes for a certificate to be uploaded to OneCRL after being revoked for different revocation reasons and certificate types (note that the y-axis is in log scale).
It is possible to see quite a similar handling for both revocation reasons ation and superseded, 25% of the certificates with revocation reason set as cessationOfOper- cessationOfOper-ation are uploaded within 20 days after being revoked and 25% with revoccessationOfOper-ation reason set as superseded is uploaded within about 25 days after being revoked. cessationOfOperation has a median of 60 days compared to superseeded which has a median of 51 days. However, it can be seen the difference that 75% of the certificates are uploaded faster than 105 days after being revoked for cessationOfOperation compared to about 160 days for superseded. At the same time, we can see that cessationOfOperation has many more outliers than superseded, but as we can see in Figure 6.1a, cessationOfOperation is more common than superseded. Both reasons have outliers with uploading times up to a few years. On the other hand, there are individual certificates that are uploaded just a couple of days after they were revoked.
The handling of the certificate types pre and leaf differs from intermediate in several ways. We see that the median for intermediate is 68 days, unlike leaf and pre, which have 6.5 days and 5 days, respectively. We can see that intermediate certificates have 75% of the certificates uploaded faster than 130 days compared to pre and leaf which have 75% of the certificates uploaded faster than 20 and 11 days, respectively. It can also be noted that intermediate has 25% of the certificates uploaded faster than 20 days, leaf has 25% of the certificates uploaded under 1 day and that pre has 25% of the certificates uploaded on the same day as being revoked. Intermediates also have more outliers, but as we can see in Section 6.4, it is by far the most common type of certificate. These outliers can stretch up to a few years, but we also see that intermediates have individual certificates that are uploaded after a few days after it has been revoked. Pre and leaf just have a few outliers around 200 days.
Figure 6.7: Box and whiskers plot for certificate revocation reason and certificate type. Figure 6.8 shows a box and whiskers plot for the number of days it takes for a certifi-cate to be uploaded to OneCRL for the top 10 certificertifi-cate issuers (note that the y-axis is in log scale). These are the same issuers that can be seen in Figure 6.4a and are the 10 most com-mon issuers in OneCRL. It is possible to note several differences in how fast the certificate is uploaded between the issuers. We see that DigiCertInc and SymantecCorporation have their certificates uploaded in a very narrow range, where for example DigiCertInc has a range
6.8. Certificate intersections
between 8 and approximately 11 days and a median of 8 days. It can be noted that all is-suers except GlobalSignv-sa have 25% of their certificates uploaded to OneCRL faster than 100 days. GlobalSignv-sa is also the issuer with the largest median by a large margin. All issuers except DigicertInc and QuoVadisLimited have a median value between 100 and 10 days. We also see that only StaatderNederlanden, DigiCertInc and Baltimore have individ-ual certificates uploaded faster than 10 days. Furthermore, Balitmore is the only issuer with an individual certificate uploaded after 1 day. It is also interesting to see that some issuers like DigicertInc, StaatderNederlanden and SymantecCorporation have the upper or lower quartile equal to the median. This can mean that many certificates with a specific issuer are uploaded with a similar time interval or that it has been an occasion where several certificates have been revoked at the same time. Interesting to see is also that Globalsign has no outlier compared to Baltimore and Verisign,Inc which have several that can take up to a few years before being uploaded to OneCRL. Significantly, none of the top 10 issuers have certificates that are uploaded the same day as they are revoked. Overall, we can see that there are a few issuers with a narrower interval which can lead to more predictability for how fast certificates for a specific issuer are handled compared to the others with a larger interval.
Figure 6.8: Box and whiskers plot for different certificate issuers.
6.8
Certificate intersections
Figure 6.9a shows a venn diagram for all certificates that could be identified in both OneCRL and CRLSet. It is possible to see that 474 certificates are available in both OneCRL and CRLSet. This means that a very large fraction of all the certificates in CRLSet also exists in OneCRL. It can also be noted that 797 unique certificates only exist in OneCRL and 238 unique certificates only exist in CRLSet.
Figure 6.9b presents a venn diagram for the number of unique certificate issuers in OneCRL and CRLSet. A large number of unique certificate issuers exist in both lists, the number amounts to 64. 36 unique certificate issuers exist only in OneCRL and 13 unique cer-tificate issuers exist only in CRLSet. Given that there are more unique cercer-tificates in OneCRL than CRLSet, it is reasonable that there are more certificate issuers that only exist in OneCRL.
6.9. Certificate validation types
(a) Unique certificates in OneCRL. (b) Unique issuers in OneCRL. Figure 6.9: Venn diagram for all unique certificates and issuers in OneCRL and CRLSet.
6.9
Certificate validation types
Table 6.9 shows different validation types of leaf certificates in OneCRL. We see that the total number of leaf certificates is 69 and 11 of them are OV certificates, 40 DV certificates and 19 are unknown. It is also possible to note that none of the leaf certificates in OneCRL were EV.
From Figure 6.10, it is possible to note that 58% of all leaf certificates in OneCRL are DV certificates, 15.9% are OV certificates and that 26.1% are unknown. We can see that DV cer-tificates are the most dominant validation type of the cercer-tificates in OneCRL.
Certificate type Number of certificates
Total OV DV EV Unknown
Leaf 69 11 40 0 18
Table 6.9: Validation types in OneCRL.
7
Discussion
In this chapter we will present thoughts on the result, the method and the work in a wider context.
7.1
Results
Here we discuss interesting findings from the results. First, we discuss the OCSP and CRL results, secondly the OneCRL and CRLSet results and lastly both results combined.
OCSP and CRL Results
In Section 5.1 we could see that of the three browsers Firefox, Chrome and Safari, only Firefox used OCSP. In Section 5.2 we saw that OCSP has no major impact on page performance, it is interesting why Chrome and Safari do not use the method to check for revoked certificates. OCSP’s disadvantages were mentioned, among other things, being that it reveals private information from the browsers to CA’s and that OCSP’s servers are sometime hard to reach [16]. If you compare it to not practicing OCSP at all instead, it constitutes significantly greater security risks for the end-users than if you use OCSP. In Section 5.1, it is also interesting to see that CRL is used very little or not at all by the three browsers. As the number of certificates increases, it is reasonable for browsers to use less of the CRL as a method to check revoked certificates when it constitutes such a large network overhead and that there are other more efficient methods now available. But as we see that CRL sometimes is practiced, it can in some cases be a method that can be good to use when these other better methods do not function, for example when a OCSP server is down.
While we see that OCSP and CRL are not used for many browsers in Firefox and even less in Chrome and Safari, it is worrying that so few web servers implement OCSP staple and must-staple, which can be seen in Section 5.3. This means that end-users cannot check whether a certificate has been revoked or not and is exposed to major security risks. In Table 5.3 we can see that from a large number of web servers it was not possible to find out the certificate information, which means that there may be more who implement OCSP staple and that there are web servers that implement OCSP must-staple. However, it is noteworthy that none of those who managed to retrieve certificate information offered OCSP must-staple as this is considered as one of the best methods to verify revoked certificates [20].
